The Assumptions of ANOVA

Info sheet · Statistics for Psychology & Neuroscience

Author

Andrew Bell

Published

August 13, 2026

Info sheet 0.2 (draft) · Prerequisites: one-way ANOVA; residuals; the normal distribution · Give feedback ↗

Working notes for the author — not shown to students once collapsed; remove before publishing.

  • [ ]

What you’ll get from this sheet

ANOVA is powerful, but it earns that power by making assumptions. By the end you should be able to:

  1. List the three assumptions of an independent ANOVA.
  2. Test each one, and know what to do if it’s violated.

An independent ANOVA assumes (1) independence of observations, (2) normality of the residuals (check with a Q–Q plot / Shapiro–Wilk), and (3) homogeneity of variance (check with Levene’s test). Violations have fixes: non‑parametric tests, transformations, or Welch’s F.

The three assumptions

ANOVA is often preferred over t‑tests and chi‑square because of its power and range — but that power rests on three assumptions:

  1. Independence of observations. Data points shouldn’t have a systematic relationship across groups — one group’s scores tell you nothing about another’s. (This is exactly what repeated‑measures designs violate on purpose, which is why they need their own model.) Independence is a matter of design, not a test you run afterwards.
  2. Normality of the residuals. As a parametric test, ANOVA assumes the residuals (each point minus its group mean) are normally distributed — not the raw data. Check it with a Q–Q plot (points on a straight line = normal) and the Shapiro–Wilk test (null = “normal”, so p > .05 means you’re fine).
  3. Homogeneity of variance. The groups should have roughly equal variances (homoscedasticity). Test with Levene’s (null = “equal variances”, so p > .05 is fine).

Read a Q–Q plot

Normality is the one you’ll eyeball most. A Q–Q plot sorts your residuals against the values a normal distribution would predict — on a line means normal; systematic bending means not. Pick a residual shape and see the tell:

Normal residuals track the green line. A right‑skew bends the points into a curve; heavy tails make the extreme points peel away at the ends. Either way the Shapiro–Wilk test would return p < .05, telling you normality is violated.

What to do about violations

  • Non‑normal residuals? Use a non‑parametric test (the Kruskal–Wallis test), transform the data (e.g. a log transform), or lean on the fact that ANOVA is fairly robust to mild non‑normality with decent sample sizes.
  • Unequal variances? Use Welch’s F‑test, which reweights each group’s contribution by its variance (it’s R’s default in oneway.test).
  • Non‑independence? That’s a design issue — model it properly with a repeated‑measures or mixed‑effects approach.

See it in code

from scipy import stats
# normality of residuals:
stats.shapiro(residuals)                 # null = normal (want p > .05)
# homogeneity of variance:
stats.levene(a, b, c)                    # null = equal variances
# Welch ANOVA (unequal variances): pingouin.welch_anova(...)
[~, tbl, stats] = anova1(y, group);
qqplot(stats.resid)          % Q-Q plot of residuals
vartestn(y, group)           % test equality of variances

The R tab runs live; Python and MATLAB are static references.

Your Shapiro–Wilk test on the residuals gives p = 0.002. What does that mean, and what are your options?

The null hypothesis of Shapiro–Wilk is that the residuals are normal, so p < .05 means you reject normality — the residuals are non‑normal. Your options: switch to a non‑parametric test (Kruskal–Wallis), transform the outcome (e.g. log) and re‑check, or — if the departure is mild and n is reasonable — proceed cautiously, since ANOVA is somewhat robust to normality violations. Don’t just ignore a severe violation.

It’s the residuals that must be normal, not the raw scores — a perfectly good ANOVA can have skewed‑looking group data. Beware the p‑value logic of the assumption tests: Shapiro–Wilk and Levene have “all is well” as the null, so a non‑significant result is what you want — and with a huge sample they’ll flag trivial, harmless departures, while with a tiny sample they miss real ones. Read the Q–Q plot alongside the test rather than trusting either alone.

Where this shows up next

If normality fails, the Kruskal–Wallis sheet is your non‑parametric fallback; if variances are unequal, Welch’s F. See Chapter (ANOVA) for the full assumptions workflow.