Repeated-Measures ANOVA

Info sheet · Statistics for Psychology & Neuroscience

Info sheet 0.2 (draft) · Prerequisites: one-way ANOVA & the F-ratio; the paired t-test · Give feedback ↗

Warning✎ Editing notes — to do / to check

Working notes for the author — not shown to students once collapsed; remove before publishing.

• [ ]

What you’ll get from this sheet

This is our first step into a new form of ANOVA: the repeated-measures ANOVA. Here we build the theory; the next sheet covers running one in practice. By the end you should be able to:

1. Say what a repeated-measures ANOVA is, and how it differs from the independent-samples ANOVAs we’ve used so far.
2. Recognise the role of the subject sum of squares (no need to calculate it by hand).
3. Understand how the error term is partitioned differently in a repeated-measures design.

A repeated-measures ANOVA is the ANOVA equivalent of a paired t-test. It carves the participants’ own individual differences out into a subject sum of squares, so those differences no longer sit in the error term — leaving a smaller residual and a more sensitive test.

The paired t-test, scaled up

Everything so far has been independent-samples ANOVA — one-way, then factorial, with their assumptions (independence, normal residuals, homogeneity of variance) and their post-hoc tests and contrasts. In all of those, factors had independent levels: a song belongs to only one decade; a participant reports only one gender. There was no shared variance between levels.

Compare that to t-tests. An independent-samples t-test contrasts two unrelated groups (heights of men vs women) — no shared variance. A paired-samples t-test contrasts two scores from the same people (pre vs post treatment) — and those scores share variance, because they come from the same individuals. So the natural question: is there an ANOVA equivalent of the paired t-test? Yes — the repeated-measures ANOVA.

It’s used when the same participants are measured under multiple conditions, so their scores aren’t independent — most often a within-subjects design where each participant provides several data points. It’s a little old-fashioned now next to linear mixed models, but still widely used in psychology and clinical research, and essential for reading the pre-2015 literature.

The Oreo taste test

A simple example: 20 participants each rate 8 Oreo flavours from 0 (“terrible”) to 10 (“amazing”). The IV is cookie type (8 levels); the DV is rating. Because each participant rated every cookie, we have repeated measures — the same individuals contribute multiple scores, which share variance. Note the data are balanced: everyone rated all 8. That matters, because repeated-measures ANOVA handles missing data badly — if some people rated only 7 cookies, the model breaks down. (That fragility is one reason it’s given ground to mixed models.)

Where does the subject variance go?

In an independent ANOVA, total variability splits into just two pieces: the bit explained by the factor, and everything left over (the residual). In a repeated-measures ANOVA we pull out a third piece — the subject sum of squares, the variance due to some participants simply being more generous raters than others. Drag the individual differences below and watch where that variance lands in each design.

md`**Individual differences (how much raters differ overall)** — currently **${(+subjSD).toFixed(1)}**`

viewof subjSD = html`<input type="range" min="0" max="6" step="0.25" value="3" style="width:340px">`

partWidget = {
  const N=8, k=4;                       // 8 participants, 4 cookie types
  const condEff=[-1.5,-0.5,0.5,1.5], noiseSD=0.7;
  let s=3;
  const rand=()=>{s|=0;s=s+0x6D2B79F5|0;let t=Math.imul(s^s>>>15,1|s);t=t+Math.imul(t^t>>>7,61|t)^t;return((t^t>>>14)>>>0)/4294967296;};
  const nrm=()=>{const u=1-rand(),v=rand();return Math.sqrt(-2*Math.log(u))*Math.cos(2*Math.PI*v);};
  const subj=[]; for(let i=0;i<N;i++) subj.push(nrm());
  const sd=+subjSD, val=[], cond=[], who=[];
  for(let i=0;i<N;i++) for(let c=0;c<k;c++){ val.push(6+condEff[c]+sd*subj[i]+noiseSD*nrm()); cond.push(c); who.push(i); }
  const M=val.length, grand=val.reduce((a,b)=>a+b,0)/M;
  const cMean=Array(k).fill(0), cN=Array(k).fill(0); val.forEach((v,i)=>{cMean[cond[i]]+=v;cN[cond[i]]++;}); cMean.forEach((_,c)=>cMean[c]/=cN[c]);
  const sMean=Array(N).fill(0); val.forEach((v,i)=>sMean[who[i]]+=v); sMean.forEach((_,i)=>sMean[i]/=k);
  let SS_cond=0; for(let c=0;c<k;c++) SS_cond+=cN[c]*(cMean[c]-grand)**2;
  let SS_subj=0; for(let i=0;i<N;i++) SS_subj+=k*(sMean[i]-grand)**2;
  let SS_tot=0; val.forEach(v=>SS_tot+=(v-grand)**2);
  const SS_res=SS_tot-SS_cond-SS_subj;
  const F_ind=(SS_cond/(k-1))/((SS_subj+SS_res)/(M-k));
  const F_rm =(SS_cond/(k-1))/(SS_res/((N-1)*(k-1)));

  const W=460,H=180,padL=12,padR=12;
  const bx=110, bw=W-bx-padR, y1=44, y2=118, bh=34;
  const draw=(y,parts)=>{ let x=bx,out=""; parts.forEach(([frac,col,lab])=>{ const w=bw*frac; out+=`<rect x="${x}" y="${y}" width="${w}" height="${bh}" fill="${col}"/>`; if(w>34) out+=`<text x="${x+w/2}" y="${y+bh/2+4}" text-anchor="middle" font-size="10" fill="#fff">${lab}</text>`; x+=w; }); return out; };
  const indParts=[[SS_cond/SS_tot,"#1F4E79","Cookie"],[(SS_subj+SS_res)/SS_tot,"#94a3b8","error (incl. subjects)"]];
  const rmParts=[[SS_cond/SS_tot,"#1F4E79","Cookie"],[SS_subj/SS_tot,"#1E7A34","Subject SS"],[SS_res/SS_tot,"#94a3b8","resid"]];

  const fig=document.createElement("figure"); fig.style.margin="0";
  fig.innerHTML=
    `<svg viewBox="0 0 ${W} ${H}" width="${Math.min(W,540)}" style="background:#f8fafc;border:1px solid #e2e8f0;border-radius:8px">`+
      `<text x="8" y="${y1-8}" font-size="11" fill="#334">Independent</text>${draw(y1,indParts)}`+
      `<text x="8" y="${y2-8}" font-size="11" fill="#334">Repeated-meas.</text>${draw(y2,rmParts)}`+
    `</svg>`+
    `<figcaption style="margin-top:.5rem">Same cookie effect, two error terms:<br>`+
    `independent <b style="color:#c0392b">F(${k-1},${M-k}) = ${F_ind.toFixed(2)}</b> vs repeated-measures <b style="color:#1E7A34">F(${k-1},${(N-1)*(k-1)}) = ${F_rm.toFixed(2)}</b>. `+
    `The <span style="color:#1E7A34">green</span> Subject SS is variance the repeated-measures design removes from the error.</figcaption>`;
  return fig;
}

In the independent design the subject variance has nowhere to go — it’s swallowed into the grey error bar, and as individual differences grow that bar swells and the F sinks. In the repeated-measures design the same variance is split off into its own green Subject SS, leaving a slim residual — so the cookie effect is tested against noise alone, and its F holds up. That green block is the subject sum of squares, and setting it aside is the whole point.

Running it in R

The mechanics resemble aov(), with one twist: you specify an error term for subjects.

R

Python

MATLAB / Octave

% wide format: rows = subjects, columns = the 4 cookie conditions
t   = array2table(ratings, 'VariableNames', {'Original','DoubleStuf','RedVelvet','Mint'});
rm  = fitrm(t, 'Original-Mint ~ 1');
ranova(rm)      % repeated-measures ANOVA table

The aov() and Python tabs run live; note R’s Error(subject/cookie) — that term is the subject SS.

The main effect of cookie type here has 7 degrees of freedom in the full 8-cookie study (8 levels − 1); a large F with a small p would say ratings genuinely differ across flavours.

TipCheck your understanding

What is the subject sum of squares, and why does isolating it make the test more powerful?

It’s the variance due to participants as individuals — some people rate everything high, some everything low. In an independent ANOVA that variance is stuck in the error term, inflating it. A repeated-measures ANOVA splits it off into its own term, so the factor is tested against a smaller residual (just the leftover noise). Smaller error → larger F → more power. That’s the same edge a paired t-test has over an independent one.

Repeated-measures ANOVA needs complete, balanced data: every participant must have a score in every condition, or most functions drop that participant entirely. It also assumes sphericity once a factor has more than two levels (see Sphericity & Mauchly’s Test). Both problems are handled more gracefully by linear mixed models — worth knowing when your design gets messy.

Where this shows up next

See Chapter (Repeated Measures) for the full partitioning and worked Oreo analysis, and the next sheet for actually running these in R.

Related topics

• Repeated Measures in R — the practical follow-up: aov() vs ez/afex, and the four types of sums of squares.
• Factorial Repeated-Measures ANOVA — two within-subjects factors and their interaction.
• Sphericity & Mauchly’s Test — the assumption that bites once a factor has 3+ levels.
• One-Way ANOVA & the F-ratio — the independent-samples cousin this builds on.
• Mixed Models vs ANOVA/Regression — the modern alternative that handles missing and nested data.