Factorial Repeated-Measures ANOVA

Info sheet · Statistics for Psychology & Neuroscience

Info sheet 0.2 (draft) · Prerequisites: one-way repeated-measures ANOVA; factorial ANOVA & interactions · Give feedback ↗

Warning✎ Editing notes — to do / to check

Working notes for the author — not shown to students once collapsed; remove before publishing.

• [ ]

What you’ll get from this sheet

So far we’ve only met repeated-measures designs with a single independent variable. But just like between-subjects designs, a repeated-measures study can have more than one IV — a factorial repeated-measures design. By the end you should be able to:

1. Describe, conceptually, how the total sum of squares is partitioned here.
2. Say what hypotheses this design tests.
3. Run a factorial repeated-measures ANOVA in R (and Python).

In a repeated-measures factorial ANOVA it’s the within-subjects variance that gets partitioned — into Factor A, Factor B, their interaction, and a residual. Individual differences are pulled out into a subject term, which shrinks the error and makes the test more powerful.

The Oreo example

To make it concrete, return to our (fake but realistic) Oreo taste-test. Previously we looked only at cookie type (8 varieties). Now we add a second factor: milk dipping (dipped vs not). Each of our 20 participants tastes all 16 cookies — 8 types, each with and without milk. Three questions follow:

• Do people rate different cookie types differently? (main effect of cookie type)
• Does dipping in milk change ratings? (main effect of milk)
• Does dipping matter more for some cookie types than others? (interaction)

That’s a two-way repeated-measures ANOVA, with rating (0–10) as the DV.

Partitioning the variability

Here’s the key contrast with a between-subjects design. In both, total variability splits into between-subjects and within-subjects. But:

• In an independent factorial ANOVA, we work with the between-subjects part and split the model variability into Factor A, Factor B, and A × B — leaving all the within-subjects noise in the residual.
• In a repeated-measures factorial ANOVA, we set aside the between-subjects part (that’s the individual differences) and partition the within-subjects variability into Factor A, Factor B, the interaction, and a residual.

That’s the whole trick: pulling individual differences out into their own term means they never land in the error against which effects are tested — which is why repeated-measures designs are so much more powerful.

Why pairing pays: watch the error term

Below, each line is one participant, rated without milk (left) and with milk (right). The milk effect and the noise are fixed. Drag how different people are from each other overall and watch the two F-tests for the milk effect: one that ignores the pairing (between-subjects) and one that uses it (repeated-measures).

md`**How much participants differ from each other overall** — currently **${(+subjSD).toFixed(1)}**`

viewof subjSD = html`<input type="range" min="0" max="8" step="0.25" value="4" style="width:340px">`

rmWidget = {
  const N=10, base=5.5, milk=1.2, noiseSD=0.7;
  let s=5;
  const rand=()=>{s|=0;s=s+0x6D2B79F5|0;let t=Math.imul(s^s>>>15,1|s);t=t+Math.imul(t^t>>>7,61|t)^t;return((t^t>>>14)>>>0)/4294967296;};
  const nrm=()=>{const u=1-rand(),v=rand();return Math.sqrt(-2*Math.log(u))*Math.cos(2*Math.PI*v);};
  const B=[],n0=[],n1=[]; for(let i=0;i<N;i++){B.push(nrm());n0.push(nrm());n1.push(nrm());}
  const sd=+subjSD;
  const x0=[],x1=[];
  for(let i=0;i<N;i++){ x0.push(base + sd*B[i] + noiseSD*n0[i]); x1.push(base + sd*B[i] + milk + noiseSD*n1[i]); }

  const all=[...x0,...x1], grand=all.reduce((a,b)=>a+b,0)/(2*N);
  const m0=x0.reduce((a,b)=>a+b,0)/N, m1=x1.reduce((a,b)=>a+b,0)/N;
  const SS_cond = N*((m0-grand)**2 + (m1-grand)**2);
  // between-subjects error (pairing ignored)
  let SS_err_b=0; x0.forEach(v=>SS_err_b+=(v-m0)**2); x1.forEach(v=>SS_err_b+=(v-m1)**2);
  const F_between = (SS_cond/1)/(SS_err_b/(2*N-2));
  // repeated-measures: pull out subject term
  let SS_subj=0; for(let i=0;i<N;i++){const sm=(x0[i]+x1[i])/2; SS_subj+=2*(sm-grand)**2;}
  let SS_tot=0; all.forEach(v=>SS_tot+=(v-grand)**2);
  const SS_err_rm = SS_tot - SS_cond - SS_subj;
  const F_rm = (SS_cond/1)/(SS_err_rm/(N-1));
  const Fb=5.32, Fr=5.12; // rough critical F at .05 for df (1,18) and (1,9)

  const W=440,H=300,padT=16,padB=34,padL=40,padR=16;
  const ymin=0, ymax=12;
  const Y=y=>H-padB-(y-ymin)/(ymax-ymin)*(H-padT-padB);
  const xL=padL+40, xR=W-padR-70;
  let lines="",dots="";
  for(let i=0;i<N;i++){
    lines+=`<line x1="${xL}" y1="${Y(x0[i])}" x2="${xR}" y2="${Y(x1[i])}" stroke="rgba(31,78,121,0.45)" stroke-width="1.5"/>`;
    dots +=`<circle cx="${xL}" cy="${Y(x0[i])}" r="3.5" fill="#1F4E79"/><circle cx="${xR}" cy="${Y(x1[i])}" r="3.5" fill="#1F4E79"/>`;
  }
  const meanBars=`<line x1="${xL-14}" y1="${Y(m0)}" x2="${xL+14}" y2="${Y(m0)}" stroke="#C0501E" stroke-width="3"/>`+
    `<line x1="${xR-14}" y1="${Y(m1)}" x2="${xR+14}" y2="${Y(m1)}" stroke="#C0501E" stroke-width="3"/>`;
  const axes=`<line x1="${padL}" y1="${padT}" x2="${padL}" y2="${H-padB}" stroke="#94a3b8"/>`+
    `<line x1="${padL}" y1="${H-padB}" x2="${W-padR}" y2="${H-padB}" stroke="#94a3b8"/>`+
    `<text x="${xL}" y="${H-12}" text-anchor="middle" font-size="12" fill="#334">no milk</text>`+
    `<text x="${xR}" y="${H-12}" text-anchor="middle" font-size="12" fill="#334">milk</text>`+
    `<text x="12" y="${(padT+H-padB)/2}" transform="rotate(-90 12 ${(padT+H-padB)/2})" text-anchor="middle" font-size="11" fill="#667">rating</text>`;

  const fig=document.createElement("figure"); fig.style.margin="0";
  fig.innerHTML=
    `<svg viewBox="0 0 ${W} ${H}" width="${Math.min(W,520)}" style="background:#f8fafc;border:1px solid #e2e8f0;border-radius:8px">${axes}${lines}${meanBars}${dots}</svg>`+
    `<figcaption style="margin-top:.5rem">Same milk effect, tested two ways:<br>`+
    `ignoring the pairing → <b style="color:#c0392b">F(1,${2*N-2}) = ${F_between.toFixed(2)}</b> · `+
    `repeated-measures → <b style="color:#1E7A34">F(1,${N-1}) = ${F_rm.toFixed(2)}</b>. `+
    `${sd>3?'Big individual differences swamp the between-subjects test but leave the repeated-measures test untouched.':'With people alike, the two tests roughly agree.'}</figcaption>`;
  return fig;
}

The lines stay roughly parallel — everyone gets a similar bump from milk — but as you raise the individual differences the whole bundle spreads vertically. The between-subjects test drowns: that vertical spread is individual differences, and it’s been dumped straight into its error term, so its F collapses. The repeated-measures test barely flinches, because it looks only at each person’s own change from left to right — the baseline differences are siphoned off into the subject term. Same effect, same noise, wildly different power. Now imagine that same benefit applied to two within-subjects factors and their interaction: that’s the factorial repeated-measures ANOVA.

Running the analysis

The syntax is nearly identical to the one-way version — you just hand it a vector of factors for the within-subjects part. The interaction term is added automatically.

R

Python

R (in practice: rstatix)

# The tidy way — automatically Type III, with effect sizes:
library(rstatix)
anova_test(
  data   = OreoRatingsFactorial,
  dv     = rating,
  wid    = participant,
  within = c(cookie_type, milk)
)
# ezANOVA() (ez) and aov_ez() (afex) do the same job.

The aov() and Python tabs run live; rstatix/ez/afex are the tools you’d reach for in a real analysis.

TipCheck your understanding

Why does the repeated-measures F stay high even when participants differ a lot from one another overall?

Because a repeated-measures ANOVA pulls each person’s overall level out into a subject term, so individual differences never enter the error the effect is tested against. It compares each participant to themselves across conditions. A between-subjects test can’t do that — those baseline differences land in its residual and inflate the error, shrinking F.

More within-subjects levels bring sphericity back into play: for any factor with more than two levels, check Mauchly’s test and apply a Greenhouse–Geisser / Huynh–Feldt correction if it’s violated (see the Sphericity sheet). Also watch missing data — most functions drop a participant entirely if any of their cells is missing, quietly costing you power.

Where this shows up next

See the Repeated-Measures ANOVA and Sphericity & Mauchly’s Test sheets for the one-factor case and the correction machinery, and Chapter (Repeated Measures) for the full partitioning and worked Oreo analysis.