Bayesian ANOVA

Info sheet · Statistics for Psychology & Neuroscience

Info sheet 0.2 (draft) · Prerequisites: one-way ANOVA; reading a Bayes factor · Give feedback ↗

Warning✎ Editing notes — to do / to check

Working notes for the author — not shown to students once collapsed; remove before publishing.

• [ ]

What you’ll get from this sheet

The Bayesian version of a one‑way ANOVA — and how painless it is. By the end you should be able to:

1. Run a Bayesian ANOVA with anovaBF().
2. Read the Bayes factor it returns.

anovaBF() compares a model with a group effect against an intercept‑only (grand‑mean) model, returning a Bayes factor. A large BF favours “the factor matters”; a BF near or below 1 favours the null. It answers the question you actually care about — how much should this data change my belief that the factor matters?

Same syntax, better question

The BayesFactor package makes Bayesian analysis remarkably painless — the syntax mirrors the frequentist functions you already know. Where a frequentist aov() hands you an F‑statistic and a p‑value, anovaBF() returns a Bayes factor comparing the model in which the factor matters against an intercept‑only model. If that Bayes factor is large, the data favour the model where (say) decade affects danceability; if it’s near or below 1, the data favour the null. Notice how directly this answers the researcher’s real question — how much should this dataset shift my belief that decade matters? — rather than the convoluted thing NHST answers.

Effect size drives the Bayes factor

Three decades of danceability. Slide how different the decades really are and watch the Bayes factor for “decade matters” grow:

md`**How different the group means are** — currently **${(+sep).toFixed(1)}**`

viewof sep = html`<input type="range" min="0" max="8" step="0.25" value="3" style="width:340px">`

banovaWidget = {
  const k=3, nper=15, N=k*nper; let s=4;
  const rnd=()=>{s|=0;s=s+0x6D2B79F5|0;let t=Math.imul(s^s>>>15,1|s);t=t+Math.imul(t^t>>>7,61|t)^t;return((t^t>>>14)>>>0)/4294967296;};
  const nrm=()=>{const u=1-rnd(),v=rnd();return Math.sqrt(-2*Math.log(u))*Math.cos(2*Math.PI*v);};
  const eff=[-1,0,1], y=[], g=[];
  for(let c=0;c<k;c++) for(let j=0;j<nper;j++){ y.push(60 + (+sep)*eff[c] + 6*nrm()); g.push(c); }
  const grand=y.reduce((a,b)=>a+b,0)/N;
  const gm=[0,0,0], gc=[0,0,0]; y.forEach((v,i)=>{gm[g[i]]+=v;gc[g[i]]++;}); gm.forEach((_,c)=>gm[c]/=gc[c]);
  let SStot=0,SSres=0; y.forEach((v,i)=>{SStot+=(v-grand)**2; SSres+=(v-gm[g[i]])**2;});
  // BIC approximation to the Bayes factor: BF10 ≈ exp((BIC_null - BIC_full)/2)
  const BICnull=N*Math.log(SStot/N)+1*Math.log(N);
  const BICfull=N*Math.log(SSres/N)+k*Math.log(N);
  const bf10=Math.exp((BICnull-BICfull)/2);
  const strength=(bf)=>bf<3?"anecdotal":bf<10?"moderate":bf<30?"strong":bf<100?"very strong":"extreme";
  const fav=bf10>=1, disp=fav?bf10:1/bf10, lab=strength(disp);
  // dotplot
  const W=460,H=210,pad=24, ymin=40,ymax=80;
  const colX=c=>pad+40+(c+0.5)*((W-pad-60)/k), Y=v=>H-30-(Math.max(ymin,Math.min(ymax,v))-ymin)/(ymax-ymin)*(H-60);
  let dots="",bars=""; y.forEach((v,i)=>{const jit=((i%nper)/(nper-1)-0.5)*40; dots+=`<circle cx="${colX(g[i])+jit}" cy="${Y(v)}" r="3" fill="rgba(31,78,121,0.5)"/>`;});
  for(let c=0;c<k;c++) bars+=`<line x1="${colX(c)-24}" y1="${Y(gm[c])}" x2="${colX(c)+24}" y2="${Y(gm[c])}" stroke="#C0501E" stroke-width="3"/><text x="${colX(c)}" y="${H-12}" text-anchor="middle" font-size="10" fill="#667">${['50s','80s','00s'][c]}</text>`;
  const fig=document.createElement("figure"); fig.style.margin="0";
  fig.innerHTML=`<svg viewBox="0 0 ${W} ${H}" width="${Math.min(W,520)}" style="background:#f8fafc;border:1px solid #e2e8f0;border-radius:8px">${dots}${bars}</svg>`+
    `<figcaption style="margin-top:.5rem">anovaBF → <b>BF₁₀ ≈ ${bf10<0.01?'<0.01':bf10>999?'>999':bf10.toFixed(2)}</b> → <b style="color:${fav?'#1E7A34':'#c0392b'}">${lab} evidence that ${fav?'decade matters':'decade does NOT matter'}</b>. (Illustrative BIC approximation; the BayesFactor package uses proper priors.)</figcaption>`;
  return fig;
}

With the decades on top of one another, the Bayes factor sits near 1 — the data don’t favour “decade matters” over the grand‑mean model. Pull the means apart and BF₁₀ climbs through moderate, strong, and beyond, quantifying exactly how much the data support an effect of decade. And unlike a p‑value, when the effect is genuinely absent the Bayes factor can drop below 1, giving you positive evidence for the null — a conclusion aov() could never license.

See it in code

library(BayesFactor)
# decade must be a factor:
bf <- anovaBF(danceability ~ decade, data = decadeDance)
bf              # BF10 for the 'decade' model vs the intercept-only (null) model

# compare against the frequentist version:
summary(aov(danceability ~ decade, data = decadeDance))   # F and p

TipCheck your understanding

anovaBF() returns BF₁₀ = 0.3 for a decade effect. How does that differ from a non‑significant aov()?

BF₁₀ = 0.3 means the data are about 3× more likely under the null (no decade effect) than under the model where decade matters — i.e. moderate evidence for the null. A non‑significant aov() (p > .05) only tells you that you failed to reject the null; it can’t distinguish “evidence for no effect” from “not enough data to tell.” The Bayesian result is the more informative: it actively supports the simpler model, rather than merely failing to overturn it.

The interactive uses a quick BIC approximation to the Bayes factor; the real anovaBF() places carefully chosen priors on the effects, so exact values differ (and depend on the prior — report it). Make sure your grouping variable is a factor, as with aov(). And a Bayesian ANOVA is still an omnibus comparison — “decade matters” doesn’t say which decades differ; for that, compare the specific models or run Bayesian follow‑up contrasts.

Where this shows up next

The same BayesFactor machinery extends to regression, where it can rank whole families of models. See Chapter (Bayesian Methods).

Related topics

• Reading a Bayes Factor — interpreting the number this returns.
• Bayesian Regression — the regression counterpart.
• One-Way ANOVA & the F-ratio — the frequentist version.
• NHST vs Bayesian Inference — why the Bayesian question is often the one you want.
• Post-Hoc Tests — following up an omnibus result.