Random Intercepts & Slopes

Info sheet · Statistics for Psychology & Neuroscience

Info sheet 0.2 (draft) · Prerequisites: fixed vs random effects; simple regression (intercept & slope) · Give feedback ↗

Warning✎ Editing notes — to do / to check

Working notes for the author — not shown to students once collapsed; remove before publishing.

• [ ]

What you’ll get from this sheet

The two ways a random effect can act. By the end you should be able to:

1. Explain what random intercepts and random slopes are.
2. Read the equation for a model with intercepts, slopes, or both.
3. Recognise when random effects add realism.

A random intercept lets groups start at different baselines; a random slope lets groups change at different rates. You can have either, or both — the most realistic (and most complex) option.

The problem with one line

A general linear model predicts an outcome as a straight line: \(y_i = \beta_0 + \beta_1 x_i + \varepsilon_i\), with \(\beta_0\) the intercept (baseline) and \(\beta_1\) the slope (rate of change). Take the London housing model — predict price from area, bedrooms, and house type — where everything was a fixed effect. That forces one slope for everybody: the effect of area on price was pinned at a single value across all house types. But in reality flats usually cost far more per square foot than houses. One line can’t say that. Random effects can — by letting a group’s intercept and/or slope vary.

Three ways to let lines vary

Same houses, coloured by type (flat, house, new development). Switch the random-effects structure and watch the fitted lines change.

md`**Random-effects structure**`

viewof model = html`<select style="font-size:15px;padding:4px 8px">
  <option value="pooled">Single line — everything fixed (one line for all)</option>
  <option value="rint" selected>Random intercept — parallel lines, different baselines</option>
  <option value="rslope">Random slope — same start, different rates</option>
  <option value="both">Random intercept + slope — fully separate lines</option>
</select>`

riWidget = {
  // three groups: [intercept, slope, colour]
  const groups = [
    {name:"flat",    b0:8, b1:0.95, col:"#1F4E79"},
    {name:"house",   b0:2, b1:0.35, col:"#1E7A34"},
    {name:"new dev", b0:5, b1:0.62, col:"#C0501E"},
  ];
  let s=4;
  const rand=()=>{s|=0;s=s+0x6D2B79F5|0;let t=Math.imul(s^s>>>15,1|s);t=t+Math.imul(t^t>>>7,61|t)^t;return((t^t>>>14)>>>0)/4294967296;};
  const nrm=()=>{const u=1-rand(),v=rand();return Math.sqrt(-2*Math.log(u))*Math.cos(2*Math.PI*v);};
  const pts=[];
  groups.forEach((g,gi)=>{ for(let i=0;i<10;i++){ const x=2+i*1.0+rand()*0.4; const y=g.b0+g.b1*x+nrm()*0.8; pts.push({x,y,gi}); } });

  // OLS helpers
  const olsAll=(P)=>{const n=P.length,mx=P.reduce((a,p)=>a+p.x,0)/n,my=P.reduce((a,p)=>a+p.y,0)/n;let sxy=0,sxx=0;P.forEach(p=>{sxy+=(p.x-mx)*(p.y-my);sxx+=(p.x-mx)**2;});const b1=sxy/sxx;return{b1,b0:my-b1*mx};};
  const pooled=olsAll(pts);
  const perGroup=groups.map((g,gi)=>olsAll(pts.filter(p=>p.gi===gi)));
  const grpMean=groups.map((g,gi)=>{const P=pts.filter(p=>p.gi===gi);return{mx:P.reduce((a,p)=>a+p.x,0)/P.length,my:P.reduce((a,p)=>a+p.y,0)/P.length};});

  // build the line(s) for the chosen structure
  let lines=[]; // {b0,b1,col}
  if (model==="pooled") lines=[{b0:pooled.b0,b1:pooled.b1,col:"#334"}];
  else if (model==="rint") lines=groups.map((g,gi)=>({b1:pooled.b1, b0:grpMean[gi].my - pooled.b1*grpMean[gi].mx, col:g.col}));
  else if (model==="rslope") { const b0=pooled.b0; lines=groups.map((g,gi)=>{const P=pts.filter(p=>p.gi===gi);let num=0,den=0;P.forEach(p=>{num+=p.x*(p.y-b0);den+=p.x*p.x;});return{b0,b1:num/den,col:g.col};}); }
  else lines=groups.map((g,gi)=>({b0:perGroup[gi].b0,b1:perGroup[gi].b1,col:g.col}));

  const W=460,H=320,pad=34;
  const xmin=0,xmax=14,ymin=0,ymax=24;
  const X=x=>pad+(x-xmin)/(xmax-xmin)*(W-2*pad);
  const Y=y=>H-pad-(y-ymin)/(ymax-ymin)*(H-2*pad);
  let dots=""; pts.forEach(p=>dots+=`<circle cx="${X(p.x)}" cy="${Y(p.y)}" r="3.5" fill="${groups[p.gi].col}" fill-opacity="0.75"/>`);
  let ln=""; lines.forEach(l=>ln+=`<line x1="${X(xmin)}" y1="${Y(l.b0+l.b1*xmin)}" x2="${X(xmax)}" y2="${Y(l.b0+l.b1*xmax)}" stroke="${l.col}" stroke-width="2.5"/>`);
  const axes=`<line x1="${pad}" y1="${H-pad}" x2="${W-pad}" y2="${H-pad}" stroke="#94a3b8"/><line x1="${pad}" y1="${pad}" x2="${pad}" y2="${H-pad}" stroke="#94a3b8"/>`+
    `<text x="${W/2}" y="${H-8}" text-anchor="middle" font-size="12" fill="#667">area (sq ft) →</text>`+
    `<text x="12" y="${H/2}" transform="rotate(-90 12 ${H/2})" text-anchor="middle" font-size="12" fill="#667">price →</text>`;
  const legend=groups.map((g,i)=>`<circle cx="${W-pad-92}" cy="${pad+6+i*16}" r="4" fill="${g.col}"/><text x="${W-pad-84}" y="${pad+10+i*16}" font-size="11" fill="#334">${g.name}</text>`).join("");
  const cap={pooled:"One slope, one intercept — the effect of area is forced to be identical for every house type.",
             rint:"Parallel lines: the same slope, but each type starts at its own baseline price.",
             rslope:"Lines fan from a shared baseline: each type gains value at its own rate per sq ft.",
             both:"Separate lines: types differ in both baseline and rate — the most realistic picture."}[model];

  const fig=document.createElement("figure"); fig.style.margin="0";
  fig.innerHTML=`<svg viewBox="0 0 ${W} ${H}" width="${Math.min(W,520)}" style="background:#f8fafc;border:1px solid #e2e8f0;border-radius:8px">${axes}${ln}${dots}${legend}</svg>`+
    `<figcaption style="margin-top:.5rem">${cap}</figcaption>`;
  return fig;
}

• Single line (all fixed). One intercept, one slope — the flats are systematically above the line and the houses below it, because a single line can’t reflect that they differ. This is ordinary regression.
• Random intercept. \(y_{ij} = (\beta_0 + u_{0j}) + \beta_1 x_{ij} + \varepsilon_{ij}\) — the intercept varies by group \(j\), slopes stay equal. Three parallel lines: same rate, different baselines.
• Random slope. \(y_{ij} = \beta_0 + (\beta_1 + u_{1j}) x_{ij} + \varepsilon_{ij}\) — one baseline, slopes differ. Lines fan out from a shared start.
• Random intercept + slope. \(y_{ij} = (\beta_0 + u_{0j}) + (\beta_1 + u_{1j}) x_{ij} + \varepsilon_{ij}\) — both vary. Separate lines per group — the fullest, most realistic model.

The \(u_{0j}\) and \(u_{1j}\) terms are the random deviations for group \(j\): how far its intercept and slope stray from the overall \(\beta_0\) and \(\beta_1\). The model doesn’t estimate each one freely — it assumes they’re drawn from a normal distribution and estimates its variance, which is what lets the effect generalise to groups you didn’t observe.

See it in code

In lme4, the bit in parentheses is the random effect: (1 | group) is a random intercept, (0 + x | group) a random slope, (1 + x | group) both.

R (lme4)

library(lme4)
lmer(price ~ area + (1 | type),        data = london)  # random intercept
lmer(price ~ area + (0 + area | type), data = london)  # random slope
lmer(price ~ area + (1 + area | type), data = london)  # both (intercept + slope)

Python (statsmodels)

import statsmodels.formula.api as smf
smf.mixedlm("price ~ area", london, groups=london["type"]).fit()                       # random intercept
smf.mixedlm("price ~ area", london, groups=london["type"],
            re_formula="~area").fit()                                                  # intercept + slope

Static reference — lme4/statsmodels aren’t in the in-page runtime.

TipCheck your understanding

Flats cost more per square foot than houses, and they start at a higher base price. Which random effect(s) do you need?

Both. The “more per square foot” part is a random slope — the effect of area varies by house type. The “higher base price” part is a random intercept — the baseline varies by type. So you’d fit price ~ area + (1 + area | type), letting each type have its own intercept and its own slope.

A “maximal” random structure (intercepts + slopes + their correlation for every grouping) is often not estimable from the data you have — the model fails to converge or reports a singular fit. Random slopes are especially data-hungry: you need a decent number of observations per group. When a model won’t converge, simplify — drop the slope, or the intercept–slope correlation — rather than forcing it.

Where this shows up next

See Chapter (Mixed-Effects Models) for the full derivation and the London housing worked example. Next up: actually building and evaluating one of these models in R.

Related topics

• Fixed vs Random Effects — the distinction these structures rest on.
• Building & Evaluating a Linear Mixed Model — fitting and comparing these in R.
• Mixed Models vs ANOVA/Regression — why this beats one fixed line.
• Ordinary Least Squares: Finding the Best Line — the single-line starting point.
• LMM Assumptions & Write-up — checking and reporting the fitted model.