Multiple Regression

Info sheet · Statistics for Psychology & Neuroscience

Author

Andrew Bell

Published

August 13, 2026

Info sheet 0.2 (draft) · Prerequisites: simple regression (OLS); correlation · Give feedback ↗

✎ Editing notes — to do / to check

Working notes for the author — not shown to students once collapsed; remove before publishing.

What you’ll get from this sheet

Regression with more than one predictor — and the crucial idea of what a coefficient means once other variables are in the model. By the end you should be able to:

Read the multiple‑regression equation.
Interpret a coefficient as an effect holding the other predictors constant.

Multiple regression adds predictors: \(Y = b_0 + b_1X_1 + b_2X_2 + \dots + \varepsilon\). Each coefficient is that predictor’s effect holding the others constant — its unique contribution, which can differ sharply from its effect considered alone when predictors are correlated.

More than one predictor

So far we’ve had one predictor \(X\) and one outcome \(Y\). But most outcomes depend on many things at once. Multiple regression extends OLS to two or more predictors, with the same structure:

\[Y = b_0 + b_1X_1 + b_2X_2 + \dots + b_kX_k + \varepsilon\]

where the \(b\)’s are the intercept and slopes and \(\varepsilon\) is the error. Geometrically, one predictor fits a line; two predictors fit a plane; more fit a hyperplane you can’t draw. Take London house prices predicted from square footage and number of bedrooms: lm(price ~ area + bedrooms).

The key phrase is in the interpretation: each coefficient is the effect of that predictor while holding the others constant. That’s what makes multiple regression more than a pile of simple regressions — and it’s why a predictor’s coefficient can change dramatically once you add a correlated variable alongside it.

“Holding constant”, made visible

Area and bedrooms are correlated (bigger homes have more bedrooms). Suppose the true driver of price is area, with bedrooms adding only a little. Slide the correlation between area and bedrooms and compare the bedrooms coefficient two ways: alone (price ~ bedrooms) versus controlling for area (price ~ area + bedrooms).

md`**Correlation between area and bedrooms** — currently **${(+rho).toFixed(2)}**`

viewof rho = html`<input type="range" min="0" max="0.95" step="0.05" value="0.7" style="width:340px">`

mrWidget = {
  const n=80; let s=3;
  const rand=()=>{s|=0;s=s+0x6D2B79F5|0;let t=Math.imul(s^s>>>15,1|s);t=t+Math.imul(t^t>>>7,61|t)^t;return((t^t>>>14)>>>0)/4294967296;};
  const nrm=()=>{const u=1-rand(),v=rand();return Math.sqrt(-2*Math.log(u))*Math.cos(2*Math.PI*v);};
  const r=+rho;
  const area=[],bed=[],price=[];
  for(let i=0;i<n;i++){const a=nrm(); const b=r*a+Math.sqrt(1-r*r)*nrm(); area.push(a); bed.push(b); price.push(100+50*a+5*b+12*nrm());}
  const mean=z=>z.reduce((x,y)=>x+y,0)/n;
  const ma=mean(area),mb=mean(bed),mp=mean(price);
  let Saa=0,Sbb=0,Sab=0,Sya=0,Syb=0; for(let i=0;i<n;i++){Saa+=(area[i]-ma)**2;Sbb+=(bed[i]-mb)**2;Sab+=(area[i]-ma)*(bed[i]-mb);Sya+=(price[i]-mp)*(area[i]-ma);Syb+=(price[i]-mp)*(bed[i]-mb);}
  const simpleBed=Syb/Sbb;
  const partialBed=(Syb*Saa - Sya*Sab)/(Sbb*Saa - Sab*Sab);

  const W=460,H=150, bx=210, bw=W-bx-16, x0=bx;
  const scale=bw/40;  // coefficient axis 0..40
  const bar=(y,val,col,lab)=>`<text x="8" y="${y+4}" font-size="12" fill="#334">${lab}</text>`+
    `<rect x="${x0}" y="${y-11}" width="${Math.max(0,val)*scale}" height="18" fill="${col}"/>`+
    `<text x="${x0+Math.max(0,val)*scale+6}" y="${y+4}" font-size="12" fill="${col}">${val.toFixed(1)}</text>`;
  const truth=`<line x1="${x0+5*scale}" y1="20" x2="${x0+5*scale}" y2="${H-16}" stroke="#1E7A34" stroke-dasharray="4 3"/>`+
    `<text x="${x0+5*scale}" y="14" text-anchor="middle" font-size="10" fill="#1E7A34">true = 5</text>`;
  const fig=document.createElement("figure"); fig.style.margin="0";
  fig.innerHTML=`<svg viewBox="0 0 ${W} ${H}" width="${Math.min(W,520)}" style="background:#f8fafc;border:1px solid #e2e8f0;border-radius:8px">`+
    truth+bar(56,simpleBed,"#C0501E","bedrooms alone")+bar(100,partialBed,"#1F4E79","+ area (holding constant)")+`</svg>`+
    `<figcaption style="margin-top:.5rem">The bedroom effect is really about <b>5</b>. On its own it looks like <b style="color:#C0501E">${simpleBed.toFixed(1)}</b> — it borrows credit from area, which it's correlated with. Add area to the model and its <b>partial</b> effect drops to <b style="color:#1F4E79">${partialBed.toFixed(1)}</b>, near the truth. That's "holding area constant."</figcaption>`;
  return fig;
}

At zero correlation the two coefficients agree — with nothing shared, bedrooms means the same thing alone or alongside area. Raise the correlation and they split apart: on its own, bedrooms looks like a powerful predictor because it’s standing in for area; controlling for area, its unique effect shrinks back to the truth. The multiple‑regression coefficient asks a sharper question — what does bedrooms add once area is already accounted for?

See it in code

T = table(area, bedrooms, price);
fitlm(T, 'price ~ area + bedrooms')   % each coefficient holds the other constant

The R and Python tabs run live; MATLAB is a static reference.

Check your understanding

In price ~ area + bedrooms the bedrooms coefficient is about 5, but price ~ bedrooms alone gives 30. Why the difference?

Because area and bedrooms are correlated and area is the real driver of price. Considered alone, bedrooms soaks up area’s effect (they move together), so its slope is inflated. In the multiple regression, area is already in the model, so the bedrooms coefficient reflects only what bedrooms adds holding area constant — its unique contribution, ≈ 5. Same variable, different question.

A coefficient is “holding others constant” only for the variables actually in the model — leave out a confounder and the remaining coefficients quietly absorb its effect (omitted‑variable bias). When predictors are strongly correlated (multicollinearity) the individual coefficients become unstable and hard to interpret — worth checking with VIF (next sheet). And you can’t compare raw coefficient magnitudes across predictors measured on different scales; standardise them first if you want to rank importance.

Where this shows up next

Next: the assumptions these models rest on and how to check them, then how to choose which predictors to include. See Chapter (Correlation & Regression).

What you’ll get from this sheet

More than one predictor

“Holding constant”, made visible

See it in code

Where this shows up next

Related topics