Regression with Categorical Predictors (Dummy Coding)
Info sheet · Statistics for Psychology & Neuroscience
Warning✎ Editing notes — to do / to check
Working notes for the author — not shown to students once collapsed; remove before publishing.
- [ ]
What you’ll get from this sheet
How regression swallows categorical predictors — the trick that makes it a true generalisation of ANOVA. By the end you should be able to:
- Turn a categorical predictor into dummy (0/1) variables.
- Interpret the intercept and coefficients against a reference level.
A categorical predictor with k levels becomes k − 1 dummy (0/1) variables; the left‑out level is the reference (baseline). In the fitted regression, the intercept is the reference group’s mean, and each dummy’s coefficient is that group’s mean minus the reference. This is why regression can do everything ANOVA can — and more.
Why we need it
So far regression has looked more flexible than ANOVA — it handles continuous predictors like danceability. But it insists on numbers, so we can’t just drop the labels “Action”, “Drama”, “Comedy” into the equation. The fix is dummy‑variable encoding: create a binary 0/1 variable for each category except one baseline, so each dummy flags whether an observation belongs to its category.
The simplest case has just two categories. Remember the fruit example — Apple vs Watermelon predicting diameter, with \(x = 0\) for Apple and \(x = 1\) for Watermelon:
\[\text{Diameter} = 7.5 + 8x + \varepsilon\]
Apple (\(x = 0\)) → predicted 7.5; Watermelon (\(x = 1\)) → predicted 15.5. Two categories need only one dummy. The intercept (7.5) is the Apple mean; the coefficient (8) is how much bigger Watermelon is.
Pick the reference and watch the coefficients
Three Netflix genres and their mean Rotten Tomatoes scores. Choose which genre is the reference and see the dummy table and the regression coefficients update. Notice the intercept is always the reference group’s mean, and every coefficient is a difference from it.
The reference is arbitrary — it just sets the zero point. Switch it and the coefficients re‑express themselves (“Drama is 12 below Action” becomes “Action is 12 above Drama”), but the predicted group means never move. Choosing a sensible reference (a control group, a gold standard) just makes the coefficients read the way you want.
See it in code
R dummy‑codes categorical predictors for you the moment you make them a factor — the “hard way” (building 0/1 columns by hand with fastDummies) gives the identical result:
import pandas as pd, statsmodels.formula.api as smf
df = pd.DataFrame({"genre": genre, "score": score})
# Treatment('Action') sets the reference level explicitly
smf.ols("score ~ C(genre, Treatment('Action'))", df).fit().paramsgenre = categorical(genre);
fitlm(table(genre, score)) % dummy-codes automatically; first category = referenceThe R tab runs live; Python and MATLAB are static references.
TipCheck your understanding
You have 4 genres. How many dummy variables do you need, and what does the intercept represent?
You need k − 1 = 3 dummy variables (one per genre except the reference). The intercept is the mean of the omitted reference genre, and each of the three coefficients is a genre’s mean minus the reference mean. Predicting a genre’s score = intercept + that genre’s dummy coefficient.
Don’t include all k dummies and an intercept — that’s the “dummy‑variable trap”: the columns become perfectly collinear, \(X^{\mathsf T}X\) is singular, and the model can’t be fit (R silently drops one for you). Also, R’s default reference is the alphabetically first level, so your coefficients may be comparing against a baseline you didn’t intend — set it deliberately with relevel() or levels =. And remember the coefficients are differences from the reference, not absolute group means.
Where this shows up next
This is the bridge from ANOVA to regression: with dummy coding, anything ANOVA can do, regression can do — plus mixed continuous‑and‑categorical predictors. Next: multiple regression. See Chapter (Correlation & Regression).