Bayesian Regression
Info sheet · Statistics for Psychology & Neuroscience
Warning✎ Editing notes — to do / to check
Working notes for the author — not shown to students once collapsed; remove before publishing.
- [ ]
What you’ll get from this sheet
Regression, Bayesian‑style — and where it really shines: comparing many models at once. By the end you should be able to:
- Run a Bayesian regression with
lmBF(). - Rank a whole family of models with
regressionBF().
lmBF() returns a Bayes factor comparing your regression to the intercept‑only null. regressionBF() — the “souped‑up” version — fits every combination of your predictors and ranks them by Bayes factor, reframing model building as a question of relative evidence across a family of models rather than a string of significance tests.
From one model to a whole family
The BayesFactor idea extends to regression through lmBF(), whose syntax is again essentially identical to lm(). It returns a Bayes factor comparing your model against the intercept‑only null — “do these predictors, together, beat the grand mean?”
Where the Bayesian approach really shines, though, is in comparing many models at once. regressionBF() fits every combination of your predictors and ranks them by Bayes factor, so you can see at a glance which set the data support best. That turns model building from a sequence of individual significance tests into a single question of relative evidence across a whole family of models — a natural fit for the model‑comparison mindset from the regression chapters.
Rank the models
Predict popularity from three candidate predictors — two that genuinely matter (danceability, energy) and one that’s noise (track length). Switch between a single‑model Bayes factor and the full ranked family:
The single‑model view (lmBF) gives one number: how decisively danceability and energy together beat the null. But flip to regressionBF and the real advantage appears — every subset of predictors is fitted and ranked. The winner is danceability + energy, the two real drivers; tacking on track length (which is pure noise) actually lowers the Bayes factor, because the data don’t reward the added complexity. You read off which predictors matter, and how much, in a single glance — no chain of individual t‑tests.
See it in code
library(BayesFactor)
# one model vs the intercept-only null:
lmBF(popularity ~ danceability + energy, data = songs)
# the whole family, ranked by Bayes factor:
bf <- regressionBF(popularity ~ danceability + energy + length, data = songs)
head(bf) # best-supported models first
TipCheck your understanding
In a regressionBF() ranking, adding a predictor makes the Bayes factor go down. What does that tell you — and how does it differ from R²?
That the predictor isn’t earning its keep: the Bayesian model comparison has a built‑in penalty for complexity, so a predictor that adds little genuine signal lowers the evidence for the model. R² would have gone up (it never falls when you add a predictor), which is exactly why R² can’t choose between models of different sizes. The Bayes factor — like adjusted R² or AIC — rewards fit and parsimony, so a falling BF is a signal to leave the predictor out.
The interactive uses a BIC approximation; the real regressionBF() uses proper priors (and the result depends on them — report the prior). Watch the combinatorics: regressionBF() fits every subset, so the number of models explodes with many predictors — fine for a handful, unwieldy for dozens. And a Bayes factor compares models on evidence, not on out‑of‑sample prediction — pair it with cross‑validation when prediction is the goal.
Where this shows up next
This closes the Bayesian thread — and the model‑comparison mindset ties back to the regression evaluation chapters. See Chapter (Bayesian Methods).