Building & Evaluating a Linear Mixed Model

Info sheet · Statistics for Psychology & Neuroscience

Author

Andrew Bell

Published

August 13, 2026

Info sheet 0.2 (draft) · Prerequisites: random intercepts & slopes; model comparison (likelihood / AIC) · Give feedback ↗

Working notes for the author — not shown to students once collapsed; remove before publishing.

  • [ ]

What you’ll get from this sheet

Fitting an LMM, and — the part that trips people up — deciding whether an effect matters. By the end you should be able to:

  1. Build a linear mixed model with fixed and random effects.
  2. Evaluate a predictor with a chi-square model comparison (full vs null).

You can’t judge an LMM with the usual p-values, F-tests, or R². Instead you compare models: fit a full model, fit a null model identical but for the term you’re testing, and compare them with a likelihood-ratio chi-square test (anova(null, full)).

The dataset

We’ll borrow a clean example from Andy Field’s textbook: 276 patients across 10 cosmetic-surgery clinics. Post-surgery quality of life (QoL, 0–100) is the outcome; baseline QoL is a covariate; surgery is binary (0 = still waiting, 1 = had surgery); and clinic (1–10) we’ll treat as a random effect. The question: does having surgery improve QoL compared to those still waiting?

Plot baseline vs post-surgery QoL by clinic and the lines don’t share slopes or intercepts — clinics differ. If every clinic behaved identically, plain regression would do; because they vary, an LMM that lets intercepts and slopes differ by clinic fits better. The model:

\[\text{QoL}_{ij} = (\beta_0 + u_{0j}) + (\beta_1 + u_{1j})\,\text{surgery}_{ij} + \beta_2\,\text{baseline}_{ij} + \varepsilon_{ij}\]

where \(u_{0j}\) is the random intercept for clinic \(j\) and \(u_{1j}\) lets the surgery effect vary by clinic.

The workflow

The plan is always roughly the same: load and inspect the data; fit fixed-effects models (ANOVA, regression) as a baseline; decide the random structure (intercept, slope, or both); fit the full LMM (fixed: surgery + baseline; random: clinic); build a null model identical but without surgery; and compare the two with a chi-square test. In this dataset the fixed-effects ANOVA and regression both flag a weak but significant surgery effect — but both ignore clinic-level variability, which likely overstates it.

Why the effect can vanish

Here’s the punchline. Surgery wasn’t assigned at random across clinics — some clinics operate on more patients and get better outcomes. Slide up that confounding and watch two likelihood-ratio tests of “does surgery help?”: one that ignores clinic (plain regression) and one that accounts for it (the LMM’s within-clinic comparison).

Turn the confounding up and the two tests split apart. The naive comparison — surgery vs no surgery, clinics ignored — sails past the χ² = 3.84 threshold and declares surgery “significant.” But that signal is really which clinics patients attended. The clinic-aware comparison looks within each clinic, finds surgery adds almost nothing, and stays non-significant. That’s exactly what happened in the real data: a weak surgery effect in the ANOVA and regression vanished once clinic-level variability was modelled. Maybe QoL improves with time regardless, or some clinics simply foster better recovery.

Evaluating the model

Standard p-values, F-tests and R² don’t transfer cleanly to mixed models, so we lean on model comparison: fit the full model, fit a null model identical but without the term of interest, and compare with anova(null, full, test = "Chisq").

library(lme4)
full <- lmer(QoL ~ surgery + baseline + (surgery | clinic), data = cosmo, REML = FALSE)
null <- lmer(QoL ~            baseline + (surgery | clinic), data = cosmo, REML = FALSE)
anova(null, full)          # likelihood-ratio chi-square test of 'surgery'
ranef(full)                # the per-clinic random intercepts & slopes
import statsmodels.formula.api as smf
full = smf.mixedlm("QoL ~ surgery + baseline", cosmo, groups=cosmo["clinic"],
                   re_formula="~surgery").fit(reml=False)
# likelihood-ratio test: 2*(llf_full - llf_null) ~ chi2(df diff)

The middle tab runs live and shows the anova() comparison logic; lme4/statsmodels aren’t in the in-page runtime.

Your LMM summary prints a coefficient for surgery. Why not just read off its p-value to decide whether surgery matters?

Because p-values for individual LMM coefficients are unreliable — the denominator degrees of freedom aren’t well defined, so the reported test is only approximate. The trustworthy route is model comparison: fit a null model without surgery, and compare it to the full model with a likelihood-ratio chi-square (anova(null, full)). A significant result means surgery genuinely improves fit beyond baseline and clinic.

For a likelihood-ratio comparison of fixed effects, both models must be fit with maximum likelihood (REML = FALSE), not REML — comparing REML fits that differ in their fixed effects is invalid. The models must also be nested (differ only by the term tested) and fit to the exact same rows (a missing value that drops cases from one model but not the other silently breaks the test). In lme4, anova() on two lmer models refits with ML for you.

Where this shows up next

Next: the assumptions of an LMM and how to write it up. See Chapter (Mixed-Effects Models) for the full cosmetic-surgery walkthrough.