t-Tests (One-Sample, Independent, Paired)

Info sheet · Statistics for Psychology & Neuroscience

Author

Andrew Bell

Published

August 13, 2026

Info sheet 0.2 (draft) · Prerequisites: the mean and SD; what a p-value is · Give feedback ↗

Working notes for the author — not shown to students once collapsed; remove before publishing.

  • [ ]

What you’ll get from this sheet

The workhorse for comparing two means — and why it doesn’t scale to many groups. By the end you should be able to:

  1. Pick the right t‑test (one‑sample, independent, paired) and read the t‑formula.
  2. Choose one‑ vs two‑tailed, and count pairwise comparisons with \(k(k-1)/2\).

A t‑test compares means by scaling the difference by its variability: \(t = \dfrac{\text{difference}}{\text{standard error}}\). Big difference or small variance → big t. Three flavours: one‑sample (mean vs a known value), independent (two unrelated groups), paired (two related scores). Comparing many groups needs \(k(k-1)/2\) t‑tests — which inflates family‑wise error.

The three t-tests

All three ask “do these means differ?”, differing only in the design:

  • One‑sample — compare a sample mean to a known value (is mean IQ different from 100?).
  • Independent‑samples — compare two unrelated groups (danceability of 2000s vs 1990s songs).
  • Paired‑samples — compare two related scores (the same people, pre vs post).

The engine is the same in each: the t‑statistic scales the difference in means by the variability. By hand, a large difference in means — or small variances — produces a large t, which infers a real difference; a small difference or large variances produces a small t, inferring none. It’s the signal‑to‑noise idea again: \(t = \text{signal} \div \text{noise}\).

Big difference or small noise → big t

Two decades of danceability. Slide the difference in their means and watch t and its p‑value respond; switch between a one‑ and two‑tailed test:

At zero difference the means sit atop each other and t ≈ 0. Widen the gap (or, equivalently, shrink the noise) and t grows until it clears the critical value and the result turns significant. Flip to one‑tailed and the same t reaches significance sooner — because a one‑tailed test spends all of α (.05) in a single tail, testing only one direction (“2000s more danceable”), whereas a two‑tailed test splits α across both tails and is more conservative (testing for any difference).

Why it doesn’t scale: pairwise comparisons

A t‑test compares two groups. To compare more, you’d need every pair — and the count grows fast:

\[\text{number of comparisons} = \frac{k(k-1)}{2}\]

So 3 groups → 3 tests, 4 → 6, and 6 groups → 15. Each test carries its own 1‑in‑20 chance of a false positive at α = .05, and those chances accumulate — much like rolling a die ten times gives roughly an 84% chance of at least one six. That runaway family‑wise error is why we don’t just spray t‑tests at a multi‑group design; we use ANOVA (one omnibus test) and, if needed, corrected post‑hoc comparisons.

See it in code

[h, p, ci, stats] = ttest2(d00, d90);   % independent-samples t-test
% [h,p] = ttest(x, 50);                 % one-sample
% [h,p] = ttest(pre, post);             % paired

The R and Python tabs run live; MATLAB is a static reference.

You compare two groups and get t = 0.4, p = .69, even though their means differ by a few points. Why isn’t it significant — and would a one‑tailed test have saved it?

Because t is the difference relative to its standard error: a few points of difference against large within‑group variability (or a small sample) gives a small t. The signal is swamped by noise. A one‑tailed test would only help modestly (it lowers the threshold from ~1.96 to ~1.64), and only if the difference is in the predicted direction — it can’t rescue a t of 0.4. The real fix is more data or less noise, not a friendlier tail.

Decide one‑ vs two‑tailed before you see the data — picking a one‑tailed test after peeking, because it nudges you under .05, is a form of p‑hacking. A one‑tailed test also can’t detect an effect in the opposite direction, so use it only when a reversal would be uninteresting or impossible. And never answer a many‑group question with a pile of t‑tests — the \(k(k-1)/2\) comparisons inflate family‑wise error; use ANOVA instead.

Where this shows up next

The pairwise‑comparison explosion motivates Family‑Wise Error and ANOVA (one test for many groups). And the independent t‑test is just a two‑group general linear model. See Chapter (Inference).