When Data Aren’t Normal

Info sheet · Statistics for Psychology & Neuroscience

Info sheet 0.2 (draft) · Prerequisites: the normal distribution; describing data · Give feedback ↗

Warning✎ Editing notes — to do / to check

Working notes for the author — not shown to students once collapsed; remove before publishing.

• Added both right-skew (reaction times) and left-skew (easy-exam scores) examples.
• Upgraded the demo to several datasets × several transforms (log, √, square, reciprocal) and mentioned Box–Cox / Yeo–Johnson / rank-inverse-normal.
• Added an early caution on why non-normal data threatens standard tests.

What you’ll get from this sheet

Real data are often stubbornly non‑normal — and pretending otherwise causes trouble. By the end you should be able to:

1. Recognise right-skew (reaction times) and left-skew (easy-exam scores).
2. See how standard transforms (log, √, and friends) reshape a skewed distribution.
3. Choose between a transformation and a non-parametric test.

Many measurements — reaction times, incomes, firing rates — can’t go below zero and have a long right tail, so they’re right‑skewed, not symmetric. Skew pulls the mean toward the tail and destabilises parametric tests. Two fixes: transform the data (a log transform tames a right tail), or use a non‑parametric test.

First, why it’s a problem

Before the fixes, the warning. Most of our default tests — t-tests, ANOVA, ordinary regression — assume the data (strictly, the model’s residuals) are roughly normal. Feed them badly skewed data and things go wrong quietly: the mean stops being “typical” (it’s dragged toward the long tail), standard errors and p-values get distorted, and a few extreme values in the tail wield outsized influence. Non-normality doesn’t always sink an analysis — but it always earns a second look before you trust the numbers.

Two shapes of skew

• Right-skew (positive skew) — a long tail to the high side. Reaction times are the textbook case: there’s a hard floor (you can’t respond in negative time) but no ceiling, so the occasional slow trial stretches the right tail. Incomes and neural firing rates behave the same way.
• Left-skew (negative skew) — a long tail to the low side, values piling up near a ceiling. Scores on an easy exam are the classic case: most students cluster near the top and a few stragglers trail off to the left. “Proportion correct” near 100% and age at death skew this way too.

In both, the mean is pulled toward the tail and away from the bulk of the data — which is exactly why the median is often the more honest summary of a skewed variable.

Fix option 1: transform the data

A transformation re-expresses the values on a new scale to pull the distribution back toward symmetry. Pick a dataset and a transform, and watch the skewness move toward zero (or away from it):

md`**Dataset**`

viewof dset = html`<select style="font-size:15px;padding:4px 8px"><option value="right" selected>Reaction times (right-skew)</option><option value="left">Easy-exam scores (left-skew)</option><option value="count">Error counts (count data)</option></select>`
md`**Transform**`

viewof tform = html`<select style="font-size:15px;padding:4px 8px"><option value="none" selected>none (raw)</option><option value="log">log</option><option value="sqrt">square root</option><option value="square">square</option><option value="recip">reciprocal (1/x)</option></select>`

transformWidget = {
  let s = 13;
  const rnd=()=>{s|=0;s=s+0x6D2B79F5|0;let t=Math.imul(s^s>>>15,1|s);t=t+Math.imul(t^t>>>7,61|t)^t;return((t^t>>>14)>>>0)/4294967296;};
  const nrm=()=>{const u=1-rnd(),v=rnd();return Math.sqrt(-2*Math.log(u))*Math.cos(2*Math.PI*v);};
  const n=500, raw=[];
  for(let i=0;i<n;i++){
    if(dset==="right") raw.push(250*Math.exp(0.45*nrm()));                      // lognormal RTs
    else if(dset==="left"){ const x=100-6*Math.exp(0.5*nrm()); raw.push(Math.max(1,Math.min(100,x))); }
    else { let L=Math.exp(-3),k=0,p=1; do{k++;p*=rnd();}while(p>L); raw.push(k-1); } // Poisson(3)
  }
  const tf={none:x=>x, log:x=>Math.log(x+(dset==="count"?1:0)), sqrt:x=>Math.sqrt(Math.max(x,0)),
            square:x=>x*x, recip:x=>1/(x+(dset==="count"?1:0))}[tform];
  const data=raw.map(tf).filter(Number.isFinite);
  const m=data.length, mean=data.reduce((a,b)=>a+b,0)/m;
  const sd=Math.sqrt(data.reduce((a,v)=>a+(v-mean)**2,0)/m) || 1;
  const skew=data.reduce((a,v)=>a+((v-mean)/sd)**3,0)/m;
  const lo=Math.min(...data), hi=Math.max(...data), span=(hi-lo)||1, bins=34, bw=span/bins, c=Array(bins).fill(0);
  data.forEach(v=>{let b=Math.floor((v-lo)/bw); if(b>=bins)b=bins-1; if(b>=0)c[b]++;});
  const W=460,H=260,pad=30, plotH=H-2*pad, mx=Math.max(...c);
  const X=v=>pad+(v-lo)/span*(W-2*pad), Y=k=>H-pad-k/mx*plotH;
  const good=Math.abs(skew)<0.5, col= good?"#1E7A34":"#C0501E";
  const bars=c.map((k,i)=>`<rect x="${X(lo+i*bw)}" y="${Y(k)}" width="${(W-2*pad)/bins-1}" height="${H-pad-Y(k)}" fill="${col}" fill-opacity="0.45"/>`).join("");
  const note = good ? 'Close to symmetric — parametric tests are back on the table.'
             : (skew>0 ? 'Still right-skewed — try log, √, or 1/x.' : 'Still left-skewed — try squaring.');
  const fig=document.createElement("figure"); fig.style.margin="0";
  fig.innerHTML=`<svg viewBox="0 0 ${W} ${H}" width="${Math.min(W,520)}" style="background:#f8fafc;border:1px solid #e2e8f0;border-radius:8px"><line x1="${pad}" y1="${H-pad}" x2="${W-pad}" y2="${H-pad}" stroke="#94a3b8"/>${bars}</svg>`+
    `<figcaption style="margin-top:.5rem">skewness = <b style="color:${col}">${skew.toFixed(2)}</b> (0 = symmetric). ${note}</figcaption>`;
  return fig;
}

The rules of thumb the demo makes visible: log and square-root pull in a right tail (log is the stronger of the two; √ is the standard choice for count data); squaring or cubing helps a left tail; and the reciprocal (1/x) is a very strong right-tail transform for heavily skewed positive data. You then analyse and report on the transformed scale — remembering that a difference of log-means is a ratio back on the original scale.

Beyond these staples sit less common tools: the Box–Cox and Yeo–Johnson families search for the best power transform automatically (Yeo–Johnson even copes with zeros and negatives), and the rank-based inverse-normal transform forces any distribution to normal by working through ranks. Powerful, but harder to interpret — so reach for the simple transforms first.

Fix option 2: go non-parametric (or lean on robustness)

• Go non-parametric when no transform helps, or when the data are ordinal to begin with: rank-based tests like Kruskal–Wallis (group comparisons) and Mann–Whitney (two groups) assume nothing about the distribution’s shape.
• Or lean on robustness — with a decent sample size, ANOVA and t-tests tolerate mild non-normality, because the CLT normalises the sampling distribution of the mean even when the raw data aren’t normal.

See it in code

R

Python

MATLAB / Octave

rng(1);
rt = 250*exp(0.45*randn(300,1));            % right-skewed reaction times
[~, p_raw] = lillietest(rt);                % Lilliefors normality test → p < .05
[~, p_log] = lillietest(log(rt));           % log transform → usually n.s.
[p_raw, p_log]

The R and Python tabs run live; MATLAB is a static reference.

TipCheck your understanding

Your reaction‑time data fail a normality test. Give two different ways to proceed.

Two good options. (1) Transform: take the log of the reaction times — this pulls in the long right tail, and if the logged data are approximately normal you can run the usual t‑test/ANOVA on them (reporting results on the log scale). (2) Go non‑parametric: use a rank‑based test that makes no distributional assumption — the Kruskal–Wallis test for comparing groups (or Mann–Whitney for two groups). A third, with a large enough sample: rely on ANOVA’s robustness to mild non‑normality.

You can’t log‑transform zero or negative values — reaction times are fine (all positive), but for data with zeros use log(x + 1) or a square‑root transform instead. Remember that results on a transformed scale need careful interpretation (a difference of log‑means is a ratio back on the raw scale). And it’s the residuals’ normality that matters for a model, not the raw variable’s — check the residuals, not just the histogram of the outcome.

Where this shows up next

If a transform won’t help, the Kruskal–Wallis sheet is your non‑parametric route; the ANOVA assumptions sheet shows how to check normality in the first place. See Chapter (Foundations).

Related topics

• The Normal Distribution & the CLT — what “normal” is and why means tend toward it.
• Describing Data — mean vs median under skew.
• The Kruskal-Wallis Test — the non‑parametric group comparison.
• The Assumptions of ANOVA — testing normality of residuals.
• What Kind of Data Is This? (levels of measurement) — when data are ordinal from the start.