When Data Aren’t Normal

Info sheet · Statistics for Psychology & Neuroscience

Info sheet 0.2 (draft) · Prerequisites: the normal distribution; describing data · Give feedback ↗

Warning✎ Editing notes — to do / to check

Working notes for the author — not shown to students once collapsed; remove before publishing.

• [ ]

What you’ll get from this sheet

Real data are often stubbornly non‑normal — and pretending otherwise causes trouble. By the end you should be able to:

1. Recognise right‑skew and why reaction times are the classic case.
2. Choose between a transformation and a non‑parametric test.

Many measurements — reaction times, incomes, firing rates — can’t go below zero and have a long right tail, so they’re right‑skewed, not symmetric. Skew pulls the mean toward the tail and destabilises parametric tests. Two fixes: transform the data (a log transform tames a right tail), or use a non‑parametric test.

The classic case: reaction times

Reaction times are the textbook example. They can’t be negative and they have a long right tail — the occasional slow response drags out to the right — so their distribution is right‑skewed rather than symmetric. Incomes, response latencies, and neural firing rates behave the same way. When a distribution is skewed, the mean is pulled toward the tail (so it overstates the “typical” value), and many standard tests — which assume roughly normal residuals — become unreliable.

So we have two moves. Either transform the data so it’s closer to normal — a log transform tames a long right tail beautifully — or reach for the non‑parametric methods (like the Kruskal–Wallis test) that make no distributional assumption.

Watch a log transform tame the tail

A right‑skewed, reaction‑time‑like distribution. Flip to the log scale and watch the long tail pull in and the shape turn symmetric:

md`**Show data on the**`

viewof scale = html`<select style="font-size:15px;padding:4px 8px"><option value="raw" selected>raw scale (skewed)</option><option value="log">log scale (transformed)</option></select>`

transformWidget = {
  const n=500; let s=13;
  const rnd=()=>{s|=0;s=s+0x6D2B79F5|0;let t=Math.imul(s^s>>>15,1|s);t=t+Math.imul(t^t>>>7,61|t)^t;return((t^t>>>14)>>>0)/4294967296;};
  const nrm=()=>{const u=1-rnd(),v=rnd();return Math.sqrt(-2*Math.log(u))*Math.cos(2*Math.PI*v);};
  const rt=[]; for(let i=0;i<n;i++) rt.push(250*Math.exp(0.45*nrm()));   // lognormal RTs
  const data = scale==="log" ? rt.map(v=>Math.log(v)) : rt;
  const sorted=[...data].sort((a,b)=>a-b);
  const mean=data.reduce((a,b)=>a+b,0)/n, median=sorted[Math.floor(n/2)];
  const sd=Math.sqrt(data.reduce((a,v)=>a+(v-mean)**2,0)/n);
  // skewness
  const skew=data.reduce((a,v)=>a+((v-mean)/sd)**3,0)/n;
  const lo=Math.min(...data), hi=Math.max(...data), bins=36, bw=(hi-lo)/bins, c=Array(bins).fill(0);
  data.forEach(v=>{let b=Math.floor((v-lo)/bw); if(b>=bins)b=bins-1; if(b>=0)c[b]++;});
  const W=460,H=270,pad=30, plotH=H-2*pad, mx=Math.max(...c);
  const X=v=>pad+(v-lo)/(hi-lo)*(W-2*pad), Y=k=>H-pad-k/mx*plotH;
  const col = scale==="log" ? "#1E7A34" : "#C0501E";
  let bars=c.map((k,i)=>`<rect x="${X(lo+i*bw)}" y="${Y(k)}" width="${(W-2*pad)/bins-1}" height="${H-pad-Y(k)}" fill="${col}" fill-opacity="0.45"/>`).join("");
  const fig=document.createElement("figure"); fig.style.margin="0";
  fig.innerHTML=`<svg viewBox="0 0 ${W} ${H}" width="${Math.min(W,520)}" style="background:#f8fafc;border:1px solid #e2e8f0;border-radius:8px"><line x1="${pad}" y1="${H-pad}" x2="${W-pad}" y2="${H-pad}" stroke="#94a3b8"/><text x="${W/2}" y="${H-8}" text-anchor="middle" font-size="11" fill="#667">${scale==="log"?"log(reaction time)":"reaction time (ms)"}</text>${bars}</svg>`+
    `<figcaption style="margin-top:.5rem">skewness = <b style="color:${Math.abs(skew)<0.4?'#1E7A34':'#c0392b'}">${skew.toFixed(2)}</b> (0 = symmetric). ${scale==="log"?'On the log scale the tail is pulled in — roughly symmetric, so parametric tests are back on the table.':'On the raw scale the long right tail makes it strongly right‑skewed — the mean overstates the typical response.'}</figcaption>`;
  return fig;
}

On the raw scale the distribution has a heavy right shoulder and a skewness well above zero. Take the log and the same data become close to symmetric (skewness near zero) — which means a t‑test or ANOVA on the log reaction times is now on safe ground. That’s the appeal of a transformation: fix the shape once, then use the familiar parametric toolkit.

Transform or go non-parametric?

• Transform when a simple function restores normality (log for a right tail; square‑root for count data). You then analyse and report on the transformed scale — remembering that a difference in log‑means is a ratio on the original scale.
• Go non‑parametric when no transform helps, or when the data are ordinal to begin with: the Kruskal–Wallis test (for group comparisons) and its friends work on ranks and assume nothing about the distribution.
• Or lean on robustness — with a decent sample size, ANOVA and t‑tests tolerate mild non‑normality, thanks to the Central Limit Theorem acting on the sampling distribution of the mean.

See it in code

TipCheck your understanding

Your reaction‑time data fail a normality test. Give two different ways to proceed.

Two good options. (1) Transform: take the log of the reaction times — this pulls in the long right tail, and if the logged data are approximately normal you can run the usual t‑test/ANOVA on them (reporting results on the log scale). (2) Go non‑parametric: use a rank‑based test that makes no distributional assumption — the Kruskal–Wallis test for comparing groups (or Mann–Whitney for two groups). A third, with a large enough sample: rely on ANOVA’s robustness to mild non‑normality.

You can’t log‑transform zero or negative values — reaction times are fine (all positive), but for data with zeros use log(x + 1) or a square‑root transform instead. Remember that results on a transformed scale need careful interpretation (a difference of log‑means is a ratio back on the raw scale). And it’s the residuals’ normality that matters for a model, not the raw variable’s — check the residuals, not just the histogram of the outcome.

Where this shows up next

If a transform won’t help, the Kruskal–Wallis sheet is your non‑parametric route; the ANOVA assumptions sheet shows how to check normality in the first place. See Chapter (Foundations).

Related topics

• The Normal Distribution & the CLT — what “normal” is and why means tend toward it.
• Describing Data — mean vs median under skew.
• The Kruskal-Wallis Test — the non‑parametric group comparison.
• The Assumptions of ANOVA — testing normality of residuals.
• What Kind of Data Is This? (levels of measurement) — when data are ordinal from the start.