Eigenvectors & Eigenvalues

Info sheet · Statistics for Psychology & Neuroscience

Author

Andrew Bell

Published

August 13, 2026

Info sheet 0.2 (draft) · Prerequisites: scatterplots and variance; the basic idea of PCA · Give feedback ↗

Working notes for the author — not shown to students once collapsed; remove before publishing.

  • [ ]

What you’ll get from this sheet

Two ideas with intimidating names but friendly meanings: eigenvectors and eigenvalues. By the end you should be able to:

  1. Say what an eigenvector and its eigenvalue are, in plain terms.
  2. See why the first eigenvector is the direction of maximum spread.
  3. Pull eigenvectors and eigenvalues out of a covariance matrix in R, Python, or MATLAB.

An eigenvector is one of the data’s own natural axes; its eigenvalue is how much variance lies along that axis. PCA finds these axes, orders them from largest eigenvalue to smallest, and calls them principal components.

The data’s own axes

Imagine we’re educational psychologists studying A-level students, and we plot each student on two correlated variables — say, hours of study and hours of sleep. The cloud of points is stretched out along a diagonal. The horizontal and vertical axes we started with (study, sleep) are not the most natural way to describe that cloud. There’s a better axis — one that runs along the direction in which the data are most spread out.

That direction of maximum spread is the first eigenvector. An eigenvector is simply a new axis — a new way of projecting the data — chosen to capture as much variance as possible. The word eigen is German for “own”: these are the data’s own, natural axes. Each eigenvector comes paired with an eigenvalue, a single number telling you how much variance is captured along that direction. The bigger the eigenvalue, the more of the data’s variation that axis explains.

Find the axis of maximum spread

Below is our study-vs-sleep cloud. Rotate the candidate axis (the blue line) and watch how much variance the data have along it — the dashed lines show each point projected onto the axis. Your job: find the angle that makes that variance as large as possible. That angle is the first eigenvector, and the variance you get there is its eigenvalue.

Notice two things. First, there’s a single angle where the captured variance peaks — swing past it in either direction and it drops. That peak direction is the first eigenvector, and the variance there is its eigenvalue (λ₁). Second, the leftover spread — everything perpendicular to that axis — is the second eigenvalue (λ₂). The two are at right angles: orthogonal, and therefore completely uncorrelated.

Eigenvectors and eigenvalues always come in pairs, and PCA generates a whole series of them — one for each original dimension. It orders the pairs from largest eigenvalue to smallest and calls them principal components: the first captures the most variance, the second the most of what’s left, and so on. Two properties make them especially useful — they’re orthogonal (so a set of components has zero multicollinearity), and they’re ordered by variance (so we can keep the first few and bin the rest, confident we’ve kept most of the information).

See it in code

By hand the maths is horrendous — so we won’t. We let the computer decompose the covariance matrix for us:

study = [2 3 3 4 4 5 5 6 6 7 7 8 8 9];
sleep = [3 3.5 4.5 4 5.5 5 6.5 6 7 6.5 8 7.5 9 8.5];
C = cov(study', sleep');      % covariance matrix
[V, D] = eig(C);              % V = eigenvectors, D = eigenvalues on the diagonal

Static reference — the R and Python tabs run live in the page.

A PCA on two variables returns eigenvalues of 4.8 and 0.3. What does that tell you?

The first component captures 4.8 / (4.8 + 0.3) ≈ 94% of the total variance. The two variables are highly correlated — nearly all the spread lies along one direction — so you could keep just the first principal component and throw the second away while losing almost nothing.

Eigenvectors from a covariance matrix depend on the scale of your variables — a variable measured in large units will hog the variance and dominate the first component for no good reason. If your variables are on different scales, standardise them first (use the correlation matrix, or prcomp(..., scale. = TRUE) in R) so each contributes fairly.

Where this shows up next

This is the engine inside PCA. See the PCA in Action sheet for the whole pipeline — decomposing the data, keeping the top few components, and feeding them into a regression — and Chapter (Dimension Reduction) for the full treatment.