Ordinary Least Squares: Finding the Best Line

Info sheet · Statistics for Psychology & Neuroscience

Author

Andrew Bell

Published

August 13, 2026

Info sheet 0.1 (draft) · Prerequisites: scatterplots; the equation of a straight line (y = a + bx) · Give feedback ↗

Working notes for the author — not shown to students once collapsed; remove before publishing.

  • [ ]

What you’ll get from this sheet

In this sheet we’ll briefly review what ordinary least squares (OLS) regression is, and then use code to run one on some bivariate data — bivariate just being a slightly snobby way of saying two variables. By the end you should be able to:

  1. Say what the “line of best fit” is, and how we use it to make predictions.
  2. Explain how OLS finds that line — by minimising the mean squared error.
  3. Run an OLS regression in R, Python, or MATLAB.

For each point, the residual = (actual Y − predicted Y). Square the residuals and average them → the Mean Squared Error (MSE). OLS is the line that minimises the MSE.

Finding the line of best fit

When we run a regression — simple or otherwise — we’re basically trying to find the line of best fit between our X values and our Y values. Once we know that line, we can use it to make predictions about Y from a value of X. If the blue line is the line of best fit, we could take any X — say, 30-ish — and read off a predicted Y of around 45.

So the critical question is: how do we fit that line? We can clearly see that a badly-fitting line (green) is worse than a decent one (blue) — but is some other line (red) better still? How do we actually know when we’ve found the best one?

Enter Ordinary Least Squares

The answer is a method called Ordinary Least Squares (OLS). OLS simply estimates the line of best fit by minimising the squared differences between the real data points and the values the line predicts.

For each point, the gap between the true Y and the predicted Y is — that’s right — a residual. If we take all of those residuals, square them (to get rid of the +/− sign), add them all up, and divide by the number of data points, we get the Mean Squared Error (MSE). OLS works by figuring out the line that minimises this squared error — hence least squares. (“Ordinary,” apparently, because it was one of the very first statistical methods ever developed… or so Google tells me.)

Try it: shrink the squares

Each pink square below is one residual, squared — and that squaring is exactly what gets rid of the sign, so points above and below the line can’t just cancel out. Their average is the MSE, and OLS is the line that makes it as small as it can be. Have a go: drag the slope and intercept to push the MSE down, and see how close you can get to the line of best fit.

Notice you can’t get the MSE below the green number no matter how you tilt the line — that minimum is the OLS solution, and R, Python, and MATLAB all compute it directly.

Here’s a real one from the Decade of Dance set. Take YMCA by the Village People — danceability of 72. Its true popularity is 71; the regression line predicts 67.7, which is not bad at all. The squared difference comes to just 10.89 — a very small error. Do that for every song, add them up, divide by the number of datapoints, and you’ve got the MSE — and the line we’ve fitted is the one that makes that MSE as small as possible.

See it in code

Using the same ten points, each language finds the least‑squares slope and intercept for you:

x = (1:10)';
y = [4;5;9;8;12;11;16;15;20;18];
p = polyfit(x, y, 1);   % p = [slope  intercept], least squares

Static reference — the R and Python tabs run live in the page.

In the book we run exactly this on real data — lm(popularity ~ danceability, data = dataset) on the Decade of Dance set — but the machinery is identical.

Why does OLS square the residuals rather than just add them up?

Two reasons. First, residuals are positive above the line and negative below it, so a plain sum would let them cancel — a terrible line could score zero. Squaring makes every miss count as positive. Second, squaring penalises big misses far more than small ones, so OLS works hard to avoid large errors.

OLS minimises the vertical distances (errors in y), assuming x is measured without error — it is not the perpendicular distance to the line. And because errors are squared, a single outlier can dominate the fit and drag the line toward itself. Always plot your data and check for influential points.

Where this shows up next

OLS is a way of solving for the coefficients of a regression — it’s not the only one (you’ll likely meet gradient descent next year in computing for brain scientists), but it’s by far the easiest to understand, and it’s what we use for most simple to moderately complex models. In R it’s just lm(), which you’ve run plenty of times already. See Chapter 9 (Correlation and Regression) for the full treatment and the assumptions that make the OLS line trustworthy.