If you widen the shelf space given to coffee in a supermarket, do sales go up? Intuition says yes. But by how much? What is one more metre of shelf worth, in packets?
The method that answers this is the oldest and most widely used tool in statistics: regression. The idea fits in a sentence — fit a line to the points, then ask the line questions.
Twelve weeks of coffee
We have twelve weeks of data. Two numbers per week: the shelf length given to coffee, and the packets sold. Shelf space is three, six or nine units; sales range from 346 to 672.
Plot the points and one thing stands out: the three-unit weeks cluster low, the nine-unit weeks high. But there is real spread within each cluster too. With the same shelf space, one week sold 560 packets and another 672. So shelf space does not determine sales; it only influences them.
Which line?
Infinitely many lines can be drawn through a cloud of points. Which one is "right"?
Least squares sets a clear criterion: take each point's vertical distance to the line, square it, add them all up. The line that makes this sum smallest wins.
Why squared? The square of the distances
If you summed the distances as they are, errors above and below the line would cancel out — even a terrible line could total zero. Squaring removes the sign and punishes large errors more heavily than small ones. It also reduces finding the minimum to a calculation clean enough to do with a pencil.
On the coffee data the calculation gives this line: one more unit of shelf space is worth an average of 34.6 packets a week. The intuitive "it goes up" has turned into a number.
Where the line stops speaking
The most common mistake here is extending the line beyond the data. The observations sit between three and nine units. On paper the line runs to infinity, and it will happily answer "what if we gave it a hundred units?" That answer rests on nothing.
A regression line only speaks within the range that fed it. Step outside, and you stop producing predictions and start producing wishes.
More than one variable
In real life shelf space alone does not determine sales: price, season, advertising and what competitors are doing all play a part. The same idea extends to several variables, each with its own coefficient.
With one caveat: if the variables are too much alike, the model gets confused. Include both "shelf length" and "shelf area" — two columns measuring nearly the same thing — and the calculation cannot tell which one is doing the work.
If the relationship is a curve rather than a straight line, the same method handles curves too. The surprising part: a model with squared and cubed terms is still called "linear," because linearity is sought not in the variables but in the coefficients.
And this is machine learning
Regression is as much a topic in a machine learning course today as in a statistics one. The difference is not in the method but in the habits.
The machine learning tradition splits the data in two: it fits the line on one part and tests it on the part it held back. Why? Because it is easy for a model to explain the data it has seen; the real question is what it does on data it has not.
Because it trains on labelled data and produces a numerical prediction, regression is by definition supervised learning. Which is to say: the first step on the road to deep learning is that line through twelve weeks of coffee sales.
What is this article based on?
This piece draws on Selim Kusursuz's survey Regression Analysis (2019). The coffee example and the calculations come from the dataset used there.
- Montgomery, Peck, Vining, Introduction to Linear Regression Analysis.
- Pedregosa et al., Scikit-learn: Machine learning in Python, JMLR, 2011.
Selim Kusursuz
Studies Computer Network Systems at Paris-Saclay, on the Autonomous Systems track. Works on mathematics and computer science. Speaks French, English and Turkish well.
See all their articles