Where the Least-Squares Line Comes From

Imagine sliding a line up, down, and tilting it through a scatterplot, checking the vertical misses, the residuals, at every position. You want the line that is overall closest to every point. If you just added up the residuals, positive and negative ones would cancel out, so a terrible line could show a sum near zero by accident, some points wildly above, some wildly below, averaging out. Squaring each residual before adding fixes that: every miss becomes a positive penalty, big misses count a lot more than small ones, and there's exactly one line that makes this total as small as possible.

That line has a clean formula. Its slope is b = r times (SD of y / SD of x), and it passes through the point of averages, (mean of x, mean of y), which pins down the intercept. As you tilt a candidate line away from this exact slope, the squared residuals on one side grow faster than they shrink on the other, so the sum of squares always increases. That's the mechanism behind calling it 'least squares': any other line, even one that happens to pass through more individual points, gives a larger total of squared misses.

This formula is also why correlation and slope are linked but not the same number: r measures strength and direction only, while b rescales that relationship into the actual units of x and y. Later, when you check whether a line was the right model at all, you'll look at the residual plot built from these same squared-minimizing residuals.