Unbiased Estimators

An estimator is a rule for using sample data to guess a population parameter, like using sample proportion p-hat to guess true proportion p. No single sample will nail the parameter exactly, since each sample is a different random slice of the population. What matters is the long-run behavior: if you imagine repeating the sampling process over and over and plotting every estimate, you get a sampling distribution. An estimator is unbiased if that sampling distribution is centered exactly on the true parameter. Being right on average, across many samples, is the only standard that makes sense for a single number computed from random data.

The sample maximum shows why this matters. If you want to estimate the largest value in a population, using the largest value you happened to observe sounds reasonable, but it systematically falls short. Almost every sample misses the one true population maximum, since that single extreme value has to actually land in your sample, and most samples it doesn't. So the sampling distribution of the sample max sits below the true max, every time. That's bias, not randomness.

This idea sets up two things coming next. Sample proportion p-hat is unbiased for p, which is why it anchors confidence intervals and significance tests for proportions. And bias is different from variability, the standard deviation of the sampling distribution, and from dividing by n versus n minus 1 when estimating variance, a separate fix for a different kind of bias that shows up once you move from proportions to spread.