Bias and Variability of a Statistic

A statistic comes from one sample, but it has a whole sampling distribution behind it: the collection of values you'd get if you repeated the sampling process over and over. Bias asks about the center of that distribution: does it sit on the true parameter value, or is it shifted off to one side because of something wrong in how the sample was collected, like a flawed survey method or a non-random selection process? Variability asks about the spread: even if the center is right, how much do individual samples scatter around it? A statistic can be unbiased and still be practically useless on any given sample if that spread is huge, because 'correct on average' says nothing about how far any one try can land.

This is why a small, carefully randomized sample can beat a massive sloppy one. Size only shrinks spread; it cannot undo bias. A huge sample built on a biased method, like a voluntary response poll, just gives you a very precise estimate of the wrong number. A smaller sample from a proper random mechanism has no systematic pull in either direction, so growing it steadily tightens the distribution around the truth.

Quadrupling the sample size shrinks the standard error, roughly cutting the spread in half, because standard error involves dividing by the square root of n. That improves precision but does nothing to correct bias baked into the sampling method. This distinction feeds directly into margin of error: margin of error is built from the standard error of the sampling distribution, so it only measures the variability problem. A tiny margin of error can still accompany a badly biased estimate, which is why checking how data was collected matters as much as checking the spread.