Undercoverage, Nonresponse, and Voluntary Response

Undercoverage happens at the design stage: the sampling frame, the actual list you draw from, leaves out part of the population before anyone is contacted. In 1936, the Literary Digest mailed 10 million surveys using phone books and car registration lists, and got 2.4 million back. That sounds huge, but in the middle of the Great Depression, owning a phone or a car meant you were relatively wealthy. The frame quietly excluded poorer voters, who leaned heavily toward Franklin Roosevelt, so the poll confidently predicted Alf Landon would win. He didn't. Size never fixes this: 2.4 million biased responses are still biased.

Nonresponse bias happens after contact: you reach the right people, but some refuse or can't be bothered to answer, and those people tend to differ systematically from the ones who do respond. Mailing more surveys just produces more nonresponse from the same kinds of people, so the bias persists at any scale. Voluntary response bias is nonresponse's louder cousin: instead of being selected and then dropping out, people choose themselves into the sample entirely, usually because they have strong feelings. An online poll about a controversial policy draws furious opponents and passionate supporters, while everyone lukewarm scrolls past.

All three are forms of sampling bias, and they matter because they attack the one thing a statistical result depends on: the ability to generalize from sample to population. A perfectly calculated margin of error means nothing if the sample itself was built to tilt one way. That's why the units on collecting data insist on random selection before any calculation happens.