Response Bias and Question Wording

Two surveys about the same law can land 30 points apart because they asked different questions, even if both talked to a representative sample. Response bias comes from the words on the page, the order of the choices, or who is asking, not from which people were chosen. A loaded question like 'Don't you agree that the wasteful new spending bill should be stopped?' plants a judgment ('wasteful') and a desired answer ('agree') right inside the question, so almost everyone nudged that way answers the same direction regardless of their real opinion. A neutral rewrite just states the policy and asks for a view: 'Do you support or oppose this spending bill?' Order matters too: people tend to favor options they hear or read first or last, so flipping the order of answer choices can shift results without changing a single word of the question itself.

This is a different problem from sampling error, and that distinction matters. A huge sample size shrinks random variability around the true average opinion, but if the question itself is biased, every single response drifts the same direction, large sample or small. Averaging ten thousand biased answers just gives a very precise wrong number. The fix is better question design, not more respondents.

Who delivers the question changes answers too, especially on sensitive topics like drug use, income, or voting for an unpopular candidate. People often shade their answers toward what seems socially acceptable when a live interviewer is listening, but answer more candidly on an anonymous form. This connects response bias to the bigger theme of the unit: how data was collected shapes whether you can trust it, before you ever compute a single statistic.