AP® Psychology · Science Practices: Research Methods and Data
4.B Peer Review, Replication, and Reasoning Biases
A single study never proves a claim because chance, bias, and error can all produce a convincing-looking result. Peer review, replication, and awareness of reasoning biases like hindsight and confirmation bias are what turn a shaky finding into trustworthy knowledge.
One study can be wrong for boring reasons: a fluke sample, a lucky coincidence, a researcher who unconsciously nudged the data. Even when nothing is done wrong, pure chance lets some studies show a significant effect that isn't real, a false positive. Run the same false claim through many independent studies and the odds that at least one comes back significant by chance alone keeps climbing. That is why a field waits for a pattern across studies, not a single headline result.
Peer review is the first filter: other experts examine the method, the statistics, and the logic before a study is published, catching errors a solo researcher might miss or excuse. Replication is the second filter: independent labs rerun the study, often with different samples or settings, to see if the effect shows up again. A finding that survives both is trusted more, not because one study got better, but because many chances to fail did not break it.
Two reasoning habits make this filtering necessary. Hindsight bias makes a result feel obvious after the fact, even if almost no one predicted it beforehand, which tricks people into skipping the checking step. Confirmation bias pushes researchers and readers to notice evidence that fits what they already believe and wave away evidence that doesn't, while overconfidence makes a first promising result feel more certain than the data actually support. Large-scale replication projects in psychology, which reran dozens of classic studies, found that many effects were weaker than first reported or didn't reappear at all, a wake-up call that reshaped how seriously the field takes replication now.
Key terms
- False positive
- A result that appears statistically significant purely by chance even though the effect it claims isn't real.
- Peer review
- Evaluation of a study's methods and conclusions by other experts before it is published.
- Replication
- Repeating a study, often with new samples or settings, to see whether the original result shows up again.
- Hindsight bias
- The tendency to see an outcome as having been obvious or predictable only after already knowing it happened.
- Confirmation bias
- The tendency to favor information that supports existing beliefs and to ignore or dismiss information that contradicts them.
Watch it explained
- How it worksHow do peer review and replication filter a claim before it becomes knowledge?
- How it worksWalk me through hindsight bias making a result feel obvious only after you hear it.
- How it worksHow do confirmation bias and overconfidence distort how evidence gets weighed?
- Going furtherWhat happened to psychology's confidence in its findings when large replication projects ran?
Check yourself
End of topic.