Standardization, Reliability, and Validity

Picture a dartboard. Reliability is a tight cluster of darts — they land close together, wherever that happens to be. Validity is darts landing on the bullseye. A test is reliable if it keeps producing the same kind of result; it's valid if that result reflects the real, true thing it's supposed to measure. These are separate questions, because a tightly clustered group of darts can sit far from the bullseye. A test can be rock-solid consistent and still be wrong every time.

Psychologists check reliability by comparing a test against itself two ways. Test-retest reliability gives the same test to the same people at two different times and checks whether scores stay similar — if a personality test says you're highly extroverted in January and highly introverted in June, that's low test-retest reliability. Split-half reliability divides one test into two halves (like odd versus even questions) and checks whether a person's score on one half matches their score on the other. High correlation in either method means the test is stable and internally consistent. Neither method asks whether the test measures the right construct — that's validity's job, often checked by seeing whether scores predict some real-world outcome the test claims to predict.

This matters because standardization — giving a test under uniform conditions and comparing scores to a norming sample — only works if that norming sample resembles the people being tested. A test normed on college-educated adults in one country produces scores that mean something specific relative to that group. Hand the same test to young children from a different country and education system, and the comparison breaks down; the score no longer tells you much, even if the test itself is perfectly reliable. This connects standardization directly to validity: a mismatched norm group is itself a validity problem.