Type I and Type II Errors

Every significance test ends in a decision: reject the null hypothesis or fail to reject it. But the null hypothesis is either true or false in reality, and you never get to see which. Cross those two decisions with those two truths and you get four boxes. Two boxes are correct decisions: rejecting a false null, or failing to reject a true one. Two boxes are errors. A Type I error means you rejected a null hypothesis that was actually true — you cried wolf. A Type II error means you failed to reject a null hypothesis that was actually false — you missed a real effect that was there.

These two errors cost different things depending on context, which is why you can't just minimize both at once. Lowering alpha from 0.05 to 0.01 makes you demand stronger evidence before rejecting the null, so Type I error becomes less likely. But that stricter standard makes it easier for a real effect to slip past undetected, so Type II error becomes more likely. The two rates move in opposite directions for a fixed sample size; the only way to shrink both is to collect more data.

A Type I error is not an arithmetic slip. The calculations can be flawless and a Type I error can still happen, purely from unlucky sampling variation — some sample that looks extreme by chance even though the null is true. That's why alpha is described as the probability of a Type I error, baked into the test by design, not a rate of mistakes. This topic sets up power, which is the probability of correctly rejecting a false null, and it closes out single-proportion inference before the course moves to comparing two proportions.