Concluding About Two Proportions

A conclusion for a two-proportion test has three parts: compare the p-value to your significance level, state whether you reject or fail to reject the null hypothesis, and translate that decision back into the context of the two groups. Something like: 'Since the p-value of 0.012 is less than alpha = 0.05, we reject the null hypothesis. There is convincing evidence that the true proportion of defective parts differs between Factory A and Factory B.' Notice the conclusion names the p-value, the comparison to alpha, and the actual groups — not just 'the null is rejected.'

A significant p-value tells you the gap between two sample proportions is too large to blame on chance alone, assuming the null is true. It does not tell you why the gap exists. If the data came from an observational study — you just measured two existing groups without assigning treatments — any lurking variable could be driving the difference. Only random assignment, which spreads confounding variables evenly between groups, lets you credit the explanatory variable itself. That is why the exact same statistical result supports causation in an experiment but only association in an observational study.

Sample size also complicates the picture. Standard error shrinks as sample size grows, so with enough data even a trivial, practically meaningless gap becomes statistically significant. A tiny sample can hide a real, important difference because the standard error is too large to detect it. This links straight back to Unit 1's design principles: random assignment is what earns you a causal claim, and adequate sample size is what gives a test the power to detect a difference worth caring about.