Statistical significance

Statistical significance tells you the observed difference is unlikely to be random. 95% confidence is standard.

Key concepts

• p-value: probability the result is random
• Confidence interval: range of likely true values
• Sample size: larger = narrower confidence

Common mistakes

• Stopping tests early (p-hacking)
• Running too many tests without correction
• Interpreting non-significance as 'no effect'

Pragmatic threshold

95% confidence, minimum sample size for effect size of interest.