Sample size

Sample size depends on baseline conversion, effect size you want to detect, and confidence level.

Calculators

• VWO, Optimizely have built-in calculators
• Evan Miller's A/B testing calculator
• Excel-based calculators

Rule of thumb

2-5% baseline conversion rate with 10% expected lift needs 10,000+ visitors per variant.

Low traffic reality

Sub-significant tests just mean 'we don't know.' Don't ship losers as winners.