Testing Parameters - A/B Testing Campaign

Print Friendly and PDF Follow

Recommended Test Group Size

Audience Size Test Group A B Winner
300 or less 100% 50% 50% 0%
400 80% 40% 40% 20%
500 70% 35% 35% 30%
600 65% 32.5% 32.5% 35%
700 55% 27.5% 27.5% 45%
800 - 900 50% 25% 25% 50%
1,000 45% 22.5% 22.5% 55%
1,100 - 1,200 40% 20% 20% 60%
1,300 - 1,400 35% 17.5% 17.5% 65%
1,500 - 1,800 30% 15% 15% 70%
1,900 - 2,300 25% 12.5% 12.5% 75%
2,400 - 3,200 20% 10% 10% 80%
3,300 - 4,900 15% 7.5% 7.5% 85%
5,000 or more 10% 5% 5% 90%


Cochran's formula was used to calculate the recommended test size group.

Test Duration Recommendation

Based on research, the winner will be correctly chosen 80% of the time if you run your test 2- 5 hours. Your own institutional results may vary.