The Math behind A/B testing to ascertain which site is better

Assume you have two website designs – A & B on your eCommerce website, and you end up with 45 conversions out of 100 visitors for design A and 50 conversions out of 100 visitors for design B.


What’s the chance that design B is better than design A?

10%? No, that’s wrong. Design B is actually 76% better than design A and to make the switch, this probability has to be > 90%. Part-2 above also provides a shortcut formula to make this calculation.

The below three part series provide very good English and Math explanation on how to evaluate results from split testing on two designs.

Part – 1, Part – 2, Part – 3


Leave a Reply

Your email address will not be published.