Tag Archives: power calculation

A/B testing statistical significance

How much traffic is needed for an A/B test to achieve statistically significant results?

The answer to this question varies considerably based on what you’re trying to test, and how you’re trying to test it. A few important questions to ask yourself are:

What page or part of the site are you trying to optimize?
Most of your sites’ pages will receive only a fraction of site-wide visits, and in order not to to inflate sample sizes you’ll want to perform lazy assignment (read more on that – Lazy Assignment: Get More Done by Getting Lazy)You can dig into your site’s analytics data to get a better understanding of what you’re working with traffic-wise. Continue reading