Calculate Sample Size to Reach 95% Confidence for Your A/B Test

Sample Size per Recipe Response Rate per Recipe
Control
Test Group
Difference Index

Frequently Asked Questions

What Is a Sample Size for A/B Testing?

A sample size is the number of observations to include in a statistical sample. Choosing the right sample size for your A/B testing is important to ensure you gain accurate data insights from your testing and make sure pages aren’t underperforming due to small or unequal sample sizes. 

Can You Run A/B Tests with Small Sample Sizes?

Yes, you can run A/B tests with small sample sizes, as long as you consider three things: 

  1. Desired lift
  2. Tolerance for risk
  3. Current conversion rate. 

These should be three areas you consider for each A/B test, especially when using small sample sizes. Learn more about these A/B testing considerations and how to make the most out of your tests. 

DISCOVER OTHER HELPFUL RESOURCES

Statistical Significance Calculator

The go-to A/B Test calculator, if you know your KPI conversion rate and sample size, you have everything you need.

Z Score to Confidence Calculator

If you have your standardized z-score, this calculator will help you convert that to a confidence level for either one-sided or two-sided tests.

Chi Square Calculator

Determine if the applied recipe influences conversion by measuring the distance of the actual counts from those that would be expected if they were not related.

Related A/B Testing Blog Posts

A/B testing and experimentation insights and best practices.

How to Analyze Inconclusive A/B Test Results
How to Analyze Inconclusive A/B Test Results

It’s all fun and games until your A/B Test is flat-tastic. Unfortunately, the ideal isn’t always real, and inconclusive A/B…

How to A/B Test During COVID-19
How to A/B Test During COVID-19

A field guide to experimentation in spite of coronavirus and finding the new normal When the effects of the coronavirus…

Why COVID-19 is a Great Time to Optimize Your A/B Testing Program
Why COVID-19 is a Great Time to Optimize Your A/B Testing Program

Now is the perfect time to start or bolster an experimentation program UnSplash It’s difficult to prioritize experimentation during an…

How to Run A/B Tests with Small Sample Sizes
How to Run A/B Tests with Small Sample Sizes

Successful digital optimization programs don’t need unlimited traffic. How to deal with small sample sizes is one of the most…