Calculate the Chi-Square Statistic of Conversions

Chi Square Calculator


rows
Successes: Failures:
Control:  Control: 
Recipe: Recipe:
Outputs:
Degrees of Freedom:
Chi Square Statistic:
p value:
Statistically Significant:

Frequently Asked Questions

What Is a Chi-Square Statistic?

A Chi-Square (χ2) statistic is a measurement of distance between expected and actual counts for categorical data, like conversions in A/B testing. 

In A/B testing, your visitors are categorized in two ways:

  1. Conversion Status: Converted
  2. Not Converted Recipe Applied: Control vs. Variant 1 vs. Variant 2

We use the Chi-Square Testing Calculator to determine if the applied recipe influences conversion status by measuring the distance of the actual counts from those that we would expect if they were not related. Specifically, it tests whether each recipe affects conversion differently than each of the others. 

What Do I Do with a Significant Test?

A significant Chi-Square Test tells you that a statistically significant difference exists, but not which difference or differences caused significance. The next step after identifying a significant Chi-Square Test is to test the difference between all pairs of treatments (e.g. Variant 1 vs. Control) using our Statistical Significance Calculator

Even if you can visually see what is likely to be a significant difference, it is important to run all tests to make sure no statistically significant differences are missed. 

What Does Confidence Level Mean for My Business?

If you are 95% confident that at least one recipe is different than another, there is a 1-in-20 chance that the patterns are all the same. If you roll out the statistically best recipe after doing your pair-wise tests, you will have a 1-in-20 chance of not seeing that lift.

DISCOVER OTHER HELPFUL RESOURCES

Statistical Significance Calculator

The go-to A/B Test calculator, if you know your KPI conversion rate and sample size, you have everything you need.

Sample Size Calculator

Calculate the sample size necessary to reach 95% confidence with your current conversion rate and desired lift.

Z-Score to Confidence Calculator

Using your standardized z-score, you can use this calculator to convert that score to a confidence level for either one-sided or two-sided A/B tests.

Related A/B Testing Blog Posts

A/B testing and experimentation insights and best practices.

How to Analyze Inconclusive A/B Test Results
How to Analyze Inconclusive A/B Test Results

It’s all fun and games until your A/B Test is flat-tastic. Unfortunately, the ideal isn’t always real, and inconclusive A/B…

Three Questions You Should Be Asking About Your A/B Testing Results
Three Questions You Should Be Asking About Your A/B Testing Results

Is your A/B Testing program failing to translate into business impact? Aleks Marinkovic via Unsplash @aleks_marinkovic Have you ever had…

How to A/B Test During COVID-19
How to A/B Test During COVID-19

A field guide to experimentation in spite of coronavirus and finding the new normal When the effects of the coronavirus…

Why COVID-19 is a Great Time to Optimize Your A/B Testing Program
Why COVID-19 is a Great Time to Optimize Your A/B Testing Program

Now is the perfect time to start or bolster an experimentation program UnSplash It’s difficult to prioritize experimentation during an…