Using Chi-square Analysis for A/B/N Tests

Evolytics Scott Sanders speaks at Experiment Nation

On September 29, 2021 Evolytics Scott Sanders spoke at the Experiment Nation conference on the following topic.

Although the two sample proportion test is well known within the marketing experimentation community, there are a host of categorical data analysis techniques that are amenable to the analysis of A\B test data. This presentation provides a quick overview of Chi-square analysis and a discussion of its use cases in marketing experimentation. Specifically, we discuss how to:

  1. Test a\b\n experiments without inflating Type I error after multiple comparisons
  2. Analyze tests with more than dichotomous outcome variables
  3. The process for calculating goodness-of-fit and tests-of-independence

Simple guidelines will be provided regarding test assumptions and use case scenarios.

The Chi-square calculator from his presentation can be downloaded here:

Download the Chi-square Calculator

Looking to do more with your testing and experimentation?

LET’S TALK.