A/B Testing Training for New and Existing Teams

Our A/B testing and experimentation training is designed to strategically prepare your organization for impactful A/B testing results. 

We have been on the ground floor, helping build digital experimentation cultures and functional A/B testing teams at enterprise organizations like Intuit and Vail Resorts, as well as consulting to reinvigorate optimization teams who need help with specific skill gaps or team dynamics.


Why Chose A/B Testing and Experimentation Training with Evolytics?

  • Proven curriculum designed from scratch
  • Hands-on, customized data analysis exercises
  • Real ideation workshop with your data, so you “graduate” with new testing ideas
  • Advanced A/B testing methods for experienced testing teams, such as multivariate testing, personalization, and multi-channel experimentation
  • Expertise with leading testing tools like: 
    • Adobe Target
    • Optimizely
    • VWO
    • Google Optimize
    • Split.io
    • Homegrown Enterprise Solutions


Evolytics A/B Testing Course (Sample Curriculum)

The Business Impact of A/B Testing

  • An overview of how businesses can maximize the return on investment (ROI) of A/B testing and how website optimization supports organizational goals and caring for customers.

Building Your A/B Testing Team

  • An introduction to the roles and responsibilities needed for A/B testing, and the key skills you should look for when hiring. 

Understanding the Testing and Experimentation Process 

  • We explain our proven IDEA methodology for A/B testing, which will act as the foundation for the rest of the class.

Using A/B Testing Tools

  • We pride ourselves on being able to support any industry-leading testing tool, so whether you’re using Optimizely, Adobe Target, or Google Optimize, we can give your team members a hands-on crash course in the most important features. Complicated interfaces will no longer feel intimidating.

Ideation Workshop

  • Developing data-driven hypotheses from actionable insights is paramount to A/B testing success. We lead this strategic ideation session with your team, so when you leave you not only understand how to generate experiment ideas to support your organization's goals, but you have a backlog of tests to kickstart your program.

A/B Test Development

  • We offer this foundational session to help train your team to go from an idea to a live test. We’ll focus on the basic statistics to estimate duration and design digital analytics requirements for your development teams. 

Executing an A/B Test

  • This session will showcase reporting best practices using your data tool of choice, as well as a cheat sheet for common testing statistics like z-score calculations and the business impact of alpha and beta errors

Post A/B Test Data Science Decision Tree

  • We walk you through how to use our Data Science Decision Tree to decide which advanced statistical methods will best serve your post-test analysis, balancing the likelihood of actionable insights with data analytic resources. 


“A/B Testing Statistics 101” for Experienced Testing Teams

If you already have a data-driven culture of digital experimentation, our "A/B Testing Statistics 101" training option will enable your A/B testing team to break free of reliance on tools and truly understand the business impact of test results and assumptions.

A/B Testing Traditional (Frequentist) Statistics

We offer this foundational session to help train your team to break free of simplified reliance on tools and truly understand the business impact of test results and assumptions. Understand confidence, power, and whether or not it’s okay to peek. 

We recommend allowing a one-month lead time for this session in which your team conducts an A/B test. This will allow us time to give you a Statistical Decision-Making Threshold Matrix (based on the variance levels from your most frequently used KPIs) that your experimentation team can lean on for simplified decision-making. 

Lean in to Bayesian Statistics 

If you use Google Optimize or VWO, this session is a must-have. We explain how Bayesian techniques differ from traditional, frequentist techniques for experimentation analysis and what a business stakeholder needs to know. 

We recommend allowing a one-month lead time for this session in which we run a Bayesian vs. Frequentist POC on your website, so we can measure speed to insight with each method, as well as understand the reliability of those decisions once the test is complete. 

Data Science for Optimization Programs

Data science doesn’t have to be complicated or scary, and it can definitely have a positive impact on your experimentation program and conversion rate optimization (CRO). This workshop is designed for A/B testing analysts to gain the specific skills necessary for common post-experiment advanced analytics methods and to bridge gaps between A/B testing program owners and data science teams. 

While we have off-the-shelf training from A/B testing program managers and analysts, we recommend working with us prior to the session to choose an interesting A/B test that you’re excited to learn more about. Our Decision Science team will apply our advanced analytics methods and use your data in a customized workshop, delivering one completed analysis to kickstart your data science initiatives. 

We are also able to customize A/B testing course material using your company’s actual data to make digital experimentation and testing training more relevant for your team.

DISCOVER OTHER HELPFUL RESOURCES

Statistical Significance Calculator

The go-to A/B Test calculator, if you know your KPI conversion rate and sample size, you have everything you need.

Z Score to Confidence Calculator

If you have your standardized z-score, this calculator will help you convert that to a confidence level for either one-sided or two-sided tests.

Chi Square Calculator

Determine if the applied recipe influences conversion by measuring the distance of the actual counts from those that would be expected if they were not related.

Related A/B Testing Blog Posts

A/B testing and experimentation insights and best practices.

How to Analyze Inconclusive A/B Test Results
How to Analyze Inconclusive A/B Test Results

It’s all fun and games until your A/B Test is flat-tastic. Unfortunately, the ideal isn’t always real, and inconclusive A/B…

How to Run A/B Tests with Small Sample Sizes
How to Run A/B Tests with Small Sample Sizes

Successful digital optimization programs don’t need unlimited traffic. How to deal with small sample sizes is one of the most…

Experiment Nation features Evolytics’ Kenya Davis
Experiment Nation features Evolytics’ Kenya Davis

Kenya Davis, Senior Manager of Decision Science, was featured in “A Panel Conversation on Experimentation Program Goals” Experiment Nation‘s mission?…

Building a Better A/B Testing Process
Building a Better A/B Testing Process

A strategy for building successful experimentation programs Leading digital experimentation tools such as Optimizely, Adobe Target, Google Optimize, Split IO, and…