Challenge

Vail Resorts was starting the journey toward a test and learn culture in support of long-term omnichannel personalization goals. Although they had an excellent ecommerce team and dedicated digital analysts, they lacked experimentation experience. 

This partnership was truly about building a testing culture from the ground up. We began with our Optimization Program audit, something that uses a mix of Microsoft’s peer-reviewed Experimentation Evolution framework and our own business consulting experiences to discuss Vail Resort’s current program state and growth goals.

Solution

We worked with a small team of leaders to build a Digital A/B Testing team of developers, analysts, marketers, and project managers. We helped ideate, develop, execute, and analyze a variety of A/B Tests, multivariate experiments, and personalization experiences. The primary objective was not a successful year of conversion rate optimization (CRO) wins, but a well-trained team that could move forward on their own. Of course, a couple key customer insights and revenue wins wouldn’t hurt as our stakeholders were working to prove the value of an A/B Testing program. 

Given the Vail Resort team’s teach-us-to-fish mentality, we collaborated closely, building a centralized experimentation playbook with everything from Adobe Target best practices to communication email templates and conducting on-site training.

Results

Since the original partnership began in 2017, the Vail Resorts Digital A/B Testing team has grown, both in size and in capabilities. We revisit the Optimization Program Audit annually to measure success and assess next year’s growth goals. Our partnership has produced some big wins along the way. 

  1. Our first multivariate experiment garnered a multi-million dollar increase in revenue 
  2. 100+ page Experimentation Program Playbook using the team’s existing Jira and Confluence setup 
  3. Team was able to run winning multi-channel personalization experiments during second season

Related A/B Testing Blog Posts

A/B testing and experimentation insights and best practices.

How to Analyze Inconclusive A/B Test Results
How to Analyze Inconclusive A/B Test Results

It’s all fun and games until your A/B Test is flat-tastic. Unfortunately, the ideal isn’t always real, and inconclusive A/B…

How to Run A/B Tests with Small Sample Sizes
How to Run A/B Tests with Small Sample Sizes

Successful digital optimization programs don’t need unlimited traffic. How to deal with small sample sizes is one of the most…

Experiment Nation features Evolytics’ Kenya Davis
Experiment Nation features Evolytics’ Kenya Davis

Kenya Davis, Senior Manager of Decision Science, was featured in “A Panel Conversation on Experimentation Program Goals” Experiment Nation‘s mission?…

Building a Better A/B Testing Process
Building a Better A/B Testing Process

A strategy for building successful experimentation programs Leading digital experimentation tools such as Optimizely, Adobe Target, Google Optimize, Split IO, and…