Now is the perfect time to start or bolster an experimentation program


It’s difficult to prioritize experimentation during an economic downturn. Afterall, it feels like a nice-to-have rather than a run-the-business necessity. However, this inclination will be harmful in the long run, and not just because businesses who invest in themselves during recessions come out stronger than those that do not.

Coronavirus is changing consumer behavior, and the more quickly you’re able to adapt, the more quickly you’ll improve your bottom line at a time when consumers are more carefully scrutinizing their purchases.

Defining COVID-19 as a Time Period

On January 7, 2020, China reported a new type of coronavirus. On January 30, the World Health Organization (WHO) declared a global health emergency. The WHO labeled the coronavirus outbreak a pandemic on March 11, and two days later, the United States declared a national state of emergency. By March 31, nearly 80% of Americans were under some form of stay-at-home order, leading to economic crisis and financial insecurity. As brick-and-mortar businesses shut down and enterprises became 100% remote overnight, businesses realized that they needed to adapt. Online businesses have been able to continue their operations with their employees working remotely. They are all still able to collaborate as a team by using Microsoft Teams, for example. Some businesses have even been able to implement direct routing for Microsoft Teams to allow external calls to the business. That is helping businesses to continue working and growing.

In May, the number of global COVID-19 cases surpassed five million. In June, the case number skyrocketed to 10 million. As I write this post on July 30, the 6 month anniversary of the WHO declaring a global health emergency, I keep my thoughts to the silver linings, and I am in awe at how adaptable humans can be.

The news cycle serves as a constant reminder, even to those of us who are privileged to still have our health and our employment, that our future is still unknown. We learn of promising medical breakthroughs one day and learn of someone from our inner circle testing positive the next. We hear the housing market is stable, strong even in some areas, only be reminded in the next segment how many Americans are currently without health insurance, and how they are worried and need answers to questions like can medicaid take your house? Which is even more relatable if they have just entered a care home and are worried about their finances for outgoing payments. Although going back to the housing market, according to Jacob Realty and other estate agencies, the process has changed despite the fact that the market remains stable.

The problems aren’t all at a macro level; parents are unsure what back-to-school looks like, and many workers don’t know when they’ll be allowed back in the office. Humans are generally more territorial and each one needs her or his space. We are all struggling with our privacy and convenience in consumption, according to the Journal of Business Research. It’s safe to say stress levels are high.

Why the State of Constant Change During COVID-19 Matters

With so much uncertainty, all consumers are acting in a state of shock, and as they settle into routines six months in, the constant push to get back to normal means few habits are really set in stone. This is especially advantageous for brands who have more to gain by increasing market share than they have to lose through retention.

Traditional consumer behavior research finds that brand preferences rarely change over the course of a consumer’s life. The exception to this rule, is of course, major life changes – consumers are most willing to try new brands or products when they go through major events such as graduating high school, becoming a homeowner, having a child, getting a big promotion, or moving to a new city. I don’t need to remind you that, right now, literally everyone is going through a major life change, and those consumer dollars are up for grabs.

How COVID-19 is Changing Consumer Behavior

Toilet paper jokes aside, American consumers learned hard lessons about stockpiling and disaster preparedness, according to Sabrina Helm, associate professor at the University of Arizona. Few of us could previously imagine walking into a grocery store and not seeing shelves fully stocked of every product imaginable. She predicts that new consumer normal will mean keeping a safety stock of key items. Overall, US consumers are:

  1. Drastically reducing discretionary spending
  2. Increasing their online buying, especially groceries.
  3. Adapting to shop online, pickup in-person models. This is absolutely necessary considering the times we live in.
  4. Stockpiling key items such as hand sanitizer and toilet paper. You could find wholesale toilet paper online.

Experts tend to agree that, overall, people will go back to their pre-coronavirus behavior patterns. However, there could be visible changes in the way people view and buy things. The exception being when alternatives are found that are more accessible, convenient, or affordable…which means, all that new market share you gain by having a user-friendly and remote-accessible product or service during COVID-19, isn’t going to disappear the moment it ends.

Why You Should Test Against the News Cycle

Nielson identified six consumer segments, detailing how these consumer groups are responding to the novel coronavirus. Most consumers will be a part of multiple consumer segments as they progress in their COVID-19 lifestyle journey. The central theme in all six segments providing early indicators across global markets? The news cycle.

When our own analyst team found that client key performance indicators seemed tied to the COVID-19 news cycle, we created the COVID-19 Calendar dataset, and we made it free to all analysts. The COVID-19 Calendar is designed to help businesses and analysts investigate the impact of related events on key metric trends such as website traffic, orders, and conversion rates. I hope you choose to correlate your performance to positive or negative events and test against it as you learn about the new normal for your customers.

The news cycle has a predictable impact on consumer behavior and mindsets during a pandemic. If your team is able to move nimbly and test against new cycle vibes, you’ll likely race past your competitors in terms of understanding new consumer mindsets and adapting to them accordingly.

How to Develop a COVID-19 Testing Roadmap

It’s highly unlikely your current roadmap and associated testing strategy applies here. Whether you’re plagued with frozen budgets, traffic shortages, supply chain unreliability, inconclusive results, or the lack of a testing culture, it’s likely that your business situation has changed as much as your consumer’s situation.

  1. Sit down and think about where you want your business to be when the pandemic ends. Define what it will look like to simply survive, and, more importantly, define what it will look like to thrive. This will be your North Star.
  2. Once you have your North Star, write down every assumption to get there. What needs to be true in order to reach your goals?
  3. Identify which assumptions are the riskiest leaps of faith. These assumptions are now the first draft of your test hypotheses – figure out a way to prove or disprove each assumption and make the leap just a little less scary.
  4. Don’t forget that your customers are people. While you’ve identified your highest priority projects, don’t forget to add simple messaging or UX best-practice updates that may better reflect the current state of consumer culture.

What’s most important to remember is that the pandemic did not happen overnight, nor will it suddenly end. Getting back to normal will happen slowly, so you’re not wasting your time getting to know your customer’s COVID-19 mindset. The mindset may change as often as the news cycle, but it’s here to stay for the foreseeable future. The businesses that grow and adapt rather than stagnating for months will have a much more optimistic new normal to look forward to.

COVID-19 is a Great Time to Optimize Your A/B Testing Program

Now is the perfect time to be testing. Numerous studies on the long term implications of the 2008 recession show that businesses who invested intelligently rather than stagnating recovered more quickly and gained market share.

Like the Great Recession, COVID-19 will have a profound impact on consumers. What you’ve learned thus far may or may not stand — now, or when society believes the pandemic is over. The more quickly you’re able to meet your customers and acquaint yourself with their new normal, however fleeting it may be, the more quickly you’ll improve your bottom line at a time when consumers are more carefully scrutinizing their purchases.

Savvy marketers are leveraging the predictive capabilities of the news cycle and building coronavirus experimentation roadmaps.

The Evolytics team of A/B Testing and Decision Science experts would love to help you make lemonade out lemons. If you want help developing a COVID-19 Testing Strategy, reach out!

Written By

Krissy Tripp

Krissy Tripp, Director of Decision Science, strives to empower her clients to make use of their data, drawing from a variety of disciplines: experimentation, data science, consumer psychology, and behavioral economics. She has supported analytic initiatives for brands such as Sephora, Intuit, and Vail Resorts.