Test Like A Formula 1 Team
All marketers know about A/B testing (if you don’t you should read this blog post for a primer).
A/B testing can be really easy to do - in fact, most digital ad platforms allow you to upload two different sets of creative and will automatically choose which ad converts better. This works especially well if you can put some budget to work for you.
Of course, you can A/B test for free as well. Using tools such as Bitly or Google Analytics, you can create two Facebook posts - perhaps with the same copy but different images or the same image and copy but a different call to action - and find out which one had a better clickthrough rate or a higher conversion. You can even A/B test content on a webpage with Google Optimize!
The Things You Can’t Control
A perfect A/B test changes only one variable, but in the real world, multiple variables are always in play. For the Facebook post mentioned above, the time of day or the day of the week might have made a difference. If the ‘A’ test ran on a February week with extreme cold, and the ‘B’ test ran the next week when the temperatures were much warmer, your test may have been impacted by an unknown (or unmeasurable) variable.
And of course, don’t forget about wanting to test something that may not have easy built-in reporting metrics, such as a radio campaign.
In that case, you can copy Red Bull, Mercedes-Benz, and Ferrari by trying ABA testing.
What Is ABA Testing?
Auto racing teams test. They test a lot. Tire compounds. Spring rates. Fuel mixtures. Wing shapes. On a good track day, the car will look drastically different at the end of the day from how it left the garage at the beginning of the day because many things will have been tested and changed.
A/B testing is great however, as the name implies, A/B testing is about testing A against B - one variable at a time. Racetracks are outside and many variables may change that the engineers didn’t plan for.
The temperature and humidity will change throughout the day. The track surface will change. The wind direction and speed might change. And the driver will change too: they’ll gain or lose confidence in the car, they’ll get fatigued or energized, and their hydration levels will change. All of these factors can impact how quickly the car goes around the track.
How do the Red Bull engineers know if the car was faster because Max Verstappen got more confident around turn 11, or if the car was faster because the new front wing provided just a bit more downforce? If the new suspension setting was better, or if it was cooler temperatures that provided a denser charge and thus more power?
They go back to the beginning.
On a regular basis, they will change the car back to the original setup and see how fast it goes. If it slows down with the original setup they can be assured that the tests they ran actually made a difference in the speed of the car. Those successful changes make it to race day.
And what if the car speeds up with the original setup or maintains pace? Then it becomes clear that an unknown variable was responsible for the speed increases during the day. They’ll know to dig into the data and find out what was really happening.
Unknown Variables In A Test
Cars (like businesses and marketing campaigns) are a system. A change that you make in one place may have ripple effects in other areas. External forces can influence your results without you being aware of them.
Whether your goal is a smoother operating company, better marketing returns, or a faster lap, ABA testing can help you get there. It can be humbling to put in a bunch of work and find out that it was all for naught and that some other variable was determining your results. But much like a science experiment, you’re not actually out to prove your assumption.
Your goal is not to be correct, but to remove every other possibility so that the one answer you’re left with is absolutely the right one. The goal is to learn and understand.
If you can reasonably control all the variables in a test and change only one, then A/B testing is for you. If there’s even a tiny chance that there are unknown variables involved, then consider using an ABA test instead.