How to Get Started with A/B Testing

Published 08-11-2021

Any business must constantly make marketing decisions. Which campaign tagline is better, which post design looks more appealing, or what are the best words to describe what you do?

Ultimately all of these choices will be geared towards a single goal, increasing revenue, so it’s important to identify the right option as quickly and effectively as possible.

One way to do this is to perform A/B testing (also known as split testing). Let’s take a quick look into what it is, why including it into your marketing activity it is a smart choice, and some tips to do so without pulling out your hair.

What is A/B Testing?

I’ll start with an analogy: if I were to compare the action of A/B testing to a more familiar scenario, I could compare it to picking a wedding cake. You pick a bakery because you like what they do and you think it will be right for your wedding, but now you have to pick one out of all the cakes they make.

What you’ll usually do then is sit down to taste a few samples, and begin eliminating the ones you don’t like so much, leaving only the samples that are best for you.

Similarly, when it comes to A/B Testing is comparing two (or more) strategies to pick the best one for your marketing goals, with the difference that all decisions are based on collected data.

Let’s say, for example, that you have two landing page designs. You could divide your audience it two groups, and show one design to each group. The page that gets the most positive audience response is the option that will work best for your business.

When running tests, you can compare anything from small details like a call to action or the design of a button, to whole pages, messaging, or campaign theme.

A/B Testing is usually ongoing, and identifying a winner of two options can inspire further tests, constantly comparing the top ideas in order to continuously improve results.

Why it’s smart for you to start A/B testing

Well executed A/B testing campaigns will not only help improve performance of your website or marketing campaigns, but also save time and money in the process.

Going back to our wedding cake analogy, when trying your options, you’ll probably taste from a collection of samples, and not go through whole cakes in order to pick one, that would be wrong.

Delicious, yes, but also wrong.

Testing things other than cake shares a similar idea. Let’s say you’re running ads and want to know which of two ads will deliver more leads over a period of one month using the same budget of $100, and same audience targeting.

Running both ads one after the other would take 2 months and cost $200, and what’s more, conditions might vary from one month to the next, so the results you see after completing both tests might not be the same results you would have gotten while running them simultaneously.

While probably obvious at this point, you could very well split your audience run both tests at the same time to cut both execution time and cost in half, and have them run in the most similar conditions possible.

See? Smart.

Note: Most advertising platforms these days will offer some sort of built in A/B testing feature, and when it comes to your website or landing pages you can use tools like Google Optimize to comfortable run tests yourself.

How to Build Your A/B Testing Procedure

Choose What to Test

Select the two variants that you’ll compare. While you can choose to test more than two options at once (A/B/n tests), if you’re just starting out or you have a relatively small audience then it’s best to start with just the two to make sure your results are statistically significant.

Make sure that your tests are variants of the same thing, such as two headlines for the same section, two color schemes, or two versions of the same call to action. Basically, avoid comparing apples to oranges.

One last suggestion, starting out with larger tests such as the design of a whole page rather than that of a button will allow you to start with broad strokes and point you in the right direction in terms of what works better with your audience.

Designate Your Audience

Make sure your split audiences are identical.

Don’t test one region vs another, for example, split your audience down the middle in every regard. Use the technical tools available to you to make it easier to fairly and randomly split your test audience.

One exception here would be if you’re specifically testing to compare the performance of two different audiences, and in that case, what should remain the same is your content (your website or your copy).

Determine Your Stats and Success Conditions

Decide what will determine the winning variant. Will it be the one that gets the most conversions, clicks, messages?

Every business might measure success differently, so have in mind what metrics mean success to yours before you begin testing.

That being said, try keep an open mind about what else you might learn from studying the test results. Insights into your audience are valuable and should always be welcome!

Set a Time Limit

Don’t run your tests forever, set a limit to more quickly iterate and incorporate your findings into the rest of your marketing plan. Each new test should reveal a finding that other areas of your business can benefit from to improve as well.

So what now?

Need help getting started with A/B testing? Want to share cake samples with us? We’re and happy to chat! Contact us for a free consultation.