A/B Tests: Guide on Split Testing

Is split testing useful and how to conduct it? We hereby prepared a complete guide on the topic with Q&A about A/B testing at the end of the article.

A/B testing is all over the news:

• Which header is better for Yandex.Zen: this or that one?
• I don’t know, conduct an A/B test and check the results.
• Should I make a creative with a blue button or a green one?
• Do a split test and check the results.

Designers, copywriters, editors, typesetters, publishers, and marketing specialists — all refer to A/B testing as a universally applicable approach to answering the question: “What’s better?”

What’s A/B Testing?

A/B testing (split testing, a-b test, AB-testing) is comparing the performance of two or more options. During testing, they check which option performed best by one of the following parameters: read-through, feedback, conversions, time on the website, etc.

For example, you must find out which creative has a higher CR. To do this, you must make two different creos and run them for the same audience, and then compare which one provided more clicks, purchases, etc., and redirect your master budget to it.

Who Needs A/B Tests & Why?

Split testing is helpful for anyone who interacts with websites or apps: webmasters, SEO specialists, analysts, copywriters, marketing specialists, and administrators.

To conduct testing, you do not require any expertise: the basic knowledge provided in this article will do.

With split testing, you can check any page element, either header or page layout. The prime target is to improve certain indicators like usability, conversions, failure number, click count, and so on.

Case study: the number of failures on your landing page reaches 99%, which means that the user leaves the page within 2-3 seconds, and your ad campaign is unprofitable. Thus, you might want to consider the fact that something is wrong with the page and conduct an A/B test.

Which Elements are Commonly Tested?

You can test any element of the website, creative, mailing list, text, or even banner. But most often, they test the following elements:

• Headers and text. This applies to both creatives and landing pages. If the headline does not catch an eye and the text is unsaleable, then the CR will be low, and an ad campaign will be unprofitable.
• Call to action. A weak CTA is a bad CTA, and even a single replaced word can affect clicks on your creo, casino sign ups count, or purchases from the landing page.
• Multimedia elements.Aggressive creatives with images of money, expensive cars, and landing pages where they promise you billions of dollars might sometimes work. But it also might discourage the user and force him/her to leave the page. By the way, they not only test images on creatives or the landing page design elements but the format of the creative as well: photos/videos/audio/animation.
• User feedback.70% of consumers purchase goods based on user reviews but at other times the feedback won’t inspire confidence and discourage the consumer.

Important: do not change several parameters at once. For example, if you want to conduct AB testing of a creative for Facebook to find the best headline, then work with the titles solely and leave all other elements untouched.

How Do I Conduct Split Testing?

Step 1. Decide on the element to test.

As we’ve already mentioned: conduct one test per element at once.

Step 2. Set your goal.

Decide on the goal you are pursuing, like increasing the click count on the creative, increasing the landing page CR, reducing the cost per click, etc.

Step 3. Analyze the raw data.

For this, you can use any tracker or statistics provided by the affiliate program to compare the results with the original metric.

Step 4. Select the element to test.

It might be, for example, the title of the push notification, description, icon, or CTA on the landing page. You can select several elements and determine the queue of split testing.

Step 5. Create options.

In case you test the CTA, then change the color of the button or its position on the page. If you test the titles, then select a second title that matches your creative.

Step 6. Select A/B testing tools.

The majority of specialists opt for Google Analytics, but you can also use CrazyEgg or Optimizely for websites.

Step 7. Create a test.

Set up the main parameters and the testing time. Some services allow you to specify the type of device, the location of users, and other criteria.

Step 8. Collect data.

It’s a “trigger mode” of sorts. At this step, you might not want to jump to conclusions after the test results and make hasty changes. Wait for the test to complete.

Step 9. Analyze the outcome of A/B testing & choose the best option.

Compare key indicators and choose the option that proved to be the most effective.

Tools for Split Testing

There are a vast number of tools that are distinguished by their cost, feature set, mastering intricacy, metrics that can be tracked, and other criteria.

Hereby, we will inform you of five tools for A/B testing: some are best for landing page testing while others are best for creatives.

Google Marketing Platform

https://marketingplatform.google.com

It’s universal solution beginners often choose to optimize landing pages. The service is free but you’ll require a Google Analytics account to enable it.

Optimizely

https://www.optimizely.com

The service is geared to split testing exclusively with a user-friendly interface. It resembles a graphic editor which you can customize to your liking. It includes the WP plugin which simplifies operations with Optimizely.

Visual Website Optimizer

https://vwo.com

It allows you to test mobile and desktop website layouts and 15 various parameters. Visual Website Optimizer offers a customizable interface and Google Analytics integration. It’s best for AB testing of websites or landing pages.

Facebook

The social network allows you to conduct a split test of the creatives you use. It’s free and rather convenient. To conduct A/B testing on Facebook, you must select a campaign and click “New Campaign Split-Test”.

myTarget

You can also conduct A/B testing in myTarget. To do this, create an ad campaign, copy it, and go to the “A/B test” block. It allows you to segment the audience.

Common Mistakes in Split Testing

Seven mistakes affect A/B testing:

1. Changing several elements and testing them all together. If you do so, you won’t be able to figure out which changes worked and which didn’t.
2. Using someone else’s hypotheses.You must consider the specifics of your offer, website, creative, platform, target audience, and so on.
3. Finalizing test too early.The optimal test duration is 14 days for the landing page and three to four days for creatives.
4. Conducting one test. You might want to conduct regular A/B tests if you want to change something.
5. Ignoring externalities.User behavior may depend on the situation in the country, holidays, seasonality, etc. Opt for stressless periods for testing.
6. Insufficient metrics tracking.You must monitor the main and secondary indicators to evaluate the performance reasonably.

These mistakes are common for beginners but in time you’ll accustom yourself to the correct split testing approach.

F.A.Q.

Q: When should I opt out of A/B testing?

А: In case you have a moderate user flow. For example, when you run ads with a small budget.

Q: Are there other options except A/B testing?

A: There are three options: usability testing, Fakedoor testing, or testing with a limited audience.

Q: How long does the A/B test take?

A: 14 days are optimal but it may take longer.

Q: What should I do if I need to check more than two options?

А: Conduct multi-variant testing. It follows the same principle as the A/B test, but it allows you to compare more than two options at once.