Contact us

1 (888) 505-5689

A/B Testing: What It Is and How to Do It

Do you ever sometimes wish you could see the future to know whether or not your marketing efforts are going to work? Well, guess what? You can! You don’t even need to be bitten by a radioactive marketer to do it! To achieve this you need to collect and analyze data through an A/B test.

What Is A/B Testing? 

Image via Flickr by ota_photos

Even though the name sounds scary and complicated, A/B testing is not as complex as it may seem. To put it simply, A/B testing is when you show small portions of your target audience slightly different versions of your marketing materials so that you can compare their effectiveness and choose the best version. For instance, you might come up with two different subject lines for an email, then send version A to half of your subscribers and version B to the other half to see which results in more views.

Why Would I Want to Do This?

A/B testing allows you to test changes that may increase the effectiveness of a piece before you put every last dime that you have into the project. It helps you refine your marketing materials and maximize their impact, which in turn boosts your bottom line.

When Should You Do An A/B Test?

Technically, you can do an A/B test for anything you want, but the more risk involved, the more you will gain from taking the time to run the test. You can run an A/B test for an expensive ad. Or you might want to perfect your marketing materials for an upcoming event or holiday. You can run this kind of testing on everything from button colors to headlines to full layouts.

How to Do It

Step 1: Pick One Variable to Test

It is important that you only pick one tweak to make per test so that you know exactly what variable is causing the difference in your metrics. You can test more than one variable over the course of your A/B testing, but you will have to do them in separate tests. Again, the variable can be something like a headline, a call to action, or maybe even the use of a specific photo or color.

Step 2: Decide on Your Goal

The next step is to decide your goal. Are you trying increase revenue? Maybe you just want to drive traffic to your site, or you could want people to perform a certain action on your site. Regardless, you will want to keep your goal in mind and ensure that you are monitoring the appropriate metrics. In this stage, you can also form a hypothesis regarding what you think or hope will happen if you make certain changes.

Step 3: Make Your Test Materials

Now it’s time to actually make the change you want to test. Make sure that you don’t make any changes besides the one you are testing. You don’t want any other factors to distort the results of your test.

Step 4: Determine Who You’re Going to Test

Test materials, check. Hypothesis, check. Now it’s time to determine your sample groups. A lot of this will stem from what you know about your target audience. Factors such as demographics, medium, and channel will come into play when selecting your sample group. If you are testing an online ad, then this process will be made simple for you, but if you are trying to test a printed material or an email, then this may be more complicated.

Step 5: Decide How Many People You’re Going to Test

In theory, you would want to collect enough data for your test to be “statistically significant” (which means the findings are reliable), but things don’t always run that smoothly. Regardless, we’re left with the question of exactly how many people that is. Luckily, the internet is full of free resources, including one that will figure that out your ideal test size.

For example, if you are a small company wanting to A/B test an email, you may only have 200 people on your mailing list. In order to be statistically significant, you would need to send it out to over half your mailing list. That defeats the purpose of doing the test. In this case, doing a percentage of your audience is the best course of action.

Step 6: Determine How Long You’re Going to Test

You know what you’re testing. You know who you’re testing. Now how long are you going to run the test? All of the factors we’ve mentioned before will influence this decision. If you are testing a large group of your target audience, it will take longer than a smaller group. Some channels have longer delivery times, and some actions take longer to complete than others. These are all things you will want to take into account when deciding a runtime for your test. Keep in mind the amount of time needed to complete what you’re testing for.

Step 7: Run the Test

When you’re ready to run the test, make sure that you are running both the control version and the tweaked version at the same time. If you don’t, factors such as time may skew your results. A number of different services can be used to make this easier, or you can do it yourself.

Step 8: Analyze the Results

Once the appropriate amount of time has passed since your target audience was exposed to your test materials, it’s time to see how version A and B did. Look at the key metric that you chose in step two and compare the results. Don’t look solely at that metric, though; see if there are other insights you can glean, even if the test didn’t go quite as you had planned.

By following eight simple steps, you can gather information that will enable you to predict the success of current and future campaigns — no superpowers necessary! And if you need a little assistance, you can always turn to Copypress for the high-quality content your business needs.

About the author

Sarah Thompson