SEO Intelligence Agency

@GoogleAnaltyics Tweets New, Step-by-Step Guide to A/B Split-Testing

A/B split-testing? That may sound intimidating, and while it is a process, it’s one that can be easily learned.
SIA Team
August 17, 2021

Last week, on the Google Analytics Twitter channel, there was a tweet that links to a how-to post on A/B split-testing using Google Optimize . 

If you’re not sure what split-testing is, the tweet gives you a functional definition:

What Split-Testing Is

If I were to try to define it, I’d say that split-testing (specifically A/B split-testing) is where you decide to test 2 variations of a given web page element (variant A vs. variant B, hence A/B) to determine which variation gives the more desirable outcome.

The reason why I took the effort to mention 2 variations (A/B) is because you can do 3 or more variations, which would be multivariate testing, but for the purposes of this article (and if you’re new to testing), it’s probably best to stick to  A/B split-testing. (And once you learn A/B split-testing, most of the same mechanics would apply to multivariate testing.)

“Half of What I do Works. I Just Don’t Know Which Half.”

That’s actually a quote I just made up, but it’s based on a quote I read a number of years ago. It’s based on something John Wanamaker said, and although his wording was different, the message was the same: he didn’t know what was working, and what wasn’t. 

If he did, he’d be able to do more of what was working, and less of what wasn’t.

He could reduce his spending on what wasn’t, and divert that to what was working, and all things being equal, his business would grow. 

That’s the value of split-testing: you can find out what works and what doesn’t (or what works better vs. not as well), and make similar adjustments. 

Running A Split-Test: A 6-Part Overview

Alright, now that we know what split-testing is and why it’s of value, I’d like to give you a 6-part overview of how to run a split test. While the article that Google tweeted about gives a more detailed process on how to do this, I think this one is good for getting a feel for what you’ll be doing. 

Part 1: Choose What You’d Like to Test

While there are many, many things you can test, I’d suggest picking something that might make a measurable difference. 

Also, it’s best to be true to the A/B methodology and keep it to just one variation, rather than testing a number of things at a time (which would be multivariate testing). 

Things you can test out are: 

Page layout elements (Do customers like it when this is here…or there?)

Ad layout elements (Which ad layout offers a better, less intrusive user experience?)

Headlines (This headline vs. that headline)

Sales copy (Do people respond better to the sales copy written by Jim, or Jane?)

Page length (Does a longer page convert better?)

Images (Do people like this image, or that? Here, or there?)

Part 2: Determine How You’ll Measure, and What Your Metrics Are

Google Optimize can be integrated with Google Analytics. There are also other options for measuring the stats of your web site and web page. 

You can also be more technical and get results faster: you can test certain technical elements of your page, like images, and see how many images you can have while still having an acceptable Page Speed score. (While that doesn’t fit the strict definition of split-testing, it is something to consider.)

Part 3: Measure How the Current Variation Is Doing Now

You want a starting benchmark. That’s why you should measure whatever it is you’re going to test. 

So, for example, if your current page has a X% conversion rate (whether it be sales, optins, clicks, views, etc.), you want to record that. 

Part 4: Set Up Your Test

You know what you’d like to test, and you’ve set up 2 variations of it. 

You know how you’re going to measure things. 

You’ve set your benchmark.

Now, you set up your test. Here’s how to do so with Google Optimize. 

Part 5: After Enough Traffic, Look at Your Results. Are They Conclusive?

You need to run enough traffic for your results to be statistically relevant. Is there a clear difference between the 2 variations? 

If you’ve done everything properly, you may now have a new insight. One of 3 things may have happened: A beat B, B beat A, or the results weren’t conclusive. 

Part 6: Make any Necessary Changes

With the info you now have, you can make the appropriate changes. 

I hope you can see that split–testing, particularly if it’s ongoing, can lead to increased profits and ROI.

Source: Google Analytics Twitter Channel