How to run your very first A/B (split) test

You’ve been watching your analytics carefully. You know you have enough traffic to your website, and you have an idea you’ve been itching to test out.

Now, all you need is a beginner’s guide to running your very first A/B (split) test!

Luckily for you, I’m going to give you exactly that in this blog post. 😉

How to stop guessing what will work best on your website and get a definite answer

If you’re wondering what an A/B test is or why you should care, here’s a quick summary for you: A/B (or split) tests help incrementally improve the conversion rate of your website – ultimately making you more money. They’re fairly simple to set up and don’t have to cost anything to run, though you can use some really cool split-testing software that’s paid if you’d like.

Basically, A/B tests send part of your traffic to one version of your page and part of your traffic to another version of your page. The first version is the “control,” or your page as it currently looks. The second version is the same as the first version, with ONE element of the page changed. It’s important to only change one thing at a time, or you can’t be sure which change caused the drop or increase in conversion rate.

Step #1: Decide what to test.

Remember, only test ONE change at a time, whether it be copy, color, size, or something else. For example, if you want to test out changes to a button on your sales page, make sure you only change one element of that button—its color, size, text styling, or copy—at at time.

If you aren’t sure what to test, take a look at your analytics (you have been looking at your analytics, haven’t you?) and see where users are leaving your site or where there seems to be a bottleneck in your funnel.

Step #2: Set up two versions of the page you’ll be testing: the control, and the version with the change applied.

This one’s pretty self-explanatory. Duplicate the page you want to test, then make the change on the new copy of your page.

Make note of the URL of your control page and the test page; you’ll need those soon.

Download the Optimize A/B testing workbook

Step #3: Sign in to your Google Analytics account.

We’ll be using Google Optimize to run our A/B test, a part of the Google Analytics suite. Optimize replaced Google Content Experiments, which replaced Google’s Website Optimizer. The free version of Optimize will be more than sufficient for our first test – no need to worry about upgrading to Optimize 360 yet.

Step #4: Set up your Google Optimize account.

1. With your Analytics account open, click the large account name beside the Analytics logo in the top left hand corner of the screen. This will open a modal window with access to the entire Analytics Suite. The window will automatically be opened to the “Analytics” panel.
   
2. You should select the “Optimize” panel (two icons to the right of the Analytics icon inside the modal window – the Optimize icon looks like two boxes overlapping each other and will turn red when you hover over it).
3. Click the “get started” button inside the Optimize panel.
   
4. A new tab will open to walk you through the Optimize set-up experience. Select the options that you’d like on the next two screens (they won’t affect the rest of this tutorial), then you’re all set up and ready to go!

Step #5: Configure your test.

1. You should be on the Optimize homepage of your newly created container (Google’s way of saying “part of your Optimize account”).
2. There’s a blue “Create Experiment” button toward the right side of the page – click it!
3. A flyout will open on the side of the page, prompting you to name your new experiment. Use a name that relates to the test you’ll be running: for example, “homepage button color test”.
4. For the URL you’ll be testing, enter the URL of the “control” or original version of your page.
5. Select “redirect test” from the available options. This is a little confusing, I know – but Google Optimize’s A/B test requires you to use their editor instead of making changes on your website itself. You can always experiment with their editor later, but for now, let’s keep it simple and perform a redirect test. 😉
6. Once you have everything filled out, click the create button to start configuring your test.

Step 5a: Link your container to your Analytics account

1. Once you have your first experiment created, you’ll need to link your Optimize container to an Analytics account. There’s a light gray box in the experiment sidebar to the right that says much the same. Click the link inside that box to go back to your container homepage.
2. In the container homepage sidebar, you’ll see a container checklist. Step 3 of this checklist is what we’re looking for right now. Click the “LINK PROPERTY” button underneath that step.
3. In the dropdown menu, select the Analytics property you’d like to link this container to, then hit the Link button.
4. A message should pop up prompting you about installing an Optimize snippet on your website, but you can click “Skip for Now;” we’ll address that later.

Step 5b: Create your variant

Adding a variant to the experiment tells Google where the “test” version of your page lives.

1. In the list of experiments to the left of the sidebar, select your new experiment.
   
2. On the experiment main page, click the “Create Variant” button in the Variants row.
3. Enter a name for your variant (again, explanatory = good), then the URL of the test page that you noted earlier.
4. Click the “Add” button, and you should see both the original version of your page and the test version of your page listed in the Variants section.

Step 5c: Set your objective

Setting your experiment objective lets Google know what you consider important in relation to this test.

1. In the Configuration section in the Objectives panel, click the “Link to a Google Analytics view” button. Select which view you’d like to use with this experiment, then click the Link button.
2. Now, the “Link to a Google Analytics view” button is replaced by an “Add Experiment Objective” button. Click that button and choose either “Choose from list” if you already have goals set up in your Analytics account that you’d like to use as the objective, or “create custom” for a new goal.
   Ex: For my test, I’m trying to determine whether a gallery-style listing of blog posts on my blog homepage gets more click throughs than a summary-style listing. So, I’m going to select “choose from list,” and “Bounces” to monitor how many people bounce off my blog homepage without visiting any other pages.
3. You can also set up to two additional objectives for your test. These help you gather more data in relation to the test, but won’t affect the test’s results. I’m going to add additional objectives of Pageviews and Session Duration since they’re also related to my original objective.
4. When you’re done adding objectives, enter a description of your test and what results you think you’ll get in the Description and hypothesis box to the right of your objectives, then click Save in the top right corner.

Download the Optimize A/B testing workbook

Step #6: Add the experiment code to your pages.

Before we actually start running our test, we need to add the Google Optimize code to our website.

1. In the top left hand corner of your screen, click the back arrow next to the name of your experiment to go back to your container settings page.
2. In the container setup sidebar on the right, find the light grey box titled “Add Optimize to your website.” Click the Install Optimize button inside this box.
   
3. A modal window will pop up with installation instructions, explaining how you should modify your existing Analytics script to enable Optimize on your site. Installation involves copying a little line of JavaScript code into your Analytics script; this line of code links your website to your Optimize container.
4. Once you’ve properly installed that line of JavaScript into your Analytics, click the Next button to go to installation step two. Step two helps you provide a smoother experience to your site users while running a test. It’ll give you a short snippet of CSS and JavaScript that you should add directly before the Analytics code on your pages that will be affected by your test.
5. After you’ve finished step two, click back over to your test’s main page. In the sidebar, you should see a message prompting you to run diagnostics on your Optimize script; click the Run Diagnostics button to get started on this final step.

NOTE: Occasionally, the diagnostics script will report false errors for redirect tests. For example, I kept getting an error that my Analytics script wasn’t installed on my control page, even though I could see the script in the page’s head and it was sending data to Google Analytics. If you know that your script is installed correctly but you’re still seeing an error here, you should be okay to proceed to the next step.

Step #7: Run your experiment.

It’s finally the big moment! On your test’s main page, click the “Start Experiment” button. This tells Google to start running your test, splitting up your traffic as you’ve dictated in the variant section. Now, all you have to do is sit back and wait on your results!

Note: If you encountered an error during diagnostics but started your experiment anyway, keep an eye on it for a day or two to make sure your test is actually recording results as it should.

Step #8: Review your experiment and make the winning page version live.

Once your test has run its course, you can log in to your Optimize account and review the results. Hopefully, you received enough traffic to reach the point of statistical significance in your test and to determine a winning variant.

If so, review which version won the test (and don’t forget to take a peek at your alternate objectives, too!). If the “test” version of the page was the winner, make sure you change the control version of your page to match it so that the better page version gets served to all of your site visitors.

Step #9: Rinse & repeat!

If you’d like, you can set up more tests right away; you can run five at once in the free version of Optimize (but make sure they’re all testing things on different pages so you don’t mess up any other currently running tests).

It may sound hard or complicated when you first read through the instructions, but using Google Optimize to run A/B tests is fairly simple once you get the hang of it! After your first couple of tests, you’ll be a lean, mean, conversion-optimizing machine. 😉

Get rid of your fear of A/B testing once and for all with this handy workbook

The Google Optimize A/B testing workbook guides you through setting up your A/B test and includes a place to plan/track your future A/B tests. So handy!