AB Test Checklist: Everything You Need to Check Before Launching

Apr 20, 2015

Building a conversion optimization strategy is a process. You’ve spent time on tracking, analyzing, building a hypothesis and building your test. Moments before you launch, take the time to go over the following AB test checklist to make sure you have everything you need and to ensure your test results are meaningful and scalable.

Everything you need to do before launching your next test:

The first thing to when you decide to start testing is ask yourself where you’re going to test. That’s where planning comes in. Usually you want to start where you can both have the biggest impact and also learn the most. Oftentimes, this is at the top of the funnel where there are the most users, but take a look at this section and see what works best for you.

  • Measure your website’s performance (Read more about where to start and what to track)

  • Use heatmaps to understand user behavior (Read more about heatmaps to increase conversion)

  • Define your goal – (e.g – registrations, sales)

  • Define KPI’s – what will you track?

  • If you have other tests planned or ongoing – would yours conflict?

  • How long will it take you to reach significance? Make sure to chose a part of the funnel that will generate results and matter.

  • Choose the best platform for your test

The Creative Checklist1 Landing Page optimization

The design process of a test is where you make the difference. Knowing what to test and how will be the difference between a successful or failed test. The steps you take to optimize your assets will determine if your test results in relevant or non-essential conclusions. Here’s what you need to make sure of before you complete your new variation:

The Call to Action Button:

  • You have one call to action (CTA)

  • Your CTA is above the fold

  • The text on your call to action contains no more than one or two words

  • The call to action button stands out – in color and size

  • Your CTA is clear – people will know what to do immediately

The Content:

  • The content focuses on value – not features

  • Your headline matches your message on ads

  • You’re using one font type

  • Your title is actionable and tells people what to do

  • Your unique selling proposition is clear

  • You’re using social proof – testimonials/ customer logos / reviews

  • Your keywords are relevant to your SEO needs

  • All text is HTML for SEO

  • You have no or a minimum of links on the page

  • You are mobile focused

The Main Visual:

  • You have no more than 2 main colors on your page

  • The visual compliments your USP

  • The visual directs user attention to the call to action

  • The visual projects trust and reliability

The Registration Form:

  • Requires a minimum of fields to get started

  • Highlights your USP and why people should sign up

  • Required fields are easy to distinguish from non required fields

  • You have a confirmation page/layer

  • You send a confirmation email to signups

But remember, these are all general guidelines that aren’t set in stone. Just because they’re someone’s best practices doesn’t necessarily mean they will work for you. Be sure to test everything.

1 Registration OptimizationPre-launch Checklist

Before launching your test, there are a few important elements you have to check. This AB testing checklist may take sometime to go through but once you move forward with your test, you’ll know it has been set up in the best way. You should always be sure that the changes you made are working well and are trackable:

  • Your goals are set up in Google Analytics

  • Your goals are set up in AdWords (if relevant)

  • Your Google Analytics tracking code has been implemented

  • Images are rendering correctly

  • Google Analytics events have been set

  • Adwords account is connected to analytics (if relevant)

  • Ecommerce tracking is enabled (if relevant)

  • Goals are firing within Google Analytics

  • Segmentation is correct: your experiment is appearing to the right audience

  • Page load time is at minimum – use Google’s speed test tool

  • Variation is compatible with all browsers using BrowserStack

  • Demographics and Interest reports are enabled on GA

  • All links work (and send to the right page)

  • All links open in new tab

  • Links have anchor text

  • Image size is at minimum

  • Variation is compatible with mobile or excludes mobile traffic

  • You’ve QA’d the entire funnel of your variation (not just the page you’re working on)

  • Heatmaps are working well

  • You’ve located any 404 non existing pages

  • Lead box is working correctly

  • Your main title is in H1

  • Experiment parameters are setup

Time to launch that test!

User Flow OptimizationThe Aftermath

A test isn’t complete until we’ve reached conclusions, analyzed our results and have a plan for our next steps:

  • Test for a minimum of 14 days – to make sure there aren’t any changes due to special occasions or unknown criteria

  • Make sure you reach statistical significance. Don’t be quick to the trigger – allow tests to run to their full extent until you’ve reached significant validity

  • Analyze the data and draw conclusions

  • Get the team involved – the best way to keep testing is to showcase your results to others and get them on board with testing

  • Take the next steps – optimize

1 Mobile OptimizationTools

There are many tools that can make your job easier and faster:

  • Invision – Use invision for better communication around design and creative. Comment and collaborate on creative projects with your team and clients.

  • Hotjar – the all in one solution for heatmaps, surveys, analysis and visitor recordings. If you haven’t tried it yet you’re definitely missing out.

  • VWO – Like Optimizely, Visual Website Optimizer allows you to edit pages via its sleek user interface and AB test them without needing developers or designers.

  • Webmaster tools – Google’s webmaster tools lets webmasters tell search engines about their site to make it understandable by bots and maximize visibility to people.

  • Kissmetrics – Analytics platforms like these, in conjunction with Google Analytics, can give you access to extra-accurate and different types of reporting

  • Site speed checker – Speed checkers such as Google PageSpeed tool allow you to assess the load time of your page and learn of ways to optimize it

  • Test Significance Calculator – Sometimes, for one reason or another, we run AB tests outside of a true testing platform and just wind up with raw data. Having a significance calculator allows you to determine statistical significance in these scenarios.

Let us know in the comments:

What do you check before launching a test?

AB Test Checklist: Everything You Need to Check Before Launching 5.00/5 (100.00%) 8 votes
Talia helps businesses build and execute their conversion optimization strategies, using emotional targeting, consumer psychology and behavioral data to generate more revenues, leads, engagement and sales . Tweet to her at @taliagw