A/B Website Testing Set-Up, Development & Analysis

Table of Contents
    Add a header to begin generating the table of contents

    There’s no point in a landing page that doesn’t convert.

    After all, what is the point in a webpage designed to get you results, if it does nothing but sit online, getting virtually no results for your business?

    If you’ve decided that now is the time to make a real difference in your organisation and start getting actual VALUE out of your website, then we have something which will be of use to you…

    In this blog, we’ve compiled our effective strategies and tactics for using A/B testing in your own business, and how you can keep it simple and to the point with a few key steps.

    Sound good?

    Then get reading!


    Top Tips

    1. Before getting started, we have something you’ll need… you can download it here.
    2. The spreadsheet you’ve downloaded is something which was created specifically to work with Google Optimise, however, you should keep in mind that it can be used with a variety of A/B testing tools. In the event that there are some major differences between this spreadsheet and the tool you’re using, simply making changes to the spreadsheet!
    3. You’re going to need to have this extension installed on Chrome: “Google Optimise” this is key for you to be able to create variations
    4. It is key to keep in mind that if you’re currently using any form of ad blocker which has the potential to stop Google Analytics (etc) showing you pop-ups you’ll need to disable it so that you are able to create variations.
    5. To follow this blog successfully, you will already need to have conversion goals up and running on the relevant Google Analytics account.
    6. You are going to need to have Google Optimise up and running on your site.



    A/B testing is vital for making sure that your webpages are as optimised as possible for getting conversions for your business by making small alterations which you can monitor and record the impact of over time.

    If you’re a Google Optimise user, then it will alert you to which of your tests performed the best, so you know which would be the best to implement!

    Once you’ve completed this blog, you will know how to successfully set up and test a variant on your website so you know if it is wise to implement.


    Which Feature Is The ‘RIGHT’ One For Testing?

    You need to take a look at your landing page and decide what you’re planning on testing. Traditionally, the features on a page that are usually tested are:

    • Titles/Sub-headings
    • Moving Sign-ups On-Page Position
    • Placement of Videos + Media
    • Call-To-Action – text & styling
    • Sales Content
    • Use of testimonials/review/case studies
    • Price Rates


    Taking A Closer Look…

    This is the time where you will need to have a look at the changes that you have made on the web pages to decide whether they have positively or negatively impacted your business’s ability to convert. To ensure you get an accurate insight, you will need to quantify your theories, although it is important to keep in mind that external factors lead to the change – i.e. a change in season has negatively impacted your conversion rate, rather than something you have A/B tested during that time.


    • Customer Feedback

    You will benefit from going through reviews and feedback that you have gotten from customers – if there are any features that they have mentioned which they needed more info about – i.e. “How does the fact your product is made from X% this make a real difference?”

    With this in mind, you can create a landing page which includes the answers to your ideal prospects’ questions, that way, you can ensure you’re giving value and informing people based on a common query you get.


    • Session Data

    There are a range of tools out there which you can use to monitor sessions on your website – i.e. Hotjar is a tool we’ve found to be of real use. Now onto how you can use it to get great results!

    1. Make sure to monitor a minimum of 10 sessions which resulted in a conversion
    2. Now look at another 10 sessions where the end result wasn’t a conversion
    3. Try and pinpoint any differences between the sessions, were there any reasons why the sessions didn’t have the same outcome – was there an obstacle in the ways of those who didn’t end up converting?
    4. If you notice any similarities between those sessions which resulted in a conversion, make a note of it, do the same with the sessions which didn’t end with a conversion.

    The reason that you will benefit from going through these 4 key stages is that it will help you to really take a closer look at the sessions on your website and highlight what could have stood in the way of you getting even more conversions – i.e. you might find a bug on your website you never noticed before.


    Pinpoint the “HOW”

    You’re at the stage now where you’re going to need to figure out how you plan on testing things, luckily, we have something which can be of use to you, this spreadsheet – you just need to go through and populate it. Below we’ve outlined how you can do that in a simplified way, making sure to explain all the columns on the spreadsheet.

    1. Test Number
    2. Start Date
    3. End Date
    4. Author
    5. Length of Test
    6. Reason – for this, you just need to outline why this test is being done – so outline what you’re planning on testing and why.

    Okay, so hear me out, it is really important that you have a reason for running A/B Tests because without clear reasoning you could end up not performing tests effectively (and after all, you can only run so many tests at a time, so why mess around with the valuable few you do have?). Don’t risk negatively impacting the performance of a landing page or website by testing things randomly, please do things with reason in mind and that will help you get GREAT results!

    7. Feature Tested – you will need to pinpoint which feature is going to be tested on the page.

    You should know that you will benefit more from changing one element at a time – this is because you will then be able to clearly see what has had an impact on your conversion rate, rather than making a series of changes and then being left to wonder what exact alteration resulted in a drop or increase in your conversion rate.

    8. Target Prospect – outline who you’re going to be aiming this landing page at – you may decide to hyperfocus your tests on a niche group.

    9. Monitoring Metrics – this bit is up top you! Pick out some key metrics that you feel would be a great marker for monitoring success for your tests – on the spreadsheet you’ll notice we’ve left 3 spaces available, so make sure to order them in terms of importance, so you know what the key focus is from looking at the spreadsheet with a glance.

    10. Variation A / Variation B – here you’ll just need to input the URL of the pages which you are planning on using for this testing process. If you want to make it even clearer which is which, take a full-page screenshot (pro tip – the chrome screenshot extension is GREAT for this, we use it all the time 🙂 ).

    11. Pages For Test – simply input the URL of the landing page where you’ll be experimenting.

    12. Test URL – here you will need to input the URL of the tool you’re using for A/B testing. In the event that you haven’t actually started testing anything yet, then feel free to leave this blank for the time being.

    13. Final Outcome – here you’re going to need to log in the results that you have gotten from testing so far – this will help you to see how much progress you are making throughout the different tests, enabling you to increase your awareness of the positive or negative impact that different changes have had.


    How To Optimise Your Use Of Google Optimise

    Okay so to get you up and running, you’ll need to use go onto this website and login with the relevant Google Account (you’ll need to use Chrome so you are able to use the correct extension).

    To get started, just follow the steps we’ve outlined below:

    1. Hit this button: “Create Experiment”
    2. You will see that you are able to give the experiment a name, you will benefit from putting it in this format: “Test Number – Descriptive Name”
    3. In the URL section, you’re going to need to insert the URL of the page which you had been using for testing prior.
    4. From the list shown to you, make sure to click on “A/B Test” and then hit “Create”

    You’re going to need to connect your “Google Analytics View” and select the relevant one from your account and then hit “Finished”.

    Now you need to input the relevant objectives, if you have already set up your objective as a Goal on Google Analytics, then you will need to click “Choose from list” and then click on the relevant objective. But, if you don’t already have the goals set up on Google Analytics, then you will need to click “Create Custom” – this will mean that you are able to use events on Google Analytics that are already set up on your site as well as particular pages which you would like to use for your testing.

    If there are any more objectives you need to add, then just go through the process of adding them in the way that we’ve shown above.

    • You’re going to need to copy this column: “Reason” into the section labelled: “Description and hypothesis”
    • In the event that you want to narrow down the target segment for this A/B test, you will need to select this tab: “Targeting”
      • You’re going to need to click the percentage of users you’re planning to target – you’ll need to do the same for weighting.
      • Now make sure to select “And”, you will find it beneath “Additional conditions”
      • You will be shown a list of rules, select the one that you’re going to need and configure it as needed.
      • Make sure that you have hit the “Save” button.
      • A message will be shown on the sidebar – you will need to select “Run Diagnostics”, this will go through your set-up and run a diagnostic of it.
      • After a few moments, you should see a success notification.


    Google Optimise – Creating A Variation

    Now you will be within your experiment on Google Optimise, now make sure to select: “Create Variant” and then make sure to give it a name and then hit the “Add” button. You will be able to see a list of variants and you should notice that this label pops up: “0 changes” – make sure to select the relevant variant it appears on.

    You will now be shown the page editor – it will be showing you the page that you specified in prior steps, and here is what you need to do:

    1. Right-click on the feature that you’re intending to edit, and choose whether you’d like to make alterations to the text or remove it altogether.
    2. After you have gone through and changed the feature as you intended to, make sure that you hit the “Finished” button.
    3. Continue to go through the above-numbered phases as much as you’d like to get the end-result variant that you’d like.
    4. After you have completely finished making changes to the page, you will need to hit the “Save” button.

    If you would like to see how the variant that you have created looks on different devices, you can click on the dropdown menu (it will probably currently say “Standard”) and be able to select the devices you’d like to test.

    If you are happy with how the variant that you have created appears on different devices, then you will need to hit “Finished”.

    • Now you will be shown the dashboard for your experiments, and here’s what you need to do next.
      • Select the “devices” icon on the relevant variant
      • Select “Share preview”
      • You will be given a link which you need to copy and then send out to your co-workers – this is so that they can have a look at the variant on their own devices (to see how well it presents).
      • After you’ve checked the created variant is presenting well across different devices and browsers, you can set the variant to active by hitting “Start Experiment”.
      • Hit “Start” and you’re A/B testing is up and running! (the shown status should confirm this)


    Using Google Optimise For Results Analysis

    1. Start by selecting the tab labelled “Reporting” within the experiment. (On Google Optimise)
    2. Here you should be able to see how well your experiments are performing according to statistics – it will present to you the how well the variant is doing when compared to the baseline.
    3. If you click on an objective you will be able to see how well the experiment is doing for each individual objective.


    And… that’s it!

    You have now successfully set up an A/B test for your website, and know how to monitor it using Google Optimise – you should be aware after 2 weeks, Google Optimise will alert you to which of your experiments have performed the best.

    If you found his blog useful (and we hope you did!) make sure to check us out on social media, so you can keep updated with all our latest news and events!