Using Google Website Optimizer

(Rachel Macdonald) #1

Hey all,


I want to do some A/B testing. Google's Website Optimizer seems like a possibility but the thing I want to test is an extra option in a menu that is created using design tags so I'm not sure how to keep this invisible to anyone not part of the test.



Has anyone had any experience with using Optimizer with Matrix? Or used any other software to do this kind of testing?

(Richard Powell) #2

I'm also interested in how to do this.
A-B testing involves having two (or more) different versions of the same page.

It is necessary to track the pages involved in the testing to understand which version of the page was shown to whom, to know which version was more effective.



The same concept can also be applied to a series of pages, or an entire site (multivariate testing) where the style and layout (and content) can be different for different people - like personalisation, except this is usually randomised.

(Duncan Robertson) #3

Done my fair share of split tests with GA. What's your case and context?

(Rachel Macdonald) #4

I guess I'm not sure how to have two versions of the same page, for a start…

(Duncan Robertson) #5

I've done two types of split testing on GA:


[list=1]

  • A/B
    [*]Multivariate


    If you're looking to compare one page vs another page (AB) of the exact same content but different layout that's straight forward. All you have to do is follow this guide:

    http://www.google.com/support/websiteoptimizer/bin/topic.py?topic=14315

    You can even use the SQ_PAINT_LAYOUT_NAME method to have multiple designs without changing the whole URL.

    I would also read this to deal with some of MSM if you're testing something like a calendar, asset listing etc:

    http://www.google.com/support/websiteoptimizer/bin/answer.py?hl=en&answer=61201

    I find Multivariate experiments far more effective for figuring out processes in my sales cycles in our travel sites. Take a look here - it does require a small code to be installed but works much more effectively.

    http://www.google.com/support/websiteoptimizer/bin/answer.py?answer=61149

    If you're doing UI testing why not compliment your A/B with a UI testing service like UserFly? It's far easier to figure out the UI issues when you see how people are traveling through your content than split testing alone.

    I hope this is helpful.

    EDIT: I cannot spell anymore.
    • (Rachel Macdonald) #6

    Great links, thanks! :smiley:

    (Duncan Robertson) #7

    Hope it helps. I find using split tests by themselves really unhelpful. It's far better combining them with some form of UI tests to just know where to improve rather than guessing. Most times, it's completely counter intuitive.

    (Richard Powell) #8

    Thanks Duncan.
    Is there a way to do A/B testing with different content (as opposed to different designs using the same content)?



    Why? This would be useful to test changes with the public before 100% committing to them, ensuring that conversion rates did not fall. My websites experience variability in traffic due to weather events, so different versions need to be tested simultaneously. If I went live with the new version and compared stats to the old version, it wouldn't mean much as the traffic and audince could have changed from one period to the next.

    (Duncan Robertson) #9

    If I'm reading you correctly, yes you can, this is what Google call multivariate testing.


    These tests allow you to test variables on the page rather than the page itself. You can choose a set of content, like a number of different intro text, and check which gets the best user response. It works best when combined with a number of variables tested simultaneously. This I think is also more effective, as IMO elements on the page are never viewed in isolation and influence each other. So make sure you user as many multivatrates as possible, headline, images, call to action text and so on.



    You'll need some good traffic to get proper results.



    This will explain more: http://www.google.com/support/websiteoptimizer/bin/answer.py?hl=en&answer=55918



    Does this help?

    (Richard Powell) #10

    [quote]
    If I'm reading you correctly, yes you can, this is what Google call multivariate testing.



    These tests allow you to test variables on the page rather than the page itself. You can choose a set of content, like a number of different intro text, and check which gets the best user response. It works best when combined with a number of variables tested simultaneously. This I think is also more effective, as IMO elements on the page are never viewed in isolation and influence each other. So make sure you user as many multivatrates as possible, headline, images, call to action text and so on.



    You'll need some good traffic to get proper results.



    This will explain more: http://www.google.com/support/websiteoptimizer/bin/answer.py?hl=en&answer=55918



    Does this help?

    [/quote]

    It does help. We are on the same page.

    Traffic volumes are high, but I would want to try A-B testing first, not jump straight into multivariate testing.



    What is needed to configure mySource matrix to deliver A-B testing of a page?

    (Duncan Robertson) #11

    Well, you need two pages to do AB tests. You can achieve this with paint layouts, design customisations or new designs. It's all about your implementation really…

    (Richard Powell) #12

    My site generates all the navigation automatically.
    Would there be some way to prevent two pages appearing in the navigation in place of one?

    (Duncan Robertson) #13

    Number of ways: it could be either link type value or status.