Determining the factors that affect user experience involves testing to determine why users do what they do. Understanding why users behave in a certain way on your web site will show you how that behavior can be influenced so as increase successful outcomes.

Two ways that testing can be performed are:

• A/B split testing
• Multivariate testing

A/B Split Testing

A/B split testing measures one variable at a time to determine its effect on an outcome. Different versions are created for the variable you want to test. For example, consider the following tests:

• Two e-mail subject lines for the same e-mail to see which produces a superior open rate
• Different placements of the “buy now” on a product page to see which results in increased sales
• Different copy styles on paid advertisements to see which gives a higher CTR (click-through rate)

In these cases, only one variable is tested at a time, and all other elements on the Web page, in the e-mail, or part of the paid advertisement remain the same. You can test more than one version of the variable; it just means that you will need to test for longer.

Traffic is then randomly distributed to the different versions, and the outcomes are measured for each version of the variable. The results are then interpreted to see if there is a statistically significant difference between the variables. The version producing the best results can then be employed.

Best way to approach A/B testing

While intuition helps us make a lot of decisions in life, data truly is king in the world of web analytics. When it comes to A/B testing, the data should inform most if not all of your decision making. Any changes to web design should also reflect the best practices in design and user experience: don’t just “wing it” because you fancy a certain colour or contemporary vernacular. Making small incremental changes instead of a massive redesign project is also a better way to control the process and continue using data to inform your changes.

A/B Testing of a paid advertisement

This example shows how an online paid ad might be tested to see which version of the ad performs better. The item being tested is the call-to-action (“CTA”) and singular conversion activity on the ad – the “order” button. In this example, the colour of the button is being A/B tested to see which one will earn more clicks – grey or red.

Which scored higher? Since this is a fictitious example, there is no real answer as it’s just meant to illustrate an example of how an A/B test might be conducted.

Multivariate Testing

Multivariate testing allows you to test many variables at once and still determine which version of each variable has a statistically significant effect on your outcomes. For Web sites, there are a number of vendors who will host pages that are being tested in this way remotely, if you do not have the technology to do this in-house.

Multivariate testing allows you to test, for example, the following:

• Subject lines and copy style for e-mails
• Color, font size, and image size for Web sites

The combinations are endless, and because of that, it is easy to get stuck analyzing every tiny detail. Successful testing relies on having clear objectives to begin with, and sufficient traffic to warrant such detail.