A/B Testing

A/B testing, while not as well-known as Search Engine Optimization and Analytics tools, is a great way for a business to delve into information that will allow them to have a better understanding of how users react to certain features on their website. With A/B testing, two versions of a site are created and then analyzed to determine which site is more successful: Site A and Site B. Site A can be an existing site, or, in the case of a new business, a hypothetical template. Site B will be an alternative version to be tested against Site A. Traffic is divided between the two sites and then the data is analyzed to determine which is more successful. This success can be measured by various metrics such as bounce rate, number of sales, conversion rate, etc. Changes can be something as simple as a button color, to something more complex, such as the layout and design of a website.

In order for A/B testing to be successful, there are several factors to consider. Firstly, it’s important to know what exactly you are looking for when testing an existing site against a hypothetical alternative. Knowing your current results through a tool like Google Analytics can be extremely helpful here. If your bounce rate is extremely high for instance, you want the alternative site (Site B) to demonstrate an improvement or decline in the bounce rate. Trial and error is bound to happen, and so several A/B tests may need to be conducted to achieve definitive results. Another factor key in successful A/B testing is simultaneous testing; in order to achieve results that are accurate the two designs must be available at the same time to rule out factors that could affect the variables. Also, the longer an A/B test is conducted, the more accurate the results. To determine how long a test should be conducted, it is helpful to use a calculator such as this one which will help by identifying the number of days required to get statistically significant and reliable results. Lastly, cookies or another mechanism such as a sign in page, will need to be used to split the audience evenly and ensure that users aren't getting exposed to both versions.

A/B testing for digital marketing can also be extended into areas outside of website design and web page features. For instance, two variations of an email could be sent and then responses to either can be measured to determine which received the better response or Ads can be monitored to determine which design or placement produces the best result. In cases like these, online tools like Analytics may be helpful as well, but also, it may benefit the user to also have some elementary knowledge of statistics in order to understand and track the results better.

Various resources exist to help a site developer conduct A/B tests. This useful tool is an excellent way to improve marketing results and better understand your customer. I really enjoyed this article from HubSpot.com demonstrating some real life A/B techniques.