{"id":3195,"date":"2017-06-16T19:23:53","date_gmt":"2017-06-16T13:53:53","guid":{"rendered":"https:\/\/www.innovationm.com\/blog\/?p=3195"},"modified":"2017-06-16T19:23:53","modified_gmt":"2017-06-16T13:53:53","slug":"ab-testing","status":"publish","type":"post","link":"https:\/\/www.innovationm.com\/blog\/ab-testing\/","title":{"rendered":"A\/B Testing"},"content":{"rendered":"

What is A\/B Testing?<\/b><\/h2>\n

A\/B Testing (also known as Split Testing or A\/B\/n Testing<\/b>) defines a way to compare two versions of an application or a web page that enables you to determine which one performs better. It is one of the easiest ways to analyze an application or a web page to create a new version. Thereafter, both these versions can be compared to find the conversion rate<\/b>, which further helps in finding the better performer of these two.<\/p>\n

\"AB<\/a><\/p>\n

Simply put, it\u2019s a way to figure out how we can get our website users to click that shiny red button instead of yesterday\u2019s green one. Or maybe discover if the green one was the better choice to begin and thus forcing us back to the previous version.<\/p>\n

Put in another way, it\u2019s a method for measuring two versions of one element so we can determine which one is more successful. Is the winner version A or version B?<\/p>\n

If we have a website or an app, we have activities that we want our users to complete (e.g., make a purchase, sign up for a newsletter) and\/or metrics that we want to improve (e.g., revenue, session duration, bounce rate). With A\/B Testing, we can test which version of a page results in the greatest improvement in conversions (i.e. completed activities that you measure as goals) or metric value.<\/p>\n

A\/B testing is a fantastic method for figuring out the best online promotional and marketing strategies for a business. It can be used to test everything from website copy to sales emails to search ads. Well-planned A\/B testing can make a huge difference in the effectiveness of the marketing efforts.<\/p>\n

The first thing to do when planning an A\/B test is to figure out what we want to test. It might be \u00a0an on-site test, or an off-site test. Now, what is an on-site test or an off-site test?<\/p>\n

Let\u2019s take an example of a site that sells subscription boxes and also the individual products. If we\u2019re running an on-site test, we\u2019ll want to think of all the sales-related pieces of our website, and then figure out which elements we want to test. In this case, it might be the number of subscription boxes sold from the cart and the number of customers ordering individual products.<\/p>\n

With off-site tests, we\u2019re probably testing either an ad, or a sales email. In this case, we might also want to test whether users are more comfortable with tapping banner ads or interstitial ads. Testing ad copy to see which ad brings in more converting visitors can help us focus our advertising efforts. Once we know the particular ad is converting as well as possible, it\u2019s easier to justify spending more money and time on it. The same goes for emails. If we send out two versions to our list (randomly selecting which half gets which email), and then track which one converts better, we can send only that version to all the contacts next time.<\/p>\n

\"AB<\/a><\/p>\n

But before we start testing, we need to have a very clear idea about the results we are looking for. We should already be aware of our baseline results. We want to test option A and B against each other, but we also want to know that whichever one does better in the test is also doing better than your current results. Alternatively, you can use A as your control (whatever you\u2019re currently using) and then use something new for B.<\/p>\n

Tests need to be run simultaneously to account for any variations in timing. We cannot test one variation today and the other one tomorrow, because various factors might change between today and tomorrow on which we might not have control. Instead, we need to split the traffic seeing your variations at the same time.<\/p>\n

The option A (current live version) is known as the control<\/b> and the option B (modified version) is known as the variation<\/b>.<\/p>\n

Before we think about optimization and testing, however, solidifying our high-level strategy is required. So the order should be:<\/p>\n

    \n
  1. Defining business objectives<\/li>\n
  2. Defining website goals<\/li>\n
  3. Defining Key Performance Indicators<\/li>\n
  4. Defining target metrics<\/li>\n<\/ol>\n

    \"AB<\/a><\/p>\n

    Case studies of A\/B Testing :<\/b><\/p>\n

    Two Magical Words Increased Conversion Rate by 28% –<\/b><\/h4>\n

    In this experiment, two simple words increased the sign-ups on this button by 28%<\/b>, demonstrating the significance of measuring call-to-action buttons. Apparently, even the smallest alterations can have surprisingly effective results. The two magic words in question? \u201cIt\u2019s Free!\u201d<\/p>\n

    \"AB<\/a><\/p>\n

    Changing a Sign-Up From Green to Red Led to 34% Growth –<\/b><\/h4>\n

    Sometimes even the most trivial change to a website can fuel substantial results. A website increased their conversion rate by 34%<\/b> by changing the color range of their sign-up button from green to red.<\/p>\n

    \"AB<\/a><\/b><\/h4>\n

    Changing the view of the banner image resulted in more conversion rate –<\/b><\/h4>\n

    The original page and variations are:<\/b><\/h4>\n

    Original <\/b> <\/b><\/h4>\n

    \"Look<\/a><\/h4>\n

    Variation #1: Banner Image <\/b><\/h4>\n

    <\/h4>\n

    \"Look<\/a><\/h4>\n

    Variation #2: Sub-heading <\/b> <\/b><\/h4>\n

    \"Look<\/a><\/h4>\n

    Variation #3: Button Text <\/b> <\/b><\/h4>\n

    \"Look<\/a><\/h4>\n

    Variation #4: Button Color <\/b><\/h4>\n

    \"Look<\/a><\/p>\n

    This is the landing page of a travel and tourism website. Variation #1 was found to have much more positive response from the audience (even more than the original one) than the rest of the variations. Variation #1 just had a different view of the same banner image.<\/p>\n

    Image courtesy : Google Analytics<\/b><\/p>\n

    Add-to-Cart Experiment –<\/b><\/h4>\n

    A website, decided to test their purchase process by making three variations of an \u201cAdd to Cart\u201d button to increase sales. One being icon focused, one with larger text, and one with a simple \u201c+Cart\u201d. The test had a major impact on \u201cAdd to Cart\u201d clicks. Variation 2 increased conversion by 15% compared to the original and Variation 1, the larger button with the clear call-to-action text, increased \u201cAdd to Cart\u201d clicks by a staggering 49%.<\/p>\n

    \"AB<\/a><\/p>\n

    Few tools that can be used for A\/B testing :<\/b><\/p>\n