A Step-By-Step Guide To Product A/B Testing (The Right Way)

A/b testing

Black or white? Heart or mind? Sweet or sour? Sugar or salt? These are the normal life question we succumb to quite often. However, these questions or their answers won’t directly affect your Rs. 1 lakh business but, creating and launching a right product will! Thus, A/B testing your product is essential and doing it the correct way is all the more essential. So, what is A/B testing all about and how it works? Read on to know more and explore your product’s or service’s true potential. 

ab testing

What Is A/B Testing? 

Basically, A/B testing also known as ‘split testing’ is comparing 2 different versions of the same product or service. The one which is more suited or the winner, is pushed ahead for the launch. 

However, there is more to “how to do A/B testing the right way”. Get well versed with the mini “how to” guide below:

How To Do A/B Testing? 

Before we explore the 5 elements of how A/B testing is done, here’s a relevant example to simplify your understanding:

Let’s consider an example of Gaurang, an online business owner. He wants to launch a new product in the market. He has 2 different versions of the same product and he is puzzled as to which one is the best for this audience. Here’s what Gaurang or you or any business should do in order to attain absolute clarity:

  1. Draft 2 beautiful and separate product launch alert emailers
  2. Divide the existing database (full or partial) into two equal groups
  3. Use marketing tools like CleverTap or the one which provides A/B testing
  4. Send emailers at the right time (Tuesday: 3-5pm and Thursday: 10-11:30am) to the selected database (accordingly) via these tools (do not forget to add trackable links)
  5. Track real-time results. But, wait for a while as all group members will not open or click on the email at the same time
  6. After a good time span of 4-5 hours, check the collective results
  7. Now choose the one which has got the highest clicks through rates (CTR). Do not focus entirely on the Open Rate, as people open emails but hardly end up clicking on it. Also, make sure the content is strong for better CTR
  8. Now Gaurang knows/you know which product version was liked by the existing audiences. Also, being in a similar horizon, increasing new users is a definite.
  9. Launch the product!

Here’s the step-by-step guide of how A/B testing is done:

1. Before Testing, Plan It Out 

Just the way Gaurang had a clear-cut goal before he set out, it is important to know the reason why you want to do A/B testing.  A/B testing is to understand what works and why. Hence, it is a good idea to have all your database groups, product version, and other tools ready. 

  1. Start by studying your website data and venture into problematic areas of your conversion funnel
  2. Use tools that analyze visitor’s behavior to find out why visitors are not getting converted into customers
  3. Now, with the customer insights, come up with a way to increase conversions like making the CTA button larger
  4. Also, select a good date range to collect accurate data of webpage or email performance.

2. What To Test

Some of the variants that affect the overall customer behavior (with regards to content A/B test): 

  • Headline: Is it catchy and inspiring users to click?
  • Copy: It needs to be crisp, short and relative to the product, website or service being tested
  • Testimonials: People connect with people and not companies. Gather motivating testimonials of existing users for the potential users
  • Images: Are they optimized, well fitted and attractive? If yes, then you have passed this test!
  • Call to Action (CTA): A compelling CTA is the source of signups along with the above elements. Make sure your CTAs are mentioned at least 2-3 places in the entire page.
  • Links: Check if your links are trackable. Tracking makes sure that you are aware of how many people clicked and where. 

Choose the variant that you want to test (for example CTA) and zero it down to what feature you want to test (Size, Color, Text or Placement).  

3. Split Your Audience

The way your audience is split is based on what you are testing. If you are A/B testing emails, then you can control who your audience is. You will get an accurate result within a short period when your audience is split into two groups equally and randomly. But with a webpage, you will have to run the test for a considerable period to get an accurate result.  

Pro-tip: At a time run only one A/B test. This measure will prevent confusion of which test is driving traffic or conversions. 

4. Run The Test Simultaneously 

Make sure you run the tests parallelly to achieve healthy comparison results. Avoid running tests separately or on separate days as the basis of comparison might not be valid ( eg: today is a good day to send emails then tomorrow)

5. Determine The Testing Period

Anything good takes time and running the test for a considerable period will give you an apt result. The time to run A/B test will vary from one organization to another, based on the number of people who are involved in it.  Run any A/B test at least for a period of 2 business cycles. This is because when you stop a test in between many times you might jump to a false conclusion and thereby ruin the reason for the test. 

Pro-Tip: Run your test long enough so you can actually differentiate between the two variations. 

How To Measure Your A/B Test Results?

There are plenty of A/B testing tools available in the market. Some of them provide the same service for free or charge a price for the same service. Tools seem essential as they are automated, time-saving and above all instant result curating.  Here are some of the best tools available in the market: Adobe Target, Google Optimizer, and Optimizely. You choose your pick and get started with your work.

Once you run the test, it’s time to read the results! Consider that your result of A/B testing is something like this: Group A gives 20% while Group B gives 23%.

  1. Before jumping to a conclusion, note that the error rate might be 5% in most cases
  2. Based on this, the conversion rate (click through rate or traffic) will be 3% 
  3. Check if the new version is the product or service you provide receives a better welcome than the previous one
  4. if yes, then go ahead with it. If no, then go back to the original one. Don’t lose hope here. Going back to the older version is not giving up. Take this as an inspiration to create something more better and epic.  

Common Mistakes To Avoid While A/B Testing 

DO NOT make these 3 common mistakes businesses usually get trapped into:

  1. Taking decisions too quickly without running the test for the full business cycle 
  2. Focusing on too many results at the same time. Before you look at the report decide on what result you want to know.  
  3. Not re-testing. This might give you false results. 

A/B testing has been widely used by businesses to figure out that one factor that converts a visitor to a customer. When you run this test for a considerable time, you can gather enough data to make strategic decisions. But the key to unlocking this treasure is being consistent, with proper planning and execution. Follow these steps mentioned above to successfully kickstart A/B testing today. 

With all your marketing efforts for your business in place, it’s a good idea to have an all – in- one payment solution in place as well. Now enjoy the best payment gateway experience and grow your business effortlessly! 

ab testing

Author: Lydia Priyadarshini