CRO from scratch. Part 2.
Generating new ideas is essential for any development. Of course, not every idea will increase conversion rates, drive traffic, or positively impact some other metric. How can you figure out if your idea is worth pursuing? There are many tools, but one of the most popular is A/B testing, which we’ll discuss in this article.
A/B testing is an experiment that you need to conduct to test hypotheses. To determine if a certain change benefits you, you need to compare the original version with the changed version and evaluate the results.
Did you know about 75% of the 500 largest online stores use A/B testing tools?
A/B testing platforms help companies figure out where customers get stuck or frustrated during the buying process. They can also help improve how visitors use the site, which leads to more sales and customer satisfaction.
A/B testing is vital not because it’ll help you achieve high CRs, but because it confirms the conclusions you have drawn from collecting and analyzing quantitative and qualitative data.
If you missed the previous post, please read it to keep up: CRO From Scratch. Part 1
So how to get started with A/B testing?
First, check how much traffic you already have. If your site isn’t getting enough traffic, consider increasing it. Otherwise, all test results would be a waste of time and money due to low statistical significance.
The selection of metrics is very important for a successful A/B test. By this, we mean important business indicators: sales, revenue, net profit, number of visitors, conversion rate, bounce rate, etc.
It’s advisable to choose one metric, but in some cases, it’s also allowed to include several indicators as additional ones. The most important thing is that you do it sparingly! If you track a dozen metrics in one test, the effectiveness of the test will be significantly reduced.
To determine how much traffic you need to get proper testing, you can use the help of the A/B test sample size calculator Optimizely.
Enter the current CR, the minimum detectable effect (desirable increase in CR), and the level of statistical significance (the default is 95%). The calculator will show the number of users needed for an A/B test.
What should be tested first?
- Qualitative and quantitative data. It makes the most sense to rely on metrics collected with Google Analytics that answer the question WHAT is wrong, and qualitative insights that answer the question WHY.
Rely on collected data and logic. Without it, we rely on chance. - Pages with the highest conversion rates. Here it’s simple: if you can improve the conversion rate even by a small percentage, the total number of targeted actions will increase significantly.
- Pages with the lowest conversion rates. It makes sense to go in the opposite direction and start testing the pages with the lowest conversion rates – they can only move upwards.
Depending on your strategy, consider looking at these elements:
Analyzing the data
Most programs that help with A/B testing offer convenient summary tables and clearly display the experiment results and statistical significance indicators. There should be no problem determining the winner. Also, use the data for further research and create new hypotheses to test.
There are plenty of a/b testing software solutions on the market, for example, Unbounce, VWO, and Optimizely.
Before conducting your first A/B test, follow this checklist:
- Determine the feature you want to test.
- Create two versions of the same ad, landing page, app, or other elements you want to test.
- Decide on an appropriate test duration, which should be at least two weeks, depending on traffic and industry.
- Choose a testing tool that will help you conduct your experiment.
- Start the test.
- After a few weeks, check which version performed better.
- Repeat the process continuously because A/B testing is most useful on an ongoing basis.
Random fun fact to consider
Traffic on your website can vary by 500% depending on the headline. Source: MOZ
Read the final part of the series: CRO from scratch. Part 3
