Do you think you test your ads or landing pages enough? From my point of view, A/B tests never suffice, especially when your project is lagging behind the initial timing or is below targets. The worst enemies of an A/B test are the on-the-go alterations of the initial campaign plan which usually leads to delays or even failures. In this respect, a recent study says that 81% of companies failed with digital projects.
In digital marketing, we choose to perform A / B tests for performance marketing campaigns in order to validate performance predictions or assumptions. In order to do it, we have to collect real usage data from users, through different methods.
If some marketers chose to test the advertisements, others focus on test the pages of the website. Each test model is based on a strategy and optimization of the investment budget. Let’s dig more into this topic.
How is an A/B test defined in Ecommerce?
A/B testing is an “in vivo” split testing method that runs simultaneously two advertisements or page variants that follow the same marketing strategy. It is an in-action eCommerce prototyping technique that allows us to determine which ads, visual elements or pages perform better and produce better results. It borrows know-how from both quantitative and qualitative marketing research because the variables in A/B testing include both customer behavior and big data analytics.
This type of split testing can take place on-site or off-site:
- On-site A/B tests: allows us to test all the sales actions on each page or on the landing page
- Off-site A/B test: obtain data about an advertisement, sales email, copywriting part. This way you will know that you are running ads for the right audience and you will justify your marketing budgets.
In this article, I will focus on the digital AB tests, that allow us to create personalized strategies for an online store.
7 Benefits of an A/B test
1. It helps increase your conversion rate by improving UX in Ecommerce
When testing the user experience in the store you can consider if certain buttons with CTA are pressed. According to hubspot.com, in order to do an AB test you need to create 2 different landing pages:
- the first will be the original, also called the Control page
- the second is the alternative landing page or the Challenger page (in fact, this page can have more variants)
After setting them you need to test them with a statistically relevant number of visitors (I recommend you >2000 visitors) and to make sure that the pages will be served randomly. At the end of your test, your analytics dashboard should show you which one received more clicks, but also which call-to-action buttons worked best.
2. Highlights good and poor sections of the Sales Funnel, where web store users or orders are lost
3. Helps identify the most efficient conversions
4. Provides pricing insights
5. Allows reducing the bounce-rate
Bounce rate is the metric used to track in Ecommerce to determine a website’s performance. For a one-page micro store, it can be ignored, but in general, we have to pay attention to it.
Leaving on the side the technical definition of Bounce Rate, it means that a visitor spends little time on the website. She/he is influenced by either too many options, misleading information, or her/his expectations are different than what sees on the respective page.
In order to lower a high bounce rate, you should test several variants of the same element to find the optimal solution. In case you decide to change also the URL, don’t forget to check the page’s redirection.
6. Changes to the store are risk-free. Isolate the cause and effect.
With the help of an A/B test, you can test smaller portions of the website, to avoid creating a new design. This way you can isolate the element that will increase the ROI or ROAS.
Regarding product inventory: In an online store, you may have hundreds of products on virtual shelves, and descriptions play an important role. Write a new version for one product at a time and test how visitors react. Draw the conclusion and discuss with your team on applying learning from split testing to other products as well.
7. Justifies a website’s redesign or structural changes
Combining UX tools like Microsoft Clarity with AB testing will provide you with plenty of insights regarding how visitors interact with your website.
What steps should I take to launch an AB test?
Start with the WHY: Why do you need to run it? Then focus on the WHAT, the Goals you expect to achieve via eCommerce campaigns. By answering first to these questions you will be able first to set your expectations, but also to align with the rest of the marketing team and managers. At the same time, you’ll get the needed legitimacy in order to use the resources required to perform this activity.
A. Quantitative and qualitative research
Start your research in the web store’s backend with a performance audit. Find out first many users visit it, which sales funnel work, which pages bring the most traffic. Focus first on quantitative data, in order to determine your rational hypothesis for the qualitative analysis that follows and which looks at how to tailor the web store’s UX based on consumer profiles. Qualitative data can be collected with the help of tools that track behaviour and interests. You can do this via questionnaires or by using heatmap tools like Microsoft Clarity or Hotjar but also recording tools.
B. Be a skilled observer and make assumptions
Synthesize data findings from quantitative and qualitative research and see what conclusions you can draw. Be inspired and track the users’ behaviour changes in the e-shop. Formulate hypothesis.
C. Create several variants for a more successful A/B test
With the hypothesis in mind, create a variation of the version you want to change. Start testing the section, CTA or page that doesn’t work for you.
For example, you want to test which homepage option brings you more clicks to the spring campaign with discounts. Thus, you find out what preferences visitors have, if the campaign idea is attractive, data about buttons, banners, copy.
D. Run the Test
Shortlist how many test variants you will use and prepare everything in the smallest detail. Let’s see how many types of split tests you can use for eCommerce:
Multivariate Test (MVT)
It is the method by which you modify multiple sections of the store. You will launch several variants, all possible combinations in one test. When assessing the results, try to focus on what improved the Conversion Rate.
Split URL Test
You test several variants of a page but on different links. This option is preferred when it is necessary to make drastic changes in the design.
During a Split URL test, the E-commerce traffic is split between the Control page and its alternatives (or Challenger pages) for which we measure the conversion rate in order to designate the winner.
Steps to run a Split A/B test:
- Create pages for the Split Test URL
- Add the estimated duration of the test and what conversions you expect
- Complete the test
- Interpret results and update the content and design
E. Assessing test results
Take each result and analyze it in detail. You will follow the entire customer journey. Analyze the results based on the following metrics:
- confidence level;
- percentage increase;
- how it impacts other metrics directly and indirectly.
Example of A/B tests in Ecommerce
Let’s say you own an online store and you want to promote a new collection of spring shoes. You’re creating a test campaign and you want to see if that landing page appeals to your target audience. For this, you should A/B test your CTA (call-to-action) elements on landing and promo pages, which usually are money-making triggers. You will be able to A/B test the text, the colors, but also the design of the button or content of the respective CTA canvas (can be an image, too!).
Using Forms on the landing page
The forms on the Landing Page are another testing solution. Try to design a few different dimensions, from simple to more complex and see when or if the visitor abandons the form and if the size is the one that discouraged her/him from completing it.
A headline can also be included in the test area. If you decide to have it, create it having in mind why a person would want your product. In eCommerce, you can test as much as you want, so try long vs short headlines, in the form of a question, expresses negative or positive emotions.
I recommend you testing also the creative elements (graphics, videos, copywriting). During the user’s acquisition process, the right words or visuals usually make a big difference. If you are not pleased with how your initial headline iterations perform during the A/B test, try to rearrange the benefits or use a different language.
References:
