Checkout A/B testing

A/B testing (also known as split testing) is a popular and efficient technique used to understand customer behaviour and identify, in real market scenarios, the best performing marketing materials. A/B testing is widely used to compare 2 versions of the same website, landing page or microsite in terms of results achieved relative to the desired conversion – whether the conversion is represented by an online sale, a newsletter signup, a product review or other desired actions being performed by your visitors. By analyzing the statistics gathered during the testing period and the manner in which users interact with each of the 2 versions tested and the elements on page, you can determine the best layout for your business objectives.

For e-commerce businesses the checkout page is the critical step in the visitor’s journey towards becoming a buyer. Therefore we believe in the fundamental importance of constantly testing and optimizing your checkout pages in order to improve the quality of your customers’ experience and, subsequently, the level of your conversion rate and revenue gained.

checkout ab testing menu sectionThe dedicated section in your Control Panel, found at Marketing – Checkout A/B testing, contains features enabling you to easily create testing campaigns for any of the checkout pages active on your website.

Within the homepage of this section you can consult the inventory of previously or currently active A/B testing campaigns and can easily edit their content and settings anytime you need to update them. In addition, you can apply Bulk changes to the campaigns listed here, including the options to Activate, Deactivate or Delete multiple campaigns at once.

In order to create a new Checkout A/B testing campaign we start by clicking on the add checkout abtest button button. Every new campaign requires 3 fundamental parameters to be defined:

  • Status – Should the campaign become active immediately or do you wish to create it for later usage, as part of your sales & marketing planning? Easily control your campaign’s status using this toggle button. Please note that when you deactivate a campaign all split test URLs associated with that campaign will also become inactive. If you choose to only deactivate a specific split test URL, the campaign and all other associated URLs will continue to run normally.
  • Campaign name – We strongly advise you to choose a relevant title for your campaign, highlighting information such as the product, period of time or target customers/ market for which you intend to use it. As the number of A/B testing campaigns in your account will begin to increase, you will find it very helpful to be able to identify a specific campaign in the dashboard list based on its name.
  • Campaign alias (URL) – Define the desired URL tag for this campaign – ex. The URL tag can be edited at any time during the campaign, however we recommend choosing one which will be relevant for you for the entire duration of the campaign. This will make it easier for you to monitor and accurately evaluate the results obtained. This URL will act as a “host” for your Checkout page variants included in the campaign, which you will create in the next step of the process.

Once you have filled in all the information detailed above your campaign will be fully set up and you can now proceed to the next step – generating the A/B test checkout pages.

Click on add checkout abtest button to create your first Checkout page version. Let’s start defining the specific settings of your Checkout A/B test page:

  • Status – Choose whether you wish to activate this Checkout page version right away or if you would like to keep it inactive until you roll out the entire campaign. You can change the status of your page anytime you wish to do so, it will not affect the overall status of the testing campaign to which it belongs.
  • Name – As with all other naming settings (products, campaigns, bundles, etc), we highly recommend selecting a relevant name, one which will help you quickly understand the content & purpose of this page version.
  • Destination URL – Input the URL tag you wish to associated with this page, keeping in mind that it will be visible to all visitors of this link. For more information regarding the URL parameters supported by our solution please visit the dedicated page within our knowledge base, here.
  • Traffic share – Select the percentage (%) of visitor traffic you wish to direct towards this new Checkout page. Move the slider to your desired value, from 1 to 100%. Please note that in order to obtain relevant results regarding the performance of your multiple Checkout page versions we recommend allocating similar traffic to each of them. For example, if you decide to run an A/B test with 3 versions of your Checkout page, we recommend allocating 33% traffic to each version.
traffic share 3
  • Allow split test URLs for specific countries only – If you wish to run this test page on a limited batch of regions or countries (and not make it available worldwide) please activate this option and select the countries of interest from the list to be displayed below.
  • Allow split test URLs for specific languages only – Check this option if you wish to only serve this Checkout page version to visitors using only a specific language or group of languages.

And you’re done with your first Checkout A/B test page! Continue adding pages to your testing campaign (at a minimum 1 more) and start your campaign whenever you wish.

To add another checkout page version simply click on and follow the same steps as above. When setting the Traffic Share values for the various pages please note that the system performs automatic adjustments, updating the available traffic for each page based on modifications you make on 1 of them. For example:

  • Let’s assume the total traffic is 300,000 visitors
  • For the 1st page version you set a traffic share of 70% => your 1st page version will receive 210,000 visitors
traffic share 1
  • When creating your 2nd page the default traffic value available will automatically be calculated as the difference between the total traffic (300,000) and the traffic allocated for the first page (210,000), namely 90,000 (30% of total traffic)
traffic share 2
  • If you decide to modify the traffic share for the 1st page to a lesser value, the traffic available for your 2nd page will automatically be adjusted accordingly (in this case it will be increased).
traffic share 1 2 traffic share 2 2

The same automatic adjustment mechanism will be applied for all consequent page versions, should you decide to create more checkout pages for your A/B testing campaign.

Please note that in order to achieve relevant results and statistics from your A/B testing campaign we recommend keeping it active for at least 1 full week.

You can review the results registered by your campaigns in the dedicated Checkout A/B test reports, available in Reports – Others – Checkout A/B testing report. Statistics such as the total number of visitors registered per each checkout page version, total number of orders finalized through your pages and the resulting conversion rate will help you gain a quick & clear perspective of the level of performance within your campaign. The information stored here will also provide you with a chronological overview of your Checkout page A/B testing results and evolution.

Developer Tools