Split testing in your Shoporama online store

Split testing is an effective way to improve your online store. There are various tools for split testing and most of them run on the client side with Javascript. This means they are somewhat uncertain and slow down your shop at the same time. If you use Shoporama's split testing tool, your shop will run just as fast as if you didn't split test (with the exception that you can't use caching while your split test is running).

Setting up split testing on Shoporama is a bit more technical than tools like Visual Website Optimizer, as you need to understand your template. But on the other hand, the experience will be a bit better for your customers as they won't experience the slight "lag" that javascript-based split testing tools provide.

This article will guide you through how to set up a simple split test in Shoporama.

How does the split test work?

Quite simply, your Shoporama shop consists of a theme, which consists of a bunch of files. For example, your product is usually displayed in the product.html file. This file often contains another file called price.html, which shows the price and one called add_to_basket.html, which shows the add to cart button. The way Shoporama split-tests is that you make a clone of price.html (so you have two or more variants of that file). If you have two variants (the original and version B), Shoporama will only show one file to every other user and the other file to the other half of your users. If you want to test multiple versions of it, just add one more version and then every third user will see that single version. You can add as many versions as you want, just be aware that the more versions you add, the longer it will take before you have a valid result to rely on.

So the first task for you is to identify which file to split test. In this example, we're going to split test the text on the button, so it's the add_to_basket.html file we need to grab. And here comes the recipe.

1. Go to Account → Online Shop → Themes (URL: /theme/list) in the left menu
2. Click on the "Create new theme" button
3. Give your theme a name, for example "Add to cart button test". The name is only used so that you can identify the test yourself
4. In the field below you can choose to lock the split test. If you don't want to keep an eye on it, you can choose for the test to run on 1000 orders and then lock to the version that converts best (based on number of orders)
5. Below this, there are two fields. One contains all the files from the Shoporama themes and the other all the files from your themes. Note that some people have multiple versions of their theme, so make sure you find the file in the theme you are using. If you have your own theme, go to the bottom box. Scroll to the file you need, in our example add_to_basket.html
6. Click "Create" at the bottom of the page
7. Check the file and click "Copy the selected templates"
8. For the sake of clarity, name the original. Click the pencil next to the top one and name it "Original", "Version A" or whatever you want to call it
9. Click "Save"
10. Click the pencil next to the bottom version and name it "Version B" or whatever you want to call it
11. Scroll down the HTML on the page. Find the thing you want to split test. In our case, it's the text of the button, so we scroll down and find the <input> field with the button
12. Edit the text and click "Save"
13. Click "Use this theme" and your split test will run

Note that you need to disable caching for your split test to run.

How to keep track of your split test

To check your split test, simply log in to Shoporama, go to Account → Online Shop → Themes and click on the theme you created. Click on the pencil next to the theme and you will now see your split test.

Have fun with your split test.

Bonus: Track your test in Google Analytics

We recommend that you track your split test in Google Analytics. This way you can follow your test in Analytics. It may be that one version works really well on mobile but not on desktop, and you will be able to see this in Analytics. To do this, you need to use Custom Variables in Analytics in each version of your code (both the original and version B). Here you should indicate at visitor level that the user has seen either the original or version B. The type of Analytics code you should use depends on which version of Google Analytics you are using on your site. Below is an example of what it might look like:

_gaq.push(['_setCustomVar', 1, 'Split test button', 'Version B', 1 ]);

Remember: Done is better than perfect. So if you can't handle the Analytics tracking, get your split test set up and let it run. You can always geek out on Analytics before the next test is up and running.

Also note that if you have few visitors to your webshop, it will take ages before your split test produces any results. If you have few visitors to your webshop, you should instead focus fully on getting visitors.

Do you need help? Contact us at support@shoporama.dk.