Sitecore DMS Quick Win #5 – Multivariate Testing

Aaron Branson
March 13, 2014
Sitecore , Digital Marketing

In part 5 of this series, titled "5 Quick Wins with Sitecore DMS" we are going to build on our DMS implementation's first four quick wins. We have all of the assets in place to now test various content options, measure results, and be fully equipped to optimize the content for improved results.

Harkening back to Quick Win #1, we defined measureable Goals and assigned an Engagement Value to each. We also implemented web content personalization in Quick Win #2. With the goals in place, and the knowledge and skillset to use personalization, you are ready to implement Multivariate (or A/B) testing of content.

Optimize Your Content's Effectiveness

First, it is important to truly understand the basis/criteria on which Sitecore DMS measures your Test Variants. How do you know that Content A is "better" than Content B? The answer is not necessarily which variant resulted in a higher conversion rate of a particular goal, but that certainly plays into it. Actually, when you run a multivariate test in an effort to optimize your content, DMS is comparing which content variant resulted in higher Engagement Value Points. So you can see, that while you may be interested in knowing which variant resulted in Goal A being achieved more frequently, in reality, the test is going to focus on which produced higher "engagement". Of course, Goal A may have contributed to that engagement, but so could have Goals B, C, D, and so on. I think it is important to make that distinction before digging in.

Sticking with the homepage feature content example we used in the previous post, let's assume we want to see which content in this component will drive more user engagement.

Step 1 - Create Alternate Content

For your homepage feature to be tested, you first need to create the content variation you want to compare. The screenshot below is an example of feature content intended for website visitors in the city of St. Louis. Your website content structure will differ.

Sitecore DMS Multivariate Testing

Step 2 - Create Your Test

With your test content available now, access your homepage via Page Editor and click your feature content component. Select the Test Component icon and you'll be able to add a Variation (or more) and choose the content it should display.

Sitecore DMS Multivariate Testing

Step 3 - Start the Test!

You'll notice the Page Editor Testing tab has a little red box with the number 'one' indicating you have one test created and ready to go.

Sitecore DMS Multivariate Testing

Click "Start Test", give it a name and it will be off and running!

When you end the test is up to you. It depends on when you feel you have a large enough sample size of data to make a fair decision on which content stays, and which one goes. Come back to this tab to see an updated Status message that will tell you: 1) the date the test started, 2) the duration it has been running, and 3) the number of experiments encountered.

Sitecore DMS Multivariate Testing

Once you click "Stop Test", you'll be presented the results, each content variant's Engagement Value produced. You'll simply click the option you deem the "winner" and that content will be displayed to users. Test complete. Real objective data. Optimized content with provable results.

That's a Wrap!

Hopefully by biting off some manageable pieces of DMS, you have successfully implemented some quick wins and are starting to see the possibilities of more sophisticated digital marketing efforts or tailoring something to your unique organization. We would love to be of service and provide Sitecore DMS Strategy and Implementation to help get you to the next level of digital marketing.

comments powered by Disqus