Introduction to experiments in Sitecore CDP

Abstract

Provides an introduction to running experiments in Sitecore CDP across multiple channels.

When you want to compare how a current variant performs against another one, you can set up an experiment. An experiment is essentially an A/B test of two or more variants. Creating a controlled, random experiment can improve the outcome of a business goal such as clicks, purchases, website metrics, and more. An experiment can test anything from a different color scheme to a completely different version of a webpage, on interactive and outbound channels.

After you start your experiment, you can view how your variants are performing against each other in analytics. This enables you to see which variant is leading or if a winner has been declared.

When you run an experiment, you can question potential changes to a customer experience, however small or large the potential change might be. Each variation of a customer experience can be captured as a variant. Each variant provides you with quantitative data, eliminating any speculation about which design or webpage element results in more conversions, or other goals.

Before you start your experiment, you must assign a primary goal to the experiment. Sitecore CDP uses the metrics of the primary goal to measure variant performance when determining a leading or winning variant. After a variant has reached statistical significance, you will see a message that a winner has been declared.

Ideas to test in a web experiment:

  • Design layout

  • Images - including placement, number, style, and whether the image in impressive

  • Headlines - varying the content, length, size, font, and so on

  • Company branding

  • Text - changing the content, style, font, size, and placement

  • Call-to-action (CTA) buttons - changes to the text on the button, variations of sizes, colors, and page placement

Ideas to test in an interactive experiment:

  • Any website elements that are managed by the back end

  • Pricing discounts

  • Subscription offers

  • Showing real-time inventory levels

  • Showing estimated manufacturing or shipping times

  • Urgency widgets