Ending tests

Abstract

Understand which tests end automatically and which tests require user intervention to end.

Page tests and basic content tests stop automatically, either when Sitecore finds a test winner, or when the maximum test duration has passed. However, if you start a component test (an A/B test, multivariate test or personalization test), you must take action to end the test. These tests do not end automatically, even if Sitecore has found a winner.

By default, Sitecore XP selects a winner automatically for page tests and basic content tests. If there is enough data to select a winner before the maximum test duration is reached, then Sitecore replaces the original experience with the winning experience and stops the test.

If you select the Select winner manually option when you start a page test, you must pick the winner before the maximum test duration has passed. You can pick a winner manually from the Test results dialog.

If the maximum test duration is reached and no winner has been selected (neither automatically by Sitecore nor manually by you), then the test automatically stops, and the new experience is declared the winner and replaces the original experience.

If you select the Automatically select a winner option when you start a multivariate or personalization test, and if Sitecore gathers enough data to determine a winner, then Sitecore sends an email message to the user who started the test. You must do one of the following to end the test:

  • Accept the test result to declare the test winner and end the test.

  • Review the test results and pick a winner manually.

Important

Even if a winner has been found in a component test, you must take action to end the test.

If the maximum test duration passes and no winner is selected (neither by you nor by Sitecore), then you receive an email message prompting you to take action. The email offers you the following options:

  • Review the results and pick a winner.

  • Extend the test for a specified period (you receive a follow-up email after the extension period).

  • Stop the test and revert to the original experience.

If you do not take one of the above actions, the test continues indefinitely and you do not receive further email reminders.

It is possible that no one receives the test results email message that Sitecore XP sends when a test winner has been found or the maximum test duration has elapsed. This might happen, for example, if the email address is no longer used. For this reason, it can be helpful to review the Active Tests report in Experience Optimization periodically. Users with access rights to Experience Optimization can see lists of tests created by all users. From the Active Tests report, you can access the test results for any test, and pick a winner or end the test.