Create and run page tests

Abstract

This topic shows how to create page tests.

Page tests are different from content tests: a page test tests a complete page against another page. The pages in the test can be different versions of the same page, or they can be completely different pages.

You can set up page tests in the Experience Optimization application, and it is also possible to start a page test from the Experience Editor.

To select the pages to test:

  1. In Experience Optimization, click Create Page Test.

  2. Select the original page to test. You can navigate to a page in the content tree, or you can search for it.

  3. Click the Language field to open the drop-down menu and then click the language variant you want to test.

  4. Click Select.

  5. Click the plus icon to select pages to test against:

    The Pages to test against section

    In the Add page dialog, you have the following options:

    • Click Use existing page to select another existing page to test against.

    • In the Test against the previous version of the page field, click the drop-down menu to select a previous version of the current page (if one exists).

    • Click New page version to create a new version of the current page. You can edit the new version before you start the test.

    The Add page dialog box
  6. Click the plus icon again to select more pages until you have selected all the pages that you want to test against each other.

When you have selected the pages that you want to test, click Review and start to review the test. Sitecore displays information about the test, and you can adjust a number of parameters for the test.

Sitecore displays a preview of the pages that you have included in the test. One page is selected and you can see a large image and details for the page. To select a different page, click the small image for that page.

You can see the following information:

  • An information bar that tells you how long the test is expected to run before it can declare a winner

    662B9E9511FC4AA1AF5212B1BB810899.png
  • Information about the currently selected page

  • A test summary

Adjust the following parameters:

  • My expected effect of changes: Make a guess about the effect your changes will have on the engagement value of the page.

    In a test with multiple pages, you are guessing about all pages against the original.

    Sitecore calculates a score to show how good you are at guessing.

  • Percentage of visitors exposed to test: Specify the percentage of visitors that you want exposed to the test.

    For example, if you set the percentage at 40%, 40% of visitors will see one of the variations that you have created. The original variation is part of these. The remaining visitors – 60% in this example – will see the original variation.

    Sitecore calculates test results based on the visitors it exposes to the test.

  • Statistics: Select the statistical confidence level that the test needs to reach before it can declare a winner. You can select 90%, 95%, or 99%. The higher the level, the longer the test needs to run.

  • Test objective: Select a test objective. You select either one of the goals specified in the Marketing Control Center, or you select Trailing Value/Visit (the engagement value based on page views occurring after the visitors encountered the page being tested, divided by the number of site visits).

  • Select how to pick a winner: Specify the method for selecting a winner of the test:

    • Automatically select a winner based on the test objective

    • Automatically select a winner based on the test objective, unless it significantly decreases engagement value

    • Manually select winner

  • Duration: Specify the minimum and maximum time for the test to run:

    • Minimum: select 3, 7, or 14 days.

    • Maximum: select 14, 30, or 90 days.

If you created a new version of the original page, or you want to change one of the pages after reviewing the test, you can edit a page before you start the test. Before you leave Experience Optimization to edit the page, save the page test as a draft.

To save the draft test and edit a page:

  1. Click Save to save the test.

  2. Navigate to the Experience Editor or the Content Editor, and edit the page. Save your changes and move the page through the workflow.

  3. In Experience Optimization, click Draft page tests. Click the page test that you created earlier.

Now you can continue to review or start the test.

When you are done reviewing and adjusting, start the test:

  • Click Start test.

When the test has found a winner, you can view the test results. You can see the test summary as well as:

  • An overview that shows the best page effect, the statistical confidence level, and the score gained.

  • A bar chart that shows the engagement value for the different pages in the test.

  • The conversion rate based on the goal that you select.

To see the test results:

  1. In Experience Optimization, expand the Tests section and click Historical Tests.

  2. In the Historical Tests list, click the relevant row.

The Test summary panel includes information such as when the test was created and what the test objective was.

The Reports pane displays the same information as the Test Results dialog box displays in the Experience Editor. For more information about the test result data that is shown, see the Content test results topic.