Headless personalization and A/B/n testing

View performance analytics for a headless experience

Important

Feature availability is part of a phased rollout. Your organization may not see this functionality yet. It will become available when your environment is included in the rollout.

After a headless experience becomes Live and your application begins running it, you can monitor its performance in the Headless personalizations dashboard.

For performance data to appear, make sure:

At least one goal is configured.
The headless experience is Live.
The application is running the experience and sending the events used to measure its goals.

To monitor the headless experience's reach, deliveries, errors, and breakdowns by language and channel, view the Personalization insights dashboard instead.

To view performance analytics:

In the navigation menu, click Optimization.
In the left sidebar, click Tests & personalizations.
Select Headless.
Select the live headless experience you want to monitor.

The Configuration page opens.
Click the Performance tab.

The Headless personalizations dashboard opens.

Tip

You can also access the dashboard from Optmization directly. In the left sidebar, click Analytics > Headless personalizations. Then select the headless experience you want to view.

Headless personalizations dashboard

The dashboard shows how the experience is performing against its configured goals. For an A/B/n test, it also shows results for each variant.

Status

The status banner shows the name and current status of the headless experience.

For an A/B/n test, it can also show:

In progress - the test is running and, when enough data is available, shows the leading variant.
Completed with a winner - the test reached statistical significance and a winning variant was identified based on the primary goal.
Ended but inconclusive - the test ended without reaching statistical significance.

Goal over time

The Goal over time graph shows performance over time for the selected goal and, when configured, its target value.

For a personalization, the graph shows performance for the personalization experience. For an A/B/n test, it shows performance for each variant.

Use the following filters to change the data displayed in the Goal over time graph and Performance per variant table:

Goal - select the goal to analyze.
Device - select a device type, such as Mobile or Web. The default is All.
Time - select a time period or specify a custom date range. The default is All available data.

The x-axis shows the selected time period, and the y-axis shows values for the selected goal.

Hover over a data point to view the goal value and, when configured, the target value. For an A/B/n test, the tooltip shows values for each variant.

Performance per variant

The Performance per variant table shows results for the selected goal. For an A/B/n test, it compares results across variants.

Depending on the experience type, the table can include:

Variations - the name of the personalization or test variant.
Uplift - the percentage difference in the selected goal metric compared with the baseline. The margin of error (±) represents a 95% confidence level.
Unique visits - the number of unique visits included in the results.
Name - the value achieved for the selected goal. The margin of error (±) represents a 95% confidence level.
Confidence index - the statistical reliability of the result. Higher values indicate greater reliability. This estimate is valid only after the minimum sample size has been reached.

Performance overview

The Performance overview bar chart shows performance against the primary goal. For an A/B/n test, it compares performance across variants.

The section also includes:

Target - the target value configured for the primary goal.
Days running - the number of days the experience has been running. This counter resets if the experience is paused and restarted.
Total unique visits - the total number of unique visits recorded toward the primary goal.
Details - performance data for the primary goal and any secondary goals. For an A/B/n test, the table compares results across variants.

If you have suggestions for improving this article, let us know!