- Headless A/B/n tests
Edit advanced options for a headless A/B/n test
Feature availability is part of a phased rollout. Your organization may not see this functionality yet. It will become available when your environment is included in the rollout.
Use advanced options to control the statistical setup and visitor assignment for a headless A/B/n test.
You can configure:
- Minimum sample size - determines how many visits each variant needs for the test.
- Assignment method - controls how visitors are assigned across tests.
- Traffic allocation - controls how much eligible traffic can participate in the test.
SitecoreAI provides default values for these settings. You can use the defaults or adjust them while the test is in Draft.
Minimum sample size
The minimum sample size is the estimated number of visits required for each variant to reach statistical significance. By default, SitecoreAI calculates 21,110 visits per variant.
To change the minimum sample size:
-
Open the A/B/n test you want to edit.
-
In the Advanced options section, click the expand icon to open minimum sample size settings.
-
Edit one or more of the following:
- Base rate - the current conversion rate of the control variant. The default is 2%, and you can set a value from 0.5% to 50%. A higher base rate reduces the required sample size.
- Minimum detectable difference - the smallest change from the base rate that you want the test to detect. The default is 20%, and you can set a value from 1% to 50%. A larger difference reduces the required sample size.
- Confidence level - the level of certainty required to determine that a difference between variants is statistically significant. The default is 95%, and you can set a value from 80% to 99%. A higher confidence level increases the required sample size.
SitecoreAI recalculates the minimum sample size as you change these values.
-
To restore the default values, click Reset.
-
Click Save.
Assignment method
The assignment method controls how SitecoreAI assigns visitors across A/B/n tests.
By default, the assignment method is Unique to A/B/n test.
To change the assignment method:
- Open the A/B/n test you want to edit.
- In the Advanced options section, locate Assignment method.
- Select one of the following:
-
Unique to A/B/n test - assigns visitors independently for this test.
Example: A visitor assigned to Variant B in one test can be assigned to Variant A in another test.
-
Universal - shares visitor assignment with other tests that also use universal assignment.
Example: A visitor assigned to Variant B in one universal test is assigned to the corresponding Variant B in other universal tests.
-
Traffic allocation
Traffic allocation controls the percentage of eligible visitors that can participate in the test. By default, SitecoreAI allocates 100% of eligible traffic.
By default, SitecoreAI allocates 100% of eligible traffic to the test. You can reduce this percentage and increase it later.
Traffic allocation controls how much eligible traffic enters the test. To control how traffic is split between variants, see Assign traffic to headless A/B/n test variants.
To change the traffic allocation:
-
Open the A/B/n test you want to edit.
-
In the Advanced options section, next to the traffic allocation percentage, click the edit icon .
-
Use the slider to select a value from 1% and 100%.
Example: Set traffic allocation to 25% to initially inlcude only 25% of eligible visitors. You can gradually increase the allocation as you expand the test.
-
Click Save.