Edit the sample size calculation for an experiment

When you run an experiment, a healthy sample size enables you to make accurate statistical conclusions. Your experiment has to reach a set sample size, otherwise there won't be enough evidence to ascertain that the test results are not due to chance.

Sitecore Personalize automatically calculates the minimum sample size required to achieve statistical significance based on your chosen primary goal. You can edit the way the sample size is calculated by entering different parameters for this calculation. The experiment must have a draft status to do this.

Warning

Do not stop an experiment before the minimum sample size is reached. If you stop the experiment before the minimum sample size is reached, the test becomes invalid.

To edit the sample size calculation:

In the Experiment options pane, in the Sample size section, click .
In the Calculate sample size pane, edit the following fields. The default values, along with their allowable minimum and maximum limits, are provided for reference:
- Base Rate % - the current conversion rate of the object you want to test. The default is 2% but you can adjust it between 0.5% and 50%.
  
  Increasing the base rate decreases the required minimum sample size.
- Minimum detectable difference % - the amount of lift or change from the Base Rate you want the test to detect. The default value is 20% but you can adjust it between 1% and 50%.
  
  Increasing this value decreases the required minimum sample size. You can adjust the test sensitivity if you want to change this.
- Confidence Level % - the amount of evidence your organization requires for proving that the difference between the variants' conversion rates is not due to chance. The default is 95% but you can adjust it between 80% and 99%.
  
  95% is an accepted standard to reach statistical significance. The confidence level you set depends on your organization's risk-tolerance for accuracy in experiment results. Increasing the confidence level increases the required minimum sample size.
Note

All new experiments will be automatically validated against the minimum and maximum limits to ensure proper configuration. If you attempt to restart an existing paused experiment with parameters outside the permitted range, you’ll be prompted to adjust them before proceeding.
Click Save.

Do you have some feedback for us?

If you have suggestions for improving this article,