Set automated post-test actions based on the experiment outcome
With automated post-test actions, you can specify what you want to happen after an experiment reaches statistical significance based on the experiment outcome. You can choose to direct all traffic to the winning variant, redirect traffic back to the control variant, or end the experiment, depending on whether there's a winner or if the results are inconclusive.
You must add at least one goal to your experiment before you can set automated post-test actions.
To set automated post-test actions based on the experiment outcome:
-
In the experiment builder, click the Post-test tile.
By default, a new experiment is configured to assign all traffic to the winner if there is one, or to keep running the experiment if the results are inconclusive.
-
To edit the default post-test actions, click Edit end of test settings.
-
In the End of test settings pane, select the action you want to happen automatically after your experiment reaches statistical significance, based on the outcome:
-
If there's a winning variant:
-
Assign all the traffic to the winning variant - to redirect all guest traffic to the best-performing variant.
-
Assign all the traffic back to the control variant - to redirect all guest traffic back to the control variant.
-
Complete the test - to end the experiment and stop traffic assignment to all variants.
-
-
If the results are inconclusive:
-
Keep running the test - to continue the experiment and maintain the current traffic assignment.
-
Assign all the traffic back to the control variant - to redirect all guest traffic back to the control variant.
-
Complete the test - to end the experiment and stop traffic assignment to all variants.
-
-
-
Click Save.
ImportantAfter saving the post-test actions and the experiment is live, you can no longer update its primary goal, whether or not it has reached statistical significance.
However, you can add or edit secondary goals at any time, though these changes won't affect the experiment's outcome.
After the experiment reaches statistical significance, a banner will appear in the Build summary when you open the experiment. This banner displays the experiment outcome and informs you that your post-test actions are in effect, including a timestamp.