Create an A/B test rule to find out which content is the best suitable for the segment you are targeting.

1. Click Zones, and then select the channel for which you want to create the A/B test rule.
2. Do any one of the following actions:
   • If no personalization is added to the zone, click Actions > Create next to the zone for which you want to create A/B test rule.
   • Click Actions > View details next to the zone for which you want to create a rule. In the Content personalization pane, click Rule editor, and then click New rule.
   • You can also create an A/B test from an existing rule. To do so, perform the following steps.
     Open the rule that you want to convert to A/B test. In the Rule details screen, click the Convert link, which is located next to This is not an A/B test.
3. In the Select the personalization type pop up, click A/B test and then click Next.
4. In the rule configuration screen, enter your unique rule name and control group percentage.
5. Select the primary goal of the test. Currently, we support only the click rate as the primary goal. For more information, see Click rate
6. Select the Rule duration.
7. Click Configure A/B test.
8. Enter the number of different content variants you want to show to the channel visitors. For example, if you want to show two different content variants to two different sets of visitors, then the number of content variants would be 2.
9. Enter the statistical significance percentage. The statistical significance percentage should be within the range 50 - 100.
10. Notify me when the test ends option is selected automatically. Click Next.
11. Here, you can see all the details that are entered in the window.
12. To select the content variant, click Select content. You can add up to 10 variants in one A/B test rule.
13. To select content for each variant, do any one of the following actions:
    1. If you select Acoustic Content as the content management system, you can either select the content from Acoustic Content Palette or you can enter the content in the form of HTML blob. If you select Acoustic Content, browse the content from the Acoustic Content Palette window. Use the Search bar to find the content type that you want.
    2. If you registered Other as the content management system, you can enter the content in the form of HTML blob or URL.

The Content location field displays the path of the selected content in Acoustic Content.

14. In the Control traffic field, enter the percentage of visitors who see the default content. In the Test traffic fields, enter the percentages of visitors who see the particular variant. You need to split the percentage among the different content variants(Test group1 up to Test group10). The total percentage of traffic allocation must equal 100%.
15. Click Create. The content variants are added to the Content variants pane.

16. In the Segment pane, click Select segment. You can search a segment by searching for the name or create a new segment. Click Select
17. Do one of the following actions:
    • Click Save. The rule is saved in an inactive state.
    • Click Save and publish.The rule is saved in active state. The rule is published but will not run or be applied until the start date. If the start date and time is set to future date, then status of the rule is Active - Scheduled.

Convert a rule to an A/B test

You can use an existing test rule and enable A/B testing to find out the best suitable content for the segment that you want to target for personalization. To create A/B test using existing rule, complete the following steps:

1. Log in to Acoustic Personalization.
2. Click Zones, and then select the zone for which you want to update the rule, and click Actions > View details.
3. Click Rule editor to view the draft rules.
4. Click action menu of the rule that you want to convert, and then click Edit.
5. Click Convert.
6. In case you want to change the rule configuration, you can edit the rule name, control group or duration, and then click Next.
7. Enter the number of different content variants you want to show to the channel visitors. For example, if you want to show two different content variants to two different sets of visitors, then the number of content variants would be 2.
8. Enter the statistical significance percentage.The statistical significance percentage should be within the range 50 - 100.
9. Click Next. Here, you can see all the details that are entered in the window.
10. To select content, you can do any one of the following actions:
    1. If you select Acoustic Content as the content management system, you can either select the content from Acoustic Content Palette or you can enter the content in the form of HTML blob. If you select Acoustic Content, browse the content from the Acoustic Content Palette window. Use the Search bar to find the content type that you want.
    2. If you registered Other as the content management system, you can enter the content in the form of HTML blob or URL.
11. The Content location field displays the path of the selected content in Acoustic Content.
12. In the Control traffic field, enter the percentage of visitors who see the default content. In the Test traffic fields, enter the percentages of visitors who see the particular variant. You need to split the percentage among the different content variants(Test group1 up to Test group10). The total percentage of traffic allocation must equal 100%.
13. Click Create.The content variants are added to the Content variants pane.
14. You can view the segment or use Change segment to select a new segment. Do one of the following actions:
    • Click Save.The rule is saved in an inactive state.
    • Click Save and publish.The rule is saved in active state. The rule will be published but will not run or be applied until the start date. If the start date and time is set to future date, then status of the rule is Active scheduled.
15. To confirm, click Ok.

The A/B test is saved and scheduled according to the rule duration. 

Edit a draft A/B test

You can edit an existing A/B test rule by updating the test rule details, content, and segment to which the personalization is targeted.

To edit a test, complete the following steps:

1. Log in to Acoustic Personalization.
2. Click Zones, and then select the zone for which you want to update the rule, and click Actions > View details.
3. Click Rule editor to view the draft rules.
4. Click action menu of the A/B test rule that you want update, and then click Edit.
   • You can make any necessary changes to the control group percentage, content variants, and segment .
   • In the A/B test configuration pane, click Edit.
   • Edit the required test details such as number of content variants, test duration, and statistical significance.
5. Click Save.The rule is updated and saved as draft.

Note Ensure that you unpublish the rule to edit the rule details. After you publish the rule, you can edit only its test duration.

Edit an ongoing A/B test

You can edit the statistical significance and duration of A/B test while your A/B test is in progress.

To edit the A/B test, complete the following steps:

1. Select the zone for which you want to view the rules. Click Actions > View details.
2. Click Published rules and then click Actions > Edit.
3. You can provide the value for statistical significance and end date and end time, and then click Save.

Note You are allowed to modify only the statistical significance and the end date while the test in progress. To modify the other details, you can create a draft of this A/B test rule.

Publish an A/B test

You can activate an A/B test rule on your channel to determine the winning content.

1. Log in to Acoustic Personalization.
2. Click Zones, and then select the zone for which you want to publish the rule, and click Actions > View details.
3. Click Rule editor to view the draft rules.
4. Click action menu of the A/B test rule that you want publish, and then click Publish.

You can publish the rule for which A/B test is already set up. If the A/B test duration is expired, you can change the A/B test duration and then publish the rule.

After you publish the draft rule, it becomes active and is used for personalizing the selected zone.

Unpublish and republish a Paused - Inconclusive rule

You can unpublish the A/B test rule if the duration of your A/B test expires and the statistical significance is still not reached.

1. Ensure that you are on the Zone details page.
2. Click Published rules to view all the published rules.
3. Click action menu of the A/B test and then click Unpublish.
4. The status of the A/B test is Paused - Inconclusive. After you unpublish, it changes to Inactive - Stopped.
5. You can later publish it again using Publish. Status of the rule changes back to Paused - Inconclusive.

You can resume your Paused - Inconclusive rule by using Edit. You can enter the new end date, end time, and click Save and publish. The status of your test is changed to Active - In progress.

View the A/B test status

You can view the status of your A/B test and the various stages that the rule goes through.

1. Click Zones. Select the channel.
2. Click Actions > View details next to the zone for which you want to view all the rules.
3. The Published rules tab lists all the published rules in the zone and the Rule editor tab lists all the draft rules that are created in the zone.

Status of A/B test

Status	Description
Active - Scheduled	A/B test rule is published and is scheduled to run during the specified date range.
Active - In progress	A/B test rule is published and the end date is in the future and statistical significance is not reached yet.
Active - Completed	A/B test rule is published and its statistical significance is reached within the schedule.
Active - Stopped	A/B test rule is stopped.
Inactive - Completed	A/B test rule is completed but the rule was unpublished.
Inactive - Scheduled	A/B test rule is unpublished, and the rule's start date can be either in the future date or set to the date when the rule is published.
Inactive - Expired	The schedule of the A/B test is over, but the rule was never published.
Inactive - Stopped	Statistical Significance for the A/B test is not reached and the test was unpublished.
Paused - Inconclusive	A/B test rule is active, and the statistical significance is not yet reached but end date has passed the current date.

You can stop an A/B test at any time to know the intermediate results of the test.

Impact of stopping an A/B test

Statistical significance value is calculated based on the click rate of visitors on the channel. If you stop an A/B test, the statistical significance is computed uses the click rate data from the limited set of visitors and may not reflect the real-world scenario.

For a more reliable result of statistical significance, it is recommended to wait for about a week from the A/B test start date.

Stop an A/B Test

To stop a test, complete the following steps:

1. Ensure that you are on the Zone details page.
2. Click Published rules to view all the published rules.
3. Click action menu of the A/B test that you want to stop and then click View details.
4. To stop the test, do any of the following actions:
   • In the Rule details pane, click Stop test.
   • In the A/B test configuration pane, click Performance. Click Stop test.
5. To confirm, click Stop test

In the A/B test configuration pane, the test status is changed to Stopped.

You can use Resume to resume the test later.

Resume a stopped test

When you stop the A/B test, you can resume the test later to continue testing and achieve the statistical significance.

1. Ensure that you are on the Zone details page.
2. Click Published rules to view all the published rules.
3. Click action menu of the A/B test that you want to resume and then click View details.
4. The status of the A/B test is Active - Stopped.
5. Do any of the following actions:
   • In the Rule details pane, click Resume test.
   • In the A/B test configuration pane, click Performance details. Click Resume test.

After you resume the test, it continues to run again. The test status is changed to Active - In progress.

Resume a stopped test that is expired

You can resume the A/B test that is expired by changing the end date of the A/ B test rule. If the duration of the A/B test is expired, then you can change the end date of the A/B test to extend the A/B test duration.

1. Ensure that you are on the Zone details page.
2. Click Published rules to view all the published rules.
3. Click action menu of the A/B test and then click View details.
4. The status of the A/B test is Active - Stopped. In the Rule details pane, click Change duration.
5. In the Change rule duration pop up, enter the end date, and end time and click Change duration and change the date and time. Click Save and publish. Resume test button is available now.
6. Click Resume test. The test status is changed to Active - In progress.

Resume a Paused - Inconclusive rule after it is unpublished

1. To unpublish Paused - Inconclusive rule, click Published rules. Click action menu of the A/B test, and then click Unpublish. The test status is changed to Inactive - Stopped.
2. Click action menu of the Unpublished A/B test and click Change duration.
3. In the Change rule duration pop up, enter the new end date, and end time and click Change duration to save the changes.
4. You can again publish it using the action menu of the test and clicking Publish. To view your rule, click Published rules The test status is changed to Active - In progress

Extend the duration of an expired A/B test

You can extend the duration of the A/B test if the duration of your A/B test expires and the statistical significance is still not reached.

To change the A/B test duration, complete the following steps:

1. Ensure that you are on the Zone details page.
2. Click Published rules to view all the published rules.
3. Click action menu of the A/B test and then click View details.
4. You can see the status of the A/B test is Paused - Inconclusive. In the Rule details pane, click Change duration and resume.
5. In the Change rule duration pop up, enter the new end date, and end time and click Change duration and resume to save the changes. The test status is changed to Active - In progress.

Your A/B test rule is completed when its statistical significance is reached within the schedule.

Scenario 1: End duration is a specified date and time

When the end date of your A/B test and the statistical significance is reached for the rule, then test is complete. The system automatically sets the winning variant according to the click rate.

Scenario 2: End duration set as "When unpublished"

You can manually unpublish the A/B test. After you unpublish, the status of your rule changes according to the following conditions:

• If statistical significance is reached, then the A/B test is completed and winner is decided. The rule status changes to Inactive - Completed.
• If statistical significance is not reached, then the A/B test is not completed and the rule status changes to Inactive - Stopped.

You will see the notifications for the A/B test that you create, publish, and whose duration is complete.

When your A/B test is published and its duration is complete, you receive notifications on the user interface, providing the test status and other relevant information.

To be able to see the notifications, during the A/B test configuration, ensure that the Notify me when the test ends option is selected by default in the Select the post-test actions field.

Here are some of the common notifications that you see.

Notifications in A/B test

Test duration complete for [TestName].

The [TestName] reached the end date and is paused because a statistically significant winner was not determined.

Notification displayed when the A/B test completes its specified duration, and is in the paused state due to lack of a statistically significant winner.

Test duration complete for [TestName].

The [TestName] reached the end date, and [WinningVariantName] is identified as the winner.

Notification displayed when the A/B test completes its specified duration; and has determined a statistically significant winner.