Auto Optimize is powered by a Multi-Armed Bandit algorithm. In a similar way to A/B tests, you can add items, for example, content such as Slots, SmartBlocks, Triggers or Experiences, to the optimization, but instead of needing to wait a set period of time, or for a specific number of impressions, the system's machine learning algorithms automatically allocate traffic to the best performing items.
Auto Optimize continues to allocate a small amount of traffic to the underperforming items and to monitor their performance. If it detects a noticeable change in an item's performance, then it automatically shifts traffic around to ensure that the best performing items continue to be shown the most.
Auto Optimize is an optimization for your personalized content. It works quickly, to ensure that you’re gaining maximum return from the items that are within the optimizations.
Key information about Auto Optimize
You can set the goal you want to achieve for the items within the optimization.
Auto Optimize works on items in the optimization being seen, and not on them being clicked. This means reporting may differ from other areas of the system, and that items in the test don’t need to be clickable for the Auto Optimize to work.
SmartBlocks used with Auto Optimize must be metered (dynamic).
Auto Optimize monitors the performance of all content, however, it doesn’t change the item it is showing unless it sees a significant change in performance. Therefore, you might see fluctuations in your goal metric, but not see a significant change in which item the Auto Optimize is showing. This is to be expected. If there is a consistent or significant change in performance, the impressions change.
The speed that Auto Optimize works at is impacted by how strong the difference in results is between the test items. If one test item shows a much greater percentage of successes towards your goal over the other content, then usage of the better performing item is quickly increased. This can be within 1000 impressions or sends. However, if the difference in success of the items is less dramatic, it takes longer to adjust.
Use cases
Individual image content (SmartBlocks) such as the hero banner: add several banners and let Auto Optimize ensure the top performing ones are being allocated the most traffic.
Trigger programs to optimize best performing email content or send times.
Whole Experiences: for example, set up a few different versions of a flash sale and Auto Optimize can quickly work out the best performing one and allocate the higher proportion of traffic to it.
Create and manage an Auto Optimize
To access Auto Optimize, you must have the Advanced Testing and Optimization module. If you’re interested in adding the module to your account, or would like to discuss the module, contact your Account Manager.
Before setting up an Auto Optimize, first you should create the content that you want to include in the Optimization; this could be multiple SmartBlocks to be added to a Slot, Trigger programs, or Experiences.
Learn more
Once you’ve created your content:
Go to Optimize in the left hand navigation menu.
Select CREATE OPTIMIZATION.
Expand the Location drop-down menu, and select where you want to add the optimization:
Experiences
Website slots
Email slots
Triggered email slots
Triggered email programs
Select the option (Slot, program, or Experience) you want to use, then select CONTINUE.
Drag the items (SmartBlocks, Trigger programs or Experiences) that you want to use from the left into the decision tree on the right. Position the items on top of one another.
When you have a group of multiple items, select the top row to open the Optimization panel on the left.
Select Auto Optimize.
Expand the Select the goal for this Optimization drop-down menu and select from:
Increase Conversion
Increase Average Order Value
Increase Identification Rate
Decrease Bounce Rate
Increase Site Visits
Increase Average Time On Site
Increase number of items in purchase
Increase revenue
It’s possible to edit an Auto Optimize to change the goal. This effectively resets the test with the new goal now the focus for success.
Optionally, enter a Sensitivity window.
Select SAVE.
The Auto Optimize is now set up and ready to go. If the Slot, Trigger or Experience Rule is live, then the Auto Optimize becomes live as well.
Your new Auto Optimize now appears in the Optimize Center.
The coloured dot indicates if it’s active.
Select the row to return to the optimization setup.
Optimization sensitivity
An Auto Optimize reacts rapidly to changes in behaviour when first started, as it has less data to judge the winner from. As impressions build up, this sensitivity gets less, since it uses all the data that has been gathered to make decisions.
This can be the right behaviour, if you want it to find the winner fast and only change the winner for substantial, prolonged changes in performance. However, if you want the optimization to remain at the same level of sensitivity to performance changes, then you can configure the sensitivity window.
For example , if content is associated to weather changes, it needs to keep the same sensitivity over time. To configure the sensitivity, you set the number of most recent impressions/sends the Optimization should use when judging the performance, to a maximum of 100,000. If it is set to a low threshold such as 1000, then the optimization is highly sensitive to any performance changes of the content, trigger, or experience. If set higher, it’s less sensitive.
This is an optional field and can be left empty if the optimization should always judge performance on all impressions/sends.
Reporting
Go to Optimize in the left side menu.
Select the options (three dots) menu for the optimization you want to view reporting for.
Metrics are person-based, and based on that person seeing an item within the optimization. This differs to other metrics that may be based on attribution. Learn more in Attribution of purchase values.
The Auto Optimize has set periods it reviews to establish the success of a test item being shown. For SmartBlocks and Experiences the goal must be achieved within a time frame of 1 day — 24 hours — of seeing the content, for the system to consider them a success, and 7 days for a Trigger program set up.
The standard report displays different data metrics collected against each of the items within the optimization.
Items within the optimization are automatically added to the report for you.
To access this report from the Optimize Center:
Select the options (three dots) menu for the optimization you want to view reporting for.
Select the Open Report icon.
Person detail report
Not available for contextual auto-optimize.
This report displays a table showing a row for each item that has been shown and the outcome — success or not. The data is only from the last 7 days or last 1000 impressions, whichever is reached first.
To access this report from the Optimize Center:
Select the options (three dots) menu for the optimization you want to view reporting for.
Select the Open Person Detail Report icon.
