Programs are here to help you maintain hands-free customer engagement. However, you still need to keep an eye on the program's performance and be aware of any unexpected events.

Will you be able to react in time if things don’t go as you planned?

On the other hand, what if things go better than expected? Can you spot an opportunity quickly enough?

It's hardly possible to spot unusual events in your program instantly. Anomaly detection for emails in a program relieves you from this necessity and notifies you whenever a program requires special attention.

What is anomaly detection

Anomaly detection is a feature in programs that tells you about unexpected changes in email metrics and helps you make the best possible decision in the right time. Instead of running a report or wading through message logs, you get notified about unusual events with an in-app notification or an email. Anomaly detection is based on an AI-powered model that monitors multiple program metrics for every email step in every program. The longer your program runs, the more the model learns about what to expect and when to grab your attention.

To check for anomalies in your program, go to the Metrics tab in the Program insights panel. At the bottom of the tab, you will see a list of email names and the number of anomalies detected in each of them. For an overview of a single email, click the email anomaly description in the sidebar to open the Email panel and go to the Metrics tab.

From the initial summary view, you can dive even deeper. Each metric summary links to an expanded view that reveals additional details about the performance of that metric over time. Any unexpected program behaviors – anomalies – are highlighted in charts that plot the history of each metric. 

The Metrics tab displays results up to the previous 450 days (30 days is a default). 

How anomaly detection works

The email anomalies evaluation model reviews program activity nightly. The model calculates expected results, compares them to the actual results, and highlights unusual behavior. 

As a program runs, the anomaly detection model continually defines a range of expected values for each email metric, based on an analysis of previous performance and customer responses. The upper and lower expected values define the bounds of the metric. Within these bounds, the model defines a specific expected value that is most likely to occur. Trends for the bounds and expected values become apparent over time. 

The model displays dates and times in GMT.

Enable anomaly detection notifications

You can automatically receive a notification when an anomaly is detected. Take a step further and also enable email notifications to receive anomaly notifications by email.

Note: You must enable anomaly detection emails for each program individually.

1. Open a program and click Metrics in the Program insights tab.
2. Turn on Notify me about anomalies. You'll start receiving in-app notifications.
3. Enable email notifications if you want to stay in the loop outside Campaign.
   1. Go to Notifications > Settings.
   2. Turn on Email notifications. We'll use the email address associated with your profile to alert you to anomalies that occur.

Availability and stored data

To display program metrics and take advantage of anomaly detection modeling, your organization must meet the following criteria:

• Provisioned for Performance Insights
• Not provisioned for Transact
• Uses the refreshed Campaign interface
• Actively using the latest programs canvas

Anomaly identification and notification data is hosted exclusively in a United States data center. The stored information consists of user notification preferences and aggregate history for program transmission and response activity. No personal data is ever required or stored.

Use cases

Monitor email deliverability

Your program sends an email with weekly promotions to all new customers and other customers who qualified for a Preferred account in the past week. Audience updates and new email content are loaded every Saturday night. The program starts sending emails on Sunday to have emails in the inboxes by Monday morning. This week, the update runs as scheduled.

On Monday morning, you receive a system notification that there’s been an anomaly for your Weekly Promotions email. Better check this out.
You click the notification, which takes you directly to that email’s Metrics tab. You see there’s been a spike in the number of hard bounces, beyond the highest expected response – this indicates an anomaly.

You click Hard bounces to get more insight. The diagram compares the current metrics to expected values. The alert makes it clear that this week’s emails aren’t getting through. 

After reviewing the database, you’ve identified the culprit. The audience data entered on Saturday contained a high number of invalid email addresses. Many of them contained typos and random characters. It will take you a couple of hours to fix. Later, you’ve fixed the addresses, updated the audience database, and the emails are going out again. 

Without those analytics proactively telling about unexpected behaviors, it would be less likely to notice the bad signs until it was too late to resolve the issue. Thanks to the analytics and anomaly detection, you can spot problems and address them instantly.

Demonstrate ROI with anomalies

We often think of anomalies as bad. In fact, sometimes anomalies are a good thing.

You have been running an online promotion for a while, which has been generally successful. Related sales have been good but have started to trend down in the past few months. You’re worried that the promotion might be getting stale, so you try investing in a major on-site event for the brand at its flagship store.

A couple of weeks before the event, you begin a program that targets customers that qualify as loyalty members but have not signed up yet. A big jump in customer interest will justify the cost. Emails to this audience promoting the event have been flowing steadily.

The event goes off on Saturday without a hitch. Multiple potential loyalty club members attended and are now entered in the audience database. Your program is configured to send a follow-up email to everyone who registered.

On Monday, you receive a system notification that an anomaly has been detected in the event follow-up email. You click the notification to go to the email’s Metrics tab directly. The Clicks (unique) metric grabs your attention. There is a big spike in the number of clicks in the email. You click Clicks (unique) to dig a little deeper into the results.

In the metric detail, the anomaly is even more striking. Last night’s email clearly exceeded the bounds expected by the program analytics. It turns out that nearly everyone who registered at the event also responded to the follow-up email.

The anomaly detected in your email will help you demonstrate a positive ROI by clearly showing that the money spent on the event directly resulted in a massive increase in customer interest and engagement.

Uncover contact fatigue

If you are running a program that's working well, it’s natural to want to tweak it to build on the success. Maybe you can change the greeting, try a new subject line, configure the program to send email more frequently, or add a new email step.

How much is too much? The analytics and anomaly detection gives you a way to quickly and easily see the results of your changes. The Unsubscribes metric can be a particularly effective indicator.

Let’s say that you run a program that sends monthly emails to your loyalty club customers. Response rates have been good, but they don’t quite measure up to what your company is looking for. You try a bolder subject line to boost the open rate. Within a couple of days, unique clicks go up a bit, so it appears that your customers are paying attention.

Next, you try switching from monthly emails to weekly emails, expecting that more is better. However, soon after you make the change, you get a system notification that you have multiple anomalies in your program. You click the notification and are taken to the Metrics tab in the Program insights panel.

It looks like the last couple of emails have shown anomalies. You click the email anomaly description in the sidebar to view more details.

Looking at the Unsubscribe metric details shows the effect of the change in email frequency. Lots of your loyalty customers are suddenly opting out. It inhibits the success of future campaigns and potentially damaging your sender reputation.

You decide it would be a good idea to go back to contacting your loyalty customers less frequently. Fortunately, you discovered this problem in time. Previously, you might have relied upon weekly or month-end reports. Now, you can depend on anomaly detection to provide more timely alerts and proactively respond to issues on the go.