Hands-free customer engagement is one of the beauties of Acoustic Campaign programs. However, you still need to keep an eye on things. As the saying goes, “expect the unexpected.”
What if things don’t go as you planned? What if customers don’t respond to your Program email messages the way you thought they would? What if there’s a problem with the customer data? Will you be able to react in time?
On the other hand, what if things go better than expected? What if there’s an overwhelmingly positive customer response? How can you take advantage of your good fortune? How will you know when it’s time to brag?
Fortunately, there’s help available through Anomaly Detection in Programs. An AI-powered anomaly detection model quietly watches multiple program metrics and tells you when the unexpected happens and shows you where to look. There's no need to run a report or wade through message logs. Plus, the longer your program runs, the more the model learns about what to expect and when to grab your attention. Everything fits unobtrusively into the Metrics tab in the Program insights panel.
From the initial summary view, you can dive deeper. Each metric summary links to an expanded view that reveals additional detail about the performance of that metric over time. Any unexpected program behaviors – anomalies – are highlighted in charts that plot the history of each metric. By default, the Metrics tab displays results for the previous 30 days. You can also see results for the previous 7, 90, or 450 days. The model evaluates program results nightly and displays dates and times using Greenwich Mean Time (GMT).
As a program runs, the anomaly detection model continually defines a range of expected values for each metric, based on an analysis of previous program performance and customer responses. The upper and lower expected values define the bounds of the metric. Within these bounds, the model defines a specific expected value that is most likely to occur. It appears in the detail charts as a dotted line. Trends for the bounds and expected values become apparent over time. When there is an anomaly, or a spike up or down in the data, it shows as a red dot on your graph.
The program anomalies evaluation model reviews program activity nightly. The model calculates expected results, compares them to the actual results, and highlights unusual behavior. Open the Metrics tab at any time to get the most recent view.
Enable anomaly detection notifications
You can automatically receive a notification when an anomaly is detected. Go a step further and also enable email notifications to receive anomaly notifications by email.
Note: You must enable anomaly detection emails for each program individually.
- Open a program, and click Metrics.
- Turn on Notify me about anomalies. You'll start receiving in-app notifications.
- In addition to in-app notifications, enable email notifications if you want to stay in the loop outside Campaign.
- Go to Notifications > Settings. By default, in-app notifications are already enabled.
- Turn on Email notifications. We'll use the email address associated with your profile to alert you to anomalies that occur.
Availability and stored data
Anomaly identification and notification data is hosted exclusively in a United States data center. The stored information consists of user notification preferences and aggregate history for program transmission and response activity. No personal data is ever required or stored. To display program metrics and take advantage of anomaly detection modeling, your organization must meet the following criteria.
- Hosted on a US-based POD
- Provisioned for Performance Insights
- Not provisioned for Transact
- Uses the new navigational experience
- Actively using the latest programs canvas with programs that contain email steps only
Contact Support to request access to Program Anomaly Insights for your organization.
The programs anomaly detection model watches several key metrics that are reliable measures of customer engagement.
Each metric type indicates summary results observed for all emails currently running in the program.
This metric measures the program’s outbound mailing activity. It is the sum of all email messages that were sent during the previous day by all emails currently running in the program.
The Opens (unique) metric identifies the total number of email messages associated with the program that have been opened at least once per email recipient. If an email recipient opens an email message three times, the system records a single unique open. However, if three different email recipients each open the same message once, the system records three unique opens.
This metric identifies the total number of email recipients who have clicked at least one link in any of the email messages sent by emails in the program. If a single email recipient clicks on three links in the same email, the system records one unique click. If the same individual clicks one link in three different email messages, the system records three unique clicks.
This metric refers to email messages that are undeliverable because of a permanent characteristic of the message or delivery address, such as a misspelled or non-existent email address.
This metric refers to email messages that are currently undeliverable because of a temporary characteristic of the message or delivery address, such as a full inbox or system delays.
The Unsubscribes metric indicates the total number of requests to stop receiving email messages sent by any of the emails currently running in the program.
- Identify delivery problems
- Demonstrate return on investment (ROI) with anomalies
- Uncover contact fatigue
Identify delivery problems
Monitoring email campaigns can demand your full attention. The need to stay on top of deliverability metrics can be a huge challenge, especially if you need to run (yet another) report, wait for somebody else to run a report, or maybe wait days, weeks, or even a month for an automated delivery report. By then, it might be too late fix any delivery problems that occur and the number of customers who missed your message has skyrocketed.
But what if it were easy to monitor deliverability? What if you could be told about problems as soon as they happen?
Let’s say that you manage a program that sends an email to describe this week’s promotions to all new customers and other customers who qualified for a Preferred account in the past week. Audience updates and new email content are loaded every Saturday night. The program is set to start sending emails on Sunday to have email in the right inboxes on Monday morning. This week, the update runs as scheduled.
You come in Monday morning, log into Acoustic Campaign, and you are greeted by a system notification that there’s been an anomaly in your weekly promotion program. Better check this out.
When you open the program and scan the Metrics tab it’s easy to notice a red dot on the Hard bounces chart.
This visual cue is telling you that something’s wrong. Click the metric to dive deeper.
It’s clear to see that there’s been a spike in the number of hard bounces, beyond the highest expected response identified by the green "bounds" area for this program. This alert makes it clear to you that this week’s emails aren't getting through.
After reviewing the database, you’ve identified the culprit. The audience data that was entered on Saturday night contained a high number of invalid email addresses. Many of the email addresses contained typos and random characters. It will take you a couple of hours to fix. Later, you’ve fixed the addresses, updated the audience database, and the emails are going out again. Good thing that you enabled anomaly notifications.
Imagine if you didn’t have this kind of program performance data at your fingertips. You might have missed this problem until it was too late. Imagine if those analytics didn’t proactively tell you about unexpected behaviors. Now, thanks to the analytics and anomaly detection, you can spot problems, respond quickly, and stay in touch with your customers.
Demonstrate ROI with anomalies
We often think of anomalies as bad. In fact, sometimes anomalies are a good thing.
Imagine that you are a brand manager who has been running an online promotion for a while. It’s been generally successful. Related sales have been good but have started to trend down in the past few months. You’re worried that the promotion might be getting stale, so you convince senior management to fund a major on-site event for the brand at your flagship store.
A couple of weeks before the event, you begin a program that targets customers that qualify as loyalty members but have not signed up yet. A big jump in customer interest by these customers will justify the cost. Emails to this audience promoting the event have been flowing steadily.
The event goes off on Saturday without a hitch. Nearly 2,000 potential loyalty club members attended and are now entered in the audience database. Your program is configured to send a follow up email to everyone who registered.
On Monday, you log into Campaign and you receive a system notification that an anomaly has been detected in the event follow up program. You’ve come to rely on the analytics and anomaly detection available in your programs to give you a quick and clear view of how they are performing. So, you immediately open the Metrics tab in your program to see what’s happening. The Clicks (unique) metric grabs your attention.
A red dot on the chart indicates a big spike in the number of clicks in the email that the program sent. You click Clicks (unique) to dig a little deeper into the results.
In the metric detail, the anomaly is even more striking. Last night’s email clearly exceeded the bounds expected by the program analytics. The red dot that identifies the anomaly is clearly outside of the green bounds area that defined expectations. When you click the dot, the anomaly detail reveals that nearly everyone who registered at the event also engaged by responding to the follow up email.
It’s easy to see the value in this data. The anomaly detected in your program will help you demonstrate a positive ROI by clearly showing that the money spent on the event directly resulted in a surprising increase in customer interest and engagement.
You have a meeting with the senior management team tomorrow. They’ll want to know if their investment paid off. Now, thanks to a detected anomaly, you have the numbers to prove that it did.
Uncover contact fatigue
If you are running a program that's working well, it’s natural to want to tweak it to build on the success. Maybe you can change the greeting, try a new subject line, configure the program to send email more frequently, or add a new email step.
How much is too much? The analytics and anomaly detection gives you a way to quickly and easily see the results of your changes. The Unsubscribes metric can be a particularly effective indicator.
Let’s say that you run a program that sends monthly emails to your loyalty club customers. Response rates have been good, but they don’t quite measure up to what your company is looking for. So, you try a bolder subject line. Within a couple of days, unique clicks go up a bit, so it appears that your customers are paying attention.
Next, you try switching from monthly emails to weekly emails, expecting that more is better. However, soon after you make the change, you get a system notification that an anomaly has been detected in the Unsubscribes metric for your loyalty program. You open the Metrics tab in the program to learn more.
The red dot on the trend line indicates an unexpected spike in unsubscribes. Perhaps this time, you realize, more is not better.
Looking at the Unsubscribe details quickly shows the effect of the change in email frequency. Lots of your loyalty customers are suddenly opting out of future communications. This isn't what you want to see, or something that can be allowed to drag out for long. It inhibits the success of future campaigns and you might even be damaging your sender reputation. Now that you’ve seen the effect of your recent change, you determine that it would be a good idea to go back to contacting your loyalty customers less frequently.
Fortunately, you discovered this problem in time. Previously, you might have relied upon weekly or month-end reports. Now, you can depend on anomaly detection to provide more timely alerts. You can trust the AI-supported insights to know when your program isn't performing as expected so that you can proactively respond and maintain just the right touch with your customers.