Leave us your email address and be the first to receive a notification when Team posts a new blog.
AWS has a Cost Management service that includes reports, budgets and recommendations, to which the company recently announced the preview of Cost Anomaly Detection (CAD). To limit surprises on the monthly bill, it can help to keep a close eye out for cost anomalies: changes in the historical consumption pattern of your cloud resources.
But then again, who doesn’t love a surprise?
How did it work in the past?
When working for my former employer this was something, we had to create ourselves, to keep track of unforeseen costs within the projects we ran for our customers. It’s always fun to see your Cloud Service Provider create your solutions for you. So I am keen to see what AWS came up with. In this blog I’ll walk you through the preview offering from AWS, show you how to implement this Cost Anomaly detection and talk a bit about how AWS can improve this service.
Starting September 25th, 2020, you can receive Cost Anomaly Detection alert notifications on your AWS spend. This comes with root cause analysis, so you can proactively take actions and minimize unintentional spend. This sounds a bit strange to me since AWS collects your money for services they are running, and how on earth will they know what defines an unintentional spend from my point of view?
What is Cost Anomaly Detection?
Cost Anomaly detection is a feature that uses machine learning to continuously monitor your cost and usage to detect unusual spends. Using Cost Anomaly Detection should include the following benefits:
- Receive alerts individually in aggregated reports. You can receive alerts in an email or via an Amazon SNS topic.
- Evaluate your spend patterns using machine learning methods to minimize false positive alerts. For example, you can evaluate weekly or monthly seasonality and organic growth.
- Analyze and determine the root cause of the anomaly, such as account, service, region or usage type that is driving the cost increase.
- Configure how you need to evaluate your costs. You can choose whether you want to analyze all of your AWS services independently, or by member accounts, cost allocation tags or cost categories.
While I’ll walk through the web interface of the service, there is some good documentation on how this works from a CLI perspective. To use the service, and since it’s a preview, you need to opt-in.
Once you have skipped the welcoming tour (because, who needs it anyway?), you will see a familiar dashboard. It shows your current spend on AWS, for me a whopping $1.80, and the increase over the last month.
Creating a monitor
There are 4 types of monitors you can create during this preview:
- AWS Services
- Linked Account
- Cost Category
- Cost Allocation Tag
Most standard users will use the monitor AWS Services, as it has all available AWS services in it, which are used in the account you deploy this monitor to. The monitor Linked Account can be interesting if you are, for example, a reseller and you want to create a CAD per customer. You can add up to 10 accounts (hard limit), so a standard AWS Landing Zone for a customer could be onboarded into this solution. Cost Category and Cost Allocation Tags are useful for those who use them.
Next you need to send an alert threshold, which is the detection bandwith, or, based on the actual spend, the 'calculated upperbound spend'. Because an anomaly is any derivation of upperbound or lowerbound values, and both are non-static, the calculated metric can differ over time. This threshold will cause an alert of a possible overspend.
In our case we will set it to $0. Whenever the anomaly will be surpassed by $0 it will alert.
After you create your monitors, anomaly detection evaluates your future spend. Based on your defined alerting preferences, you might start receiving alerts within 24 hours.
So, let’s spend some money, and see what happens...
This morning, I have received an e-mail, that there was an anomaly detected.
Within the console I could see the anomaly. Apparently I have used too much of S3 storage on my account.
When we go into the details, it shows nothing fancy.
Is it any good?
I was expecting a proper Root Cause Analysis; which files, timestamps added, which role did it, a.k.a. a proper who dun it story. Apparently, we are not there yet. I’ll let it run for some more days and update this blog if this changes over time.
People may wonder whether it is really in the interest of AWS to provide a service that helps customers spend less money. It is true that, like every company, the cloud providers are always trying to persuade their users to adopt new services or add premium features. I like the fact that it gives insight and a deep dive into where the costs are coming from. Though Cost Anomaly Detection is not the complete answer to overpaying. After all, if an organization paid more than it needed to last month, it is not an anomaly if it does so again.
As of now, I’m missing the details of an anomaly, like it was in the past with Amazon Detective. Maybe it will come over time, but it is simply not there yet. Therefore this service is no good at this moment.
AWS Cost Anomaly Detection is part of the AWS Cost Management suite and is currently in preview at no cost.
Questions, Remarks & Comments
If you have any questions and need more clarification, we are more than happy to dig deeper. Any comments are also appreciated. You can either post it online or send it directly to the author, it’s your choice.