First, time series data must be aggregated correctly. Andrew used a standard counter of http_requests_total as the data source for this demonstration, although many other metrics can be applied using the same techniques. http_requests_total{
job="apiserver",
method="GET",
controller="ProjectsController",
status_code="200",
environment="prod"
} This example metric has some extra dimensions: method, controller, status_code, environment, as well as the dimensions that Prometheus adds, such as instance and job. Next, you must choose the correct level of aggregation for the data you are using.
Calculating seasonality with Prometheus required that we iterate on a few different statistical principles. In the first iteration, we calculate by adding the growth trend we’ve seen over a one-week period to the value from the previous week. Calculate the growth trend by subtracting the rolling one-week average for last week from the rolling one-week average for now. - record: job:http_requests:rate5m_prediction
expr: >
job:http_requests:rate5m offset 1w # Value from last period
+ job:http_requests:rate5m:avg_over_time_1w # One-week growth trend
- job:http_requests:rate5m:avg_over_time_1w