Photo by Markus Winkler on Unsplash

Photo by Markus Winkler on Unsplash

A new approach to more attractive histograms in Prometheus

Off the Chart

Article from ADMIN 75/2023
Histograms are a proven means of displaying latencies in Prometheus, but until now, they have had various restrictions. Native histograms now provide a remedy.

Prometheus histograms provide a method for displaying the distribution of continuous values. They provide information about the range and shape of the data and are often used to calculate percentiles. To do this, data is divided into 100 distribution areas. The x th percentile is then the value below which x percent of the observations fall.

The classical histogram metric divides a range of values into small sections ("buckets" in Prometheus) and counts the number of observations per area. In classical histograms, you first need to define these areas. Each range is represented by its upper limit. A range of 5s contains the number of all observations with a value less than or equal to five seconds. Besides the ranges, two other values are interesting: the sum total of all observations and the number of observations.

To illustrate, I'll look at a practical example wherein a histogram is defined with three buckets: 1s , 2.5s , and 5s . Two queries, one of which lasts two seconds and the other four seconds, are then observed. The first observation lies within the 2.5s , 5s , and +Inf ranges, and the second lies within the 5s and +Inf ranges (Figure 1). From this data, a histogram of HTTP request duration in seconds can be created in Prometheus (Listing 1).

Listing 1

HTTP Request Duration

# HELP http_request_duration_seconds Histogram of latencies for HTTP requests in seconds.
# TYPE http_request_duration_seconds histogram
http_request_duration_seconds_bucket{le="1"} 0
http_request_duration_seconds_bucket{le="2.5"} 1
Use Express-Checkout link below to read the full article (PDF).

Buy this article as PDF

Express-Checkout as PDF
Price $2.95
(incl. VAT)

Buy ADMIN Magazine

Get it on Google Play

US / Canada

Get it on Google Play

UK / Australia

Related content

  • Time-series-based monitoring with Prometheus
    As Prometheus gave fire to mankind, the distributed monitoring software with the same name illuminates the admin's mind in native cloud environments, offering metrics for monitored systems and applications.
  • Monitoring, alerting, and trending with the TICK Stack
    If you are looking for a monitoring, alerting, and trending solution for large landscapes, you will find all the components you need in the TICK Stack.
  • Detect anomalies in metrics data
    Anomalies in an environment's metrics data are an important indicator of an attack. The Prometheus time series database automatically detects, alerts, and forecasts anomalous behavior with the Fourier and Prophet models of the Prometheus Anomaly Detector.
  • Getting started with Prometheus
    Prometheus is a centralized time series database with metrics, scraping, and alerting logic built in. We help you get started monitoring with Prometheus.
  • I/O Profiling at the Block Level

    Understanding how applications perform I/O is important not only because of the volume of data being written and read, but because the performance of some applications is dependent on how I/O is conducted. In this article we profile I/O at the block layer to help you make the best storage decisions.

comments powered by Disqus
Subscribe to our ADMIN Newsletters
Subscribe to our Linux Newsletters
Find Linux and Open Source Jobs

Support Our Work

ADMIN content is made possible with support from readers like you. Please consider contributing when you've found an article to be beneficial.