KAHIBARO
Discord Login Register ☰
☰

20.6. Metrics

Table of Contents

Why Metrics Matter

Metrics are numeric measurements that describe how your backend behaves over time. They tell you:

Logs tell you what happened, for specific events. Metrics tell you how often and how much, in an aggregated way that you can chart and alert on.

Key idea: Metrics are numbers over time that you can aggregate, visualize, and alert on. They are essential for understanding performance, reliability, and capacity.

You will later connect these ideas to tools like Prometheus and Grafana, but here we focus on understanding what to measure and how to think about metrics.

Types of Metrics

There are three big families you should know: infrastructure metrics, application metrics, and business metrics.

Infrastructure Metrics

These come from servers, containers, and databases.

Examples:

AreaExample MetricDescription
CPUcpu_usage_percentHow busy the CPU is
Memorymemory_used_bytesRAM currently used
Diskdisk_used_percentUsed disk as a percentage of total
Networknetwork_bytes_in_totalBytes received over the network
OS Processesprocess_countNumber of processes/threads
Databasedb_active_connectionsOpen DB connections from your app

You usually get these from the OS, container runtime, cloud provider, or database.

Application Metrics

These come from your backend code. You add these manually through your application.

Typical examples:

CategoryExample Metric
Requestshttp_requests_total
Errorshttp_requests_errors_total
Latencyhttp_request_duration_seconds
Background jobsjobs_processed_total
Cachingcache_hits_total, cache_miss_total
External callsexternal_api_duration_seconds

You will often expose these through an endpoint like /metrics for tools to scrape.

Business Metrics

These describe what users and the business are doing, not just the technical system.

Examples for an e‑commerce backend:

MetricMeaning
orders_created_totalNumber of orders created
payments_failed_totalNumber of payment failures
active_usersCurrently logged-in users
cart_abandon_ratePercentage of carts never checked out

Business metrics help you connect technical issues to business impact.

Core Backend Metrics: RED and USE

Two widely used frameworks help you remember the important metrics.

RED: Requests, Errors, Duration

The RED method is for services that handle requests (APIs, web backends).

Examples:

You usually break these down by:

USE: Utilization, Saturation, Errors

The USE method is for resources like CPU, memory, disk, and databases.

Examples:

ResourceUtilizationSaturationErrors
CPUcpu_usage_percentrun_queue_lengthcpu_errors_total (rare)
Diskdisk_usage_percentdisk_io_queue_lengthdisk_errors_total
DBdb_active_connections vs maxdb_query_queue_lengthdb_connection_errors_total

Use RED to see if your service is “slow or failing”. Use USE to see “which resource is the bottleneck”.

Metric Types: Counter, Gauge, Histogram, Summary

Monitoring tools categorize metrics into a few types. Understanding them helps you choose the right one.

Counters

A counter is a number that only goes up (or resets to zero).

Examples:

Use counters for “how many times something happened”.

You often compute rates from counters, for example:

If you have a counter http_requests_total, the rate over time window $\Delta t$ is:

$$\text{requests\_per\_second} = \frac{\Delta\text{http\_requests\_total}}{\Delta t}$$

Rule: Use counters for values that only increase (requests, errors, jobs). Calculate rates from counters instead of incrementing “per second” values yourself.

Gauges

A gauge goes up and down.

Examples:

Use gauges for “current state right now”.

Histograms

A histogram samples observations into buckets. It is used to measure distributions like latencies.

Example: http_request_duration_seconds with buckets

The system counts how many requests fall into each bucket. From this you can compute:

Summaries

A summary is similar to a histogram but often precomputes quantiles directly on the application side. In practice:

For this course, focus on histograms and counters.

Useful Metrics for a Typical Backend

Here is a practical set of metrics to aim for when you build a backend API.

HTTP Request Metrics

You can define these for every endpoint:

Metric nameTypeDescription
http_requests_totalCounterTotal number of HTTP requests
http_requests_errors_totalCounterTotal 4xx and 5xx responses
http_request_duration_secondsHistogramRequest handling time
http_in_progress_requestsGaugeRequests currently being processed
request_body_size_bytesHistogramSize of incoming request bodies
response_body_size_bytesHistogramSize of responses

You should label these by:

Example label set:

Database Metrics

Important database metrics:

MetricTypeMeaning
db_active_connectionsGaugeCurrent connections from your app
db_connection_errors_totalCounterFailed connection attempts
db_query_duration_secondsHistogramTime each query takes
db_deadlocks_totalCounterNumber of detected deadlocks
db_rows_read_totalCounterRows read by queries
db_rows_written_totalCounterRows inserted or updated

Later, in the PostgreSQL and performance sections, you will see how to collect DB metrics in more detail.

Background Job Metrics

For background workers:

MetricTypeMeaning
jobs_enqueued_totalCounterJobs added to the queue
jobs_started_totalCounterJobs started
jobs_completed_totalCounterJobs finished successfully
jobs_failed_totalCounterJobs that ended with an error
job_duration_secondsHistogramTime taken per job
job_queue_lengthGaugeNumber of pending jobs

Cache Metrics

For Redis or in‑memory caches:

MetricTypeMeaning
cache_hits_totalCounterNumber of cache hits
cache_misses_totalCounterNumber of cache misses
cache_evictions_totalCounterItems removed due to memory limits
cache_sizeGaugeItems currently stored

You can compute the cache hit ratio:

$$
\text{hit\_ratio} = \frac{\text{cache\_hits\_total}}{\text{cache\_hits\_total} + \text{cache\_misses\_total}}
$$

Important: A higher cache hit ratio usually means better performance and less load on your database.

Latency Metrics and Percentiles

Average latency is often misleading. Averages hide “tail latency” where a small percentage of requests are very slow.

For example:

The average is:

$$
\text{avg} = \frac{90 \cdot 0.05 + 10 \cdot 5}{100} = 0.545 \text{ seconds}
$$

But:

So you need percentiles:

Typical SLOs use p95 or p99 latency.

Example SLO:

When you define histograms for latencies, you can compute these percentiles with your monitoring tools.

From Metrics to Alerts and SLOs

Metrics are not useful if nobody looks at them. You use them to define:

Simple Alert Examples

Using the RED and USE metrics:

You will learn more about alerting in monitoring and production chapters, but keep these patterns in mind as you design metrics.

How to Instrument a Backend (Conceptually)

The exact code depends on your language and monitoring stack, but the steps are similar.

1. Choose a Metrics Library

For Python backends, popular choices include:

2. Define Metrics

You typically define metrics as globals near your app startup:

python 
from prometheus_client import Counter, Histogram
REQUEST_COUNT = Counter(
    "http_requests_total",
    "Total HTTP requests",
    ["method", "path", "status_code"],
)
REQUEST_LATENCY = Histogram(
    "http_request_duration_seconds",
    "HTTP request latency",
    ["method", "path", "status_code"],
    buckets=[0.01, 0.05, 0.1, 0.3, 1.0, 3.0]
)

Note: Do not overdo labels. Stick to a small set like method, path, status_code.

3. Instrument Requests (e.g., via Middleware)

Use middleware to measure every request:

python 
import time
from starlette.middleware.base import BaseHTTPMiddleware
class MetricsMiddleware(BaseHTTPMiddleware):
    async def dispatch(self, request, call_next):
        start = time.perf_counter()
        response = await call_next(request)
        latency = time.perf_counter() - start
        method = request.method
        path = request.url.path
        status_code = str(response.status_code)
        REQUEST_COUNT.labels(method=method, path=path, status_code=status_code).inc()
        REQUEST_LATENCY.labels(method=method, path=path, status_code=status_code).observe(latency)
        return response

Later, you expose a /metrics endpoint that returns all metrics in a format that your monitoring system understands.

You will go deeper into specific tools in the Prometheus and Grafana sections, but this pattern is common across stacks.

Good Practices for Metrics Design

Start Simple

Begin with a small set:

You can add more when you see a clear need.

Be Careful With Labels

Each unique set of labels creates a new time series. Too many labels can cause high memory usage.

Avoid:

Prefer:

Use Consistent Naming

You do not have to copy these exactly, but consistency helps:

Rule: A metric name should be clear, consistent, and unit‑aware. Include the unit in the name, such as _seconds or _bytes.

Align Metrics With Requirements

Think about questions you want to answer:

Choose metrics that let you answer those questions quickly, without digging through logs.

Summary

In this chapter you learned:

Later chapters on Prometheus, Grafana, and observability will show how to collect, store, visualize, and alert on these metrics in a production backend.

Views: 5

Comments

Please login to add a comment.

Don't have an account? Register now!

© 2025 Kahibaro
Visitors: 3044 / 24h
Impressum / Imprint
Datenschutz / Data Policy