KAHIBARO
Discord Login Register ☰
☰

20.10. Grafana

Table of Contents

Why Grafana Matters for Backend Developers

Grafana is a visualization and dashboard tool. It takes metrics, logs, or traces from different data sources and shows them in graphs, tables, and alerts.

In a backend system, Grafana usually sits on top of tools like Prometheus, Loki, Elasticsearch, or other databases. You use it to see how your application behaves in real time, and to investigate problems.

Typical uses:

Grafana does not collect metrics itself. It connects to existing data sources and visualizes their data.

Key idea: Grafana is a read‑only visualization and alerting layer on top of other monitoring and logging systems. It does not replace them, it uses them.

Grafana Architecture Basics

From a backend perspective, you can think of Grafana as a web app with:

Typical components:

ComponentRole
Grafana serverMain process, serves UI, handles API, alerts
DatabaseStores dashboards, users, data source configs
Data sourcesStore metrics/logs (Prometheus, Loki, PostgreSQL)
Notification chan.Email, Slack, etc for alerts

Common data sources in backend projects:

Installing Grafana (Developer View)

The exact install method will depend on your OS and environment. As a backend developer, you mainly need to know:

Running Grafana with Docker

For local experiments, Docker is usually easiest.

docker-compose.yml example:

yaml 
version: "3.8"
services:
  grafana:
    image: grafana/grafana:latest
    container_name: grafana
    ports:
      - "3000:3000"
    environment:
      - GF_SECURITY_ADMIN_USER=admin
      - GF_SECURITY_ADMIN_PASSWORD=admin
    volumes:
      - grafana-storage:/var/lib/grafana
volumes:
  grafana-storage:

Then:

bash 
docker compose up -d

Open http://localhost:3000 and log in with admin / admin (or your env values).

Basic Configuration Files

Grafana configuration is usually in grafana.ini or environment variables starting with GF_.

Examples:

In production, these go into your deployment system (Docker, Kubernetes, etc.).

Connecting Data Sources

Grafana can connect to many backends, but as a backend engineer, you will most often integrate:

Adding a Prometheus Data Source

Assume Prometheus is running at http://prometheus:9090 inside your Docker network.

In Grafana UI:

  1. Go to Configuration → Data sources
  2. Click Add data source
  3. Select Prometheus
  4. Set URL to http://prometheus:9090
  5. Click Save & test

If you prefer an example with plain Docker:

yaml 
services:
  prometheus:
    image: prom/prometheus:latest
    ports:
      - "9090:9090"
    volumes:
      - ./prometheus.yml:/etc/prometheus/prometheus.yml
  grafana:
    image: grafana/grafana:latest
    ports:
      - "3000:3000"
    depends_on:
      - prometheus

Then use http://prometheus:9090 as the URL from Grafana’s point of view.

Adding a PostgreSQL Data Source

You can query your app database directly, for example to show:

Steps:

  1. Configuration → Data sources → Add data source
  2. Select PostgreSQL
  3. Set host, database, user, password
  4. Choose correct SSL and time zone options
  5. Save & test

Example connection settings:

FieldValue
Hostpostgres:5432
Databasemy_app_db
Usergrafana_reader
TLS/SSLAs required by your setup

In PostgreSQL, you usually want a read‑only user:

sql 
CREATE USER grafana_reader WITH PASSWORD 'strongpassword';
GRANT CONNECT ON DATABASE my_app_db TO grafana_reader;
GRANT USAGE ON SCHEMA public TO grafana_reader;
GRANT SELECT ON ALL TABLES IN SCHEMA public TO grafana_reader;

Building Dashboards

Grafana dashboards are collections of panels. Each panel shows data from one or more queries.

Basic Panel Types

Common panel types useful for backends:

Panel typeGood for
Time seriesLatency, throughput, CPU usage over time
GaugeSingle value like error rate or usage
Bar chartComparing values (top endpoints by errors)
TableLog summaries, queries, raw metrics
StatSingle KPI (current users, RPS)

Example: API Latency Dashboard with Prometheus

Assume you have a Prometheus metric:

text 
http_request_duration_seconds_bucket{path="/api/tasks", method="GET", le="..."}

You can create panels like:

PromQL:

promql 
  histogram_quantile(
    0.95,
    sum(rate(http_request_duration_seconds_bucket[5m])) by (le)
  ) * 1000
promql 
  sum(rate(http_requests_total[5m]))
promql 
  (
    sum(rate(http_requests_total{status=~"5.."}[5m]))
  /
    sum(rate(http_requests_total[5m]))
  ) * 100

Key metrics to visualize for a backend:

  • Latency: P50, P95, P99 response times
  • Traffic: Requests per second, concurrent users
  • Errors: 4xx and 5xx error rates
  • Saturation: CPU, memory, DB connections, queue sizes

Example: Business Metrics with PostgreSQL

Suppose you have a table:

sql 
CREATE TABLE orders (
  id SERIAL PRIMARY KEY,
  created_at TIMESTAMP NOT NULL,
  total_amount NUMERIC(10, 2) NOT NULL,
  status TEXT NOT NULL
);

You can use a PostgreSQL panel query:

sql 
SELECT
  date_trunc('day', created_at) AS time,
  SUM(total_amount) AS total_revenue
FROM orders
WHERE $__timeFilter(created_at)
GROUP BY time
ORDER BY time;

Here __$timeFilter(created_at) is a Grafana macro that applies the current dashboard time range.

This will show a time series of daily revenue.

Variables and Reusable Dashboards

Grafana variables let you change what a dashboard shows without editing the panels.

You can create variables for:

Example: Environment Variable

  1. Dashboard settings → Variables → New
  2. Name: env
  3. Type: Custom
  4. Values: dev, staging, prod

Then in PromQL use:

promql 
sum(rate(http_requests_total{env="$env"}[5m]))

Switching the dropdown between dev, staging, prod will instantly change all panels.

Example: Path Variable from Prometheus

Create a variable:

promql 
  label_values(http_requests_total, path)

Panels can then filter with {path="$path"}.

Alerts with Grafana

Grafana can evaluate alert rules on your metric queries and send notifications.

Alerting Concepts

Example: High Error Rate Alert

Imagine you want an alert if the error rate is above 5 percent for 10 minutes.

PromQL expression:

promql 
(
  sum(rate(http_requests_total{status=~"5.."}[5m]))
/
  sum(rate(http_requests_total[5m]))
) * 100

In Grafana alert rule:

This means: average error rate over 10 minutes is above 5 percent.

Alert rules should focus on user‑visible problems, not just technical details.
Examples:

  • "Error rate > 5 percent"
  • "P95 latency > 1 s"
  • "DB connections > 90 percent of pool"

Simple Notification Setup

  1. Go to Alerting → Contact points
  2. Create a contact point, for example:
    • Email: oncall@example.com
    • Slack: webhook URL
  3. Go to Alerting → Notification policies
  4. Set default policy to send all alerts to that contact point

Now, when a rule fires, Grafana sends notifications.

Logs and Traces with Grafana

Grafana can also show logs and traces if you use tools like Loki, Elasticsearch, or Tempo.

Logs with Loki (Overview)

Typical setup:

Log query example in Loki:

logql 
{app="api", level="error"} |= "database"

You can:

Traces with Tempo (High‑Level)

If you use distributed tracing:

This is especially helpful for microservices architectures.

Example: Metrics for a FastAPI Application

Assume you have a FastAPI app with Prometheus metrics using prometheus_client.

python 
from fastapi import FastAPI
from prometheus_client import Counter, Histogram, generate_latest
from prometheus_client import CONTENT_TYPE_LATEST
import time
from starlette.responses import Response
app = FastAPI()
REQUEST_COUNT = Counter(
    "http_requests_total",
    "Total HTTP requests",
    ["method", "path", "status"]
)
REQUEST_LATENCY = Histogram(
    "http_request_duration_seconds",
    "Request latency",
    ["method", "path"]
)
@app.middleware("http")
async def metrics_middleware(request, call_next):
    method = request.method
    path = request.url.path
    start = time.time()
    response = await call_next(request)
    latency = time.time() - start
    REQUEST_LATENCY.labels(method=method, path=path).observe(latency)
    REQUEST_COUNT.labels(
        method=method,
        path=path,
        status=response.status_code
    ).inc()
    return response
@app.get("/metrics")
def metrics():
    return Response(
        generate_latest(),
        media_type=CONTENT_TYPE_LATEST
    )

Steps to visualize in Grafana:

  1. Run Prometheus, scrape http://api:8000/metrics
  2. Connect Grafana to Prometheus
  3. Create a dashboard with:
    • P95 latency per path using http_request_duration_seconds
    • Requests per second per path using http_requests_total
    • Error rate (status >= 500)

Now you have a basic observability setup for your FastAPI backend.

Grafana in Production Environments

In production, you usually care about:

Example of provisioning a data source via YAML (used with Docker/Kubernetes):

yaml 
apiVersion: 1
datasources:
  - name: Prometheus
    type: prometheus
    url: http://prometheus:9090
    access: proxy
    isDefault: true

This file goes under /etc/grafana/provisioning/datasources/ in the container.

How Grafana Fits Your Backend Monitoring Stack

A minimal, practical stack for a backend project often looks like:

ConcernToolRole
MetricsPrometheusCollects metrics from services and systems
LogsLoki or log systemStores and indexes application logs
TracingTempo (optional)Stores distributed traces
ViewGrafanaDashboards, alerts, correlation

Your job as a backend engineer is usually to:

With this understanding, you can effectively use Grafana to monitor, debug, and improve your backend systems.

Views: 8

Comments

Please login to add a comment.

Don't have an account? Register now!

© 2025 Kahibaro
Visitors: 3496 / 24h
Impressum / Imprint
Datenschutz / Data Policy