KAHIBARO
Discord Login Register ☰
☰

20.4. Request Logging

Table of Contents

Why Request Logging Matters

When a real user calls your API, many things can go wrong. A request might be slow, might fail with a 500 error, or might be abused by a bot. Without request logs, you only know that “something is wrong.” With good request logging, you know who called what, when, how, and what happened.

Request logging focuses on recording information about each incoming HTTP request and its corresponding response. It is one of the main tools for:

Key rule: Every production backend should log every HTTP request with at least:

  • Time
  • Method
  • Path
  • Status code
  • Duration
  • Some kind of correlation or request ID

What To Log for Each Request

You do not need to log everything, but you should log enough to reconstruct what happened.

Minimal Request Log Fields

A very typical minimal log entry for a single HTTP request:

text 
2026-08-28T10:15:30Z INFO request_id=5f3c9d
method=GET path=/api/users/42
status=200 duration_ms=23 ip=203.0.113.5 user_id=42
user_agent="Mozilla/5.0 ..."

Useful fields:

FieldWhy it matters
TimestampWhen did the request happen.
HTTP methodShows intent (GET, POST, etc).
Path / routeWhich endpoint was hit.
Status codeSuccess, client error, server error.
DurationPerformance measurement and bottleneck detection.
Request IDCorrelate logs from the same request across services.
Client IPAbuse detection, rate limiting analysis, geo info.
User IDSee which user triggered the action.
User-AgentBrowser, app, or client type.

In a JSON log, the same record might look like:

json 
{
  "ts": "2026-08-28T10:15:30Z",
  "level": "info",
  "event": "http_request",
  "request_id": "5f3c9d",
  "method": "GET",
  "path": "/api/users/42",
  "status": 200,
  "duration_ms": 23,
  "ip": "203.0.113.5",
  "user_id": 42,
  "user_agent": "Mozilla/5.0"
}

JSON makes it much easier for log tools to filter and aggregate.

Additional Optional Fields

Depending on your needs, you might also log:

You should be careful with anything that could contain sensitive data. This is covered further below.

Correlation IDs and Traceability

When an HTTP request enters your system, it might flow through:

  1. Load balancer.
  2. API gateway or reverse proxy.
  3. Application server.
  4. Multiple internal services.
  5. Database queries and background jobs.

If each component logs separately, you need a way to connect the logs that belong to the same original request. That is where correlation IDs come in.

Request IDs

A request ID is a unique identifier associated with one HTTP request.

Workflow:

  1. Client sends a request.
  2. Your system checks if the request has a header like X-Request-ID.
    • If yes, reuse it.
    • If no, generate a new random ID, for example b5e4f1f2-38fb-4f7d-9c5f-9b1e1e3a9d7a.
  3. Attach this ID to:
    • Your log context.
    • The response header, for example X-Request-ID: b5e4f1....

Example log line:

text 
ts=... level=info request_id=b5e4f1... method=POST path=/orders status=201 ...

When a user reports “My POST /orders at 12:05 failed,” you can ask for the response header X-Request-ID and search for that ID in your logs.

Important rule: Always include a request ID or correlation ID in every request and add it to every log line produced while that request is handled.

Distributed Tracing Headers

In more complex systems you might use standardized tracing headers, such as:

These allow external tracing systems to draw a timeline of a request across services. For this chapter, you only need to understand that they are more advanced versions of correlation IDs that can represent both entire traces and single spans.

Structured Request Logs vs Plain Text

You can log requests as plain human-readable text or as structured data like JSON.

Plain Text Example

text 
2026-08-28 10:15:30 INFO GET /api/users/42 200 23ms ip=203.0.113.5 user_id=42

Pros:

Cons:

Structured JSON Example

json 
{
  "ts": "2026-08-28T10:15:30Z",
  "level": "info",
  "event": "http_request",
  "method": "GET",
  "path": "/api/users/42",
  "status": 200,
  "duration_ms": 23,
  "ip": "203.0.113.5",
  "user_id": 42
}

Pros:

Cons:

For production backends, structure is extremely helpful. Many teams use more human-friendly logs in local development, but JSON in production.

Avoiding Sensitive Data in Request Logs

Request logs must be useful, but they must also be safe. It is easy to accidentally log:

If you log full request bodies or full headers blindly, you will almost certainly leak something sensitive.

Common Dangerous Patterns

Some examples of what not to log:

text 
POST /login body={"email":"user@example.com","password":"mysecret"}
Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9...

Better:

text 
POST /login body={"email":"user@example.com","password":"***"}
Authorization: Bearer ***redacted***

Or even safer for high-risk operations:

text 
POST /login body=***redacted***
headers=***sensitive***

Never log:

  • Plaintext passwords.
  • Raw authentication tokens.
  • Complete credit card numbers or CVV.

Instead, you can log:

Whitelisting vs Blacklisting

To control what is logged, two common strategies are:

Whitelisting is safer but less flexible. Many production systems use a combination: structured whitelisted request logs and separate, more detailed logs for debugging, with extra safeguards.

Performance and Log Volume

Logging every request adds some cost. Each log line:

For a high traffic API, this can be millions of requests per day. You need strategies to control volume without losing important information.

Summarizing vs Sampling

Two common approaches:

Summarized Metrics

You might use metrics (covered in another chapter) to count:

This is not a replacement for request logs, but it reduces the need to inspect every single log.

Sampling

Instead of logging every request, you could log:

Example logic:

python 
if status >= 500:
    log_request()
elif duration_ms > 1000:
    log_request()
elif random.random() < 0.01:
    log_request()

This keeps enough data to understand errors and performance issues while keeping volume manageable.

Rule of thumb: Always log all 5xx errors and very slow requests, even if you sample normal traffic.

Log Levels for Requests

You can use different log levels for different outcomes:

OutcomeTypical log level
Normal successful requestsINFO or DEBUG
Client errors (4xx)INFO or WARN
Server errors (5xx)ERROR
Security-related anomaliesWARN or ERROR

For example:

text 
INFO  request_id=... method=GET path=/items status=200 duration_ms=18
WARN  request_id=... method=POST path=/login status=401 reason=invalid_credentials
ERROR request_id=... method=GET path=/orders status=500 error="DBTimeout"

This allows you to highlight serious problems more clearly and filter by level.

Implementing Request Logging in a Typical Web Stack

Exact implementation details depend on the framework and language, but the pattern is similar.

Request Logging with Middleware

Common strategy:

  1. Create middleware that runs before and after each request.
  2. Before the request:
    • Note the start time.
    • Generate or extract the request ID.
    • Add it to the logging context.
  3. After the response:
    • Compute the duration.
    • Gather path, method, status code, user ID, IP.
    • Log one structured entry.
    • Add the request ID to the response headers.

Simplified pseudocode:

python 
async def request_logging_middleware(request, call_next):
    start = now()
    request_id = get_or_create_request_id(request)
    set_log_context(request_id=request_id)
    try:
        response = await call_next(request)
    except Exception as exc:
        response = make_500_response()
        log_error("unhandled_exception", exc=exc)
    duration_ms = elapsed_ms(start)
    log_info(
        "http_request",
        request_id=request_id,
        method=request.method,
        path=request.url.path,
        status=response.status_code,
        duration_ms=duration_ms,
        ip=get_client_ip(request),
        user_id=get_user_id_or_none(request),
    )
    response.headers["X-Request-ID"] = request_id
    return response

You do not need to know this middleware syntax in detail yet. The idea is that there is central code that logs every request in a consistent way.

Multi Layer Logging

Request logging can happen at different layers:

You can use all three:

It is common to send all of them to a central logging system where you can search and correlate by timestamp and request ID.

Using Request Logs in Practice

Once you have good request logs, you can use them for many tasks.

Debugging a Single Problem

Example: A user says “My request to /orders/123 just failed.”

Steps:

  1. Ask for the approximate time and their request ID from the X-Request-ID response header.
  2. Search your logs for request_id=<that id>.
  3. Inspect the request log entry:
    • Path and method: confirm they hit the right endpoint.
    • Status code: see if it was 4xx or 5xx.
    • Duration: see if it was a timeout or very slow.
  4. Filter surrounding logs with the same request ID to see internal error logs from that request.

This is far more effective than guessing based on timestamps alone.

Finding Performance Problems

Example: You run a query against your logs:

You might discover:

text 
/api/orders   avg=250ms  p95=700ms
/api/reports  avg=1500ms p95=4000ms

Clearly /api/reports is slow and you can focus your optimization efforts there.

Monitoring Error Rates

You can compute:

If error rates spike, your alerting system can notify you. Request logs are the raw data that feeds such alerts.

Simple formula:
$$\text{error\_rate\_percent} = \frac{\text{5xx\_count}}{\text{total\_count}} \times 100$$
Keep this value low in production. A sudden increase is a red flag.

Designing a Request Logging Strategy

To integrate request logging into your backend, answer these questions:

  1. What fields will I log for each request?
    Define a minimal, consistent set, for example: timestamp, method, path, status, duration, request ID, IP, user ID.
  2. How will I avoid sensitive data?
    Decide whether to log query strings, request headers, or bodies, and how to redact or omit sensitive fields.
  3. Will I use structured logs?
    For production, prefer JSON or another structured format.
  4. Where will the logs go?
    File, stdout (for Docker), or a log collector. This is usually combined with a centralized logging solution.
  5. How will I control volume?
    Possibly sample successful requests, always log errors and slow requests, and set appropriate log levels.
  6. How will I correlate logs across services?
    Introduce a request ID header and make sure every service passes it along and logs it.

With clear answers to these, your request logging will be a powerful tool instead of random text prints in your code.

Views: 15

Comments

Please login to add a comment.

Don't have an account? Register now!

© 2025 Kahibaro
Visitors: 3040 / 24h
Impressum / Imprint
Datenschutz / Data Policy