KAHIBARO
Discord Login Register ☰
☰

20.5. Error Logging

Table of Contents

Why Error Logging Matters

When something goes wrong in your backend, users usually see only a generic error message. You see nothing, unless you log it.

Error logging is the process of recording information about problems that happen in your application. Without it, you are effectively blind in production. With it, you can:

Important: A backend in production must always have structured, centralized error logging. Relying on print statements or local log files is not enough.

This chapter focuses on logging errors specifically, not general application logging or metrics, which are covered elsewhere.

What Counts as an Error?

Not everything that looks bad is an error, and not every error is equally important. It helps to classify problems so you can react correctly.

Types of errors

Some common categories:

CategoryExampleShould you log?
Programming bugsZeroDivisionError, KeyError, AttributeErrorAlways
Data validationUser sends invalid email or missing required fieldUsually, at lower level
External failuresDatabase timeout, Redis connection error, email serverAlways
Client misuse404 on random path, invalid API keyOften, but not as errors
Security-relatedMany failed logins, suspicious inputs, SQLi attemptsYes, and monitor closely

Many frameworks attach a log level to each log entry:

For error logging, you mostly work with ERROR and CRITICAL.

Rule: Use ERROR when a request or job fails but the system still runs. Use CRITICAL only when the whole application or a key component is unusable.

Logging Exceptions Correctly

Errors in backend code often appear as exceptions. It is not enough to log "Error happened". You need:

Example: naive vs proper exception logging

Imagine a route that fails:

python 
def divide(a, b):
    return a / b
try:
    result = divide(1, 0)
except Exception:
    print("Something went wrong")

This tells you nothing. A better version:

python 
import logging
logger = logging.getLogger(__name__)
try:
    result = divide(1, 0)
except Exception:
    logger.exception("Error while dividing numbers")

logger.exception:

Equivalent pattern if you already have the exception object:

python 
except Exception as exc:
    logger.error("Error while dividing numbers", exc_info=exc)

In a web framework, you often do this in middleware or error handlers instead of every route.

Example: logging errors in a request handler

Pseudo code for a generic HTTP handler:

python 
def handle_request(request):
    try:
        return process_request(request)
    except Exception:
        logger.exception("Unhandled error in request handler")
        return make_500_response()

This pattern:

  1. Logs all unhandled exceptions with full details.
  2. Returns a safe HTTP 500 response without leaking internals.

Including Context in Error Logs

A stack trace is helpful, but context turns a log into a real debugging tool.

What context should you include?

For a typical web request, useful fields are:

Example of a structured context dictionary:

python 
context = {
    "path": request.path,
    "method": request.method,
    "query": dict(request.query_params),
    "user_id": request.user.id if request.user else None,
    "request_id": request.headers.get("X-Request-ID"),
}

Then:

python 
logger.exception("Unhandled error in HTTP request", extra=context)

If you use JSON logging, these context keys become fields in the JSON object.

Example: missing context problem

Log without context:

text 
ERROR:root:Database error
Traceback (most recent call last):
  ...
psycopg2.OperationalError: timeout

You know the database failed, but not which request. With context:

json 
{
  "level": "ERROR",
  "msg": "Database error",
  "exception": "psycopg2.OperationalError: timeout",
  "path": "/orders/checkout",
  "method": "POST",
  "user_id": 42,
  "request_id": "8a1f6b8c-...-e2f1"
}

Now you can:

Separating Client Errors from Server Errors

Not every failed request is a server error. This matters for both HTTP status codes and logging.

Mapping HTTP codes to log levels

A common pattern:

HTTP statusMeaningLog level suggestion
2xxSuccessUsually no error log
3xxRedirectINFO at most
4xx (client error)Client input or behaviorINFO or WARNING
5xx (server error)Server failedERROR or CRITICAL

For example:

Example: logging 4xx vs 5xx

Imagine a validation error:

python 
def create_user(request):
    data = request.json()
    if "email" not in data:
        logger.info("User creation failed, missing email", extra={"path": "/users"})
        return json_response({"error": "email is required"}, status=400)
    ...

And a server error:

python 
def create_user(request):
    try:
        ...
        db.save(user)
    except DatabaseTimeoutError:
        logger.exception("Database timeout when creating user")
        return json_response({"error": "Internal server error"}, status=500)

This keeps your error logs focused on things you must fix on the server side.

Avoiding Sensitive Data in Logs

Error logs often capture request information and data values. That is useful, but it is dangerous if you log sensitive data.

Sensitive data examples:

Rule: Never log passwords, tokens, or full payment details. Mask or remove sensitive fields before logging.

Example: masking sensitive fields

Suppose you receive this JSON:

json 
{
  "email": "user@example.com",
  "password": "secret123",
  "credit_card": "4111111111111111"
}

Do not log it directly. Instead, sanitize it:

python 
def sanitize_payload(payload: dict) -> dict:
    masked = {}
    for key, value in payload.items():
        if key.lower() in {"password", "token", "api_key"}:
            masked[key] = "***"
        elif key.lower() in {"credit_card", "card_number"} and isinstance(value, str):
            masked[key] = value[:4] + "****" * 3
        else:
            masked[key] = value
    return masked

Then in your error logging:

python 
try:
    process_payment(payload)
except Exception:
    safe_payload = sanitize_payload(payload)
    logger.exception("Payment processing failed", extra={"payload": safe_payload})

This keeps logs useful but safe to share with teammates and support staff.

Centralized Error Logging

In a real system, you typically have many instances of your backend running on multiple servers or containers. Looking at local log files is not practical.

Instead, you send error logs to a centralized logging or error tracking system.

What centralized error logging gives you

Common patterns:

Example: attaching environment and version

When logging errors, attach metadata:

python 
logger.exception(
    "Unhandled error",
    extra={
        "env": "production",
        "service": "orders-api",
        "version": "1.3.5",
    }
)

In a centralized system, this lets you:

Error Aggregation and Deduplication

In production, a single bug can generate thousands of identical error logs. You need ways to handle that volume.

Grouping similar errors

Error tracking tools usually group errors using a fingerprint based on:

This means:

If you build your own system, you can approximate grouping with hashes of messages and key stack frames.

Throttling noisy errors

Sometimes an error explodes, for example if a database is down. You might want to:

Example idea in code:

python 
from collections import defaultdict
import time
error_counts = defaultdict(lambda: {"count": 0, "last_reset": time.time()})
SUPPRESS_AFTER = 10
RESET_WINDOW = 60
def log_error_key(key: str, message: str):
    now = time.time()
    state = error_counts[key]
    if now - state["last_reset"] > RESET_WINDOW:
        state["count"] = 0
        state["last_reset"] = now
    state["count"] += 1
    if state["count"] <= SUPPRESS_AFTER:
        logger.error(message)
    elif state["count"] == SUPPRESS_AFTER + 1:
        logger.error("Suppressed further '%s' errors", key)

You usually do not implement this manually for all errors, but the idea is useful for understanding how tooling manages noisy logs.

Error Logging in Background Jobs

Not all errors happen in HTTP requests. Many backends use background workers for tasks like sending emails or processing payments.

Important points:

Example: logging errors in a worker

Pseudo code for a worker that runs tasks:

python 
def run_task(task_name: str, payload: dict, task_id: str):
    try:
        TASKS[task_name](payload)
    except Exception:
        logger.exception(
            "Error in background task",
            extra={
                "task_name": task_name,
                "task_id": task_id,
            }
        )
        # You might also mark task as failed in a database or retry it

If the worker is separate from your API process, ensure its logs also go to the centralized logging system.

Connecting Error Logs to Monitoring and Alerts

Error logs are not only for debugging after the fact. They are also a signal that something is wrong right now.

Turning error logs into signals

You can create alerts when:

Example alert rules:

ConditionExample alert
> 100 ERROR logs in 5 minutes"High error rate in orders-api in production"
New error group appears"New unhandled exception in checkout endpoint"
Any CRITICAL log in production"Service outage or misconfiguration detected"

These alerts often integrate with chat tools or incident management systems.

Making Error Logs Developer Friendly

Finally, error logs are for humans. Good error logging makes your future investigation easier.

Good practices

Compare:

Bad:

text 
ERROR:root:Error
Traceback (most recent call last):
  ...

Better:

text 
ERROR:orders.checkout:Failed to charge card
user_id=42 order_id=987 path=/orders/987/charge
Traceback (most recent call last):
  ...
PaymentGatewayError: Connection timeout

This tells you immediately:

Checklist

Use this quick checklist for error logging in a backend:

If you satisfy these points, your backend is much easier to maintain, debug, and operate in production.

Views: 16

Comments

Please login to add a comment.

Don't have an account? Register now!

© 2025 Kahibaro
Visitors: 3040 / 24h
Impressum / Imprint
Datenschutz / Data Policy