KAHIBARO
Discord Login Register ☰
☰

20.1. Application Logging

Table of Contents

Why Application Logging Matters

Application logging is how your backend “remembers” what happened. Every important event, error, and decision can be written as a log entry.

Without logs:

With good logs you can:

Key idea: Logging is not optional in backend systems. A production backend without logs is effectively blind.

In this chapter you will learn what to log, how to structure logs, and how to use Python’s logging effectively in backend applications.

Logs vs Other Observability Data

Modern systems use three main types of data:

TypeExample questionsExamples
Logs“What exactly happened at 12:01:05?”Error stack traces, info messages
Metrics“Is error rate up today?”Request count, latency histograms
Traces“How did this single request flow through services?”Distributed tracing spans

Logs are the most detailed. Metrics and traces are covered in other chapters; here we focus on logs as text/structured records of events.

Basic Logging Concepts

Log entries

A log entry is one record of something that happened at a specific time. For example:

text 
2026-08-28T12:01:05Z INFO  user_id=42 path=/login "Login successful"

Most log entries contain:

Logging levels (overview)

Details of log levels are in the next chapter, but you must understand that every log has a severity. Typical levels:

LevelUse for
DEBUGDetailed dev-only information
INFONormal events, like “user registered”
WARNINGSomething odd that still works
ERRORSomething failed for this request or operation
CRITICALApplication or key part is unusable

The level lets you filter and control how noisy logs are.

What to Log in a Backend

Request related events

You should log the key lifecycle events of an HTTP request.

Typical examples:

Example (concept, not language specific):

text 
2026-08-28T12:01:05Z INFO  request_id=abc123 method=POST path=/api/orders user_id=42 "Create order"
2026-08-28T12:01:05Z INFO  request_id=abc123 status=201 duration_ms=134 "Request completed"

Notice how request_id appears in both lines. That lets you search all logs related to a single request.

Business events

Some logs are about business events, not just technical ones:

These help with debugging and also with analytics. Later you might turn them into metrics or events in a data pipeline, but logging them is the first step.

Errors and exceptions

Every error that affects a user should produce a log. For uncaught exceptions you should log:

In Python, this usually means using logger.exception in exception handlers so that the stack trace is recorded.

Background jobs and tasks

Background workers are often hidden from direct user interaction, so logs are your only view of what they are doing.

Log for each job:

Example:

text 
2026-08-28T12:10:00Z INFO  job=email_send job_id=xyz user_id=42 "Sending verification email"
2026-08-28T12:10:01Z ERROR job=email_send job_id=xyz user_id=42 "SMTP connection failed"
2026-08-28T12:10:10Z INFO  job=email_send job_id=xyz user_id=42 attempt=2 "Sending verification email"

What Not to Log

Logging too much is harmful. It can:

Sensitive data

Never log secrets, even at DEBUG:

Obfuscate or remove confidential fields.

Rule: Treat logs as if any employee and possibly attackers could read them. Do not log secrets or personal data unless absolutely necessary and allowed by law.

Example, bad:

text 
INFO "User login" email=john@example.com password=supersecret

Better:

text 
INFO "User login" email_hash=fd5a... ip=203.0.113.10

You can hash or partially mask values (for example show last 4 digits) when you need to correlate but not reveal.

Large payloads

Avoid logging:

Instead log:

Bad:

text 
DEBUG "Received payload" payload={very_large_json_here}

Better:

text 
DEBUG "Received payload" body_size=8453 items_count=37

Text Logs vs Structured Logs

Plain text logs

Plain text logs are simple strings. They are easy to read but harder for machines to parse.

Example:

text 
2026-08-28 12:01:05 [INFO] User 42 created order 1001 in 134ms

Problems:

Structured logs (JSON)

Structured logs store fields in a machine readable format, usually JSON.

Example:

json 
{
  "timestamp": "2026-08-28T12:01:05Z",
  "level": "INFO",
  "message": "Order created",
  "user_id": 42,
  "order_id": 1001,
  "duration_ms": 134,
  "request_id": "abc123"
}

Benefits:

Most modern backend teams prefer structured logging with JSON, especially once logs are sent to tools like Elasticsearch, Loki, or cloud logging services.

For local development you can still pretty print JSON or use a human friendly formatter.

Python Logging Basics

You will use Python for backend development in this course, so you need a basic mental model of the logging system.

The logger, handler, formatter pipeline

Python’s logging has three main parts:

PartWhat it doesExample
LoggerWhere your code writes messageslogger.info("Order created")
HandlerWhere logs are sentConsole, file, HTTP, syslog
FormatterHow logs are turned into text or JSON'%(asctime)s %(levelname)s %(message)s'

When you call logger.info(...):

  1. The logger receives your log record.
  2. It passes it to one or more handlers.
  3. Each handler uses a formatter to produce the final string or JSON.
  4. The handler writes to a destination like stdout or a file.

A minimal setup

Very simple example that logs to the console:

python 
import logging
logging.basicConfig(
    level=logging.INFO,
    format="%(asctime)s %(levelname)s %(name)s %(message)s",
)
logger = logging.getLogger(__name__)
logger.info("Application started")
logger.warning("Low disk space")

Output:

text 
2026-08-28 12:00:00,123 INFO  __main__ Application started
2026-08-28 12:00:01,456 WARNING __main__ Low disk space

In real backends you will use more advanced configuration, but the pattern is always:

  1. Configure logging once at startup.
  2. Get a logger in each module.
  3. Use logger.debug/info/warning/error/exception/critical to write logs.

Adding Context to Logs

Logs become very powerful when you include context fields.

Common context fields in a backend:

FieldDescription
request_idUnique ID for each HTTP request
user_idAuthenticated user ID or anonymous
pathRequest path, for example /api/orders
methodHTTP method, such as GET or POST
statusHTTP status code
duration_msTime spent handling the request
serviceService name in a microservice architecture

Example: logging with context in Python

Imagine you have request_id and user_id available. One simple pattern:

python 
logger.info(
    "Order created",
    extra={"user_id": user_id, "order_id": order_id, "request_id": request_id},
)

A JSON formatter could turn this into:

json 
{
  "timestamp": "2026-08-28T12:01:05Z",
  "level": "INFO",
  "message": "Order created",
  "user_id": 42,
  "order_id": 1001,
  "request_id": "abc123"
}

Your FastAPI middleware can automatically attach request_id, path, and method to all logs created during a request, so you do not have to pass them manually every time. That pattern is covered in later chapters, but you should already understand why it is important.

Logging in a Web Application

Where to log in request handling

In a typical FastAPI application you might log in:

  1. Startup and shutdown:
    • “Application started,” “Connected to database.”
  2. Middleware:
    • Incoming request details.
    • Outgoing response status and time.
  3. Endpoints (controllers):
    • Key domain actions, for example “Created task,” “User updated profile.”
  4. Services:
    • Non trivial operations, for example “Charging card,” “Inventory reserved.”
  5. Error handlers:
    • Unexpected exceptions with stack traces.

Example flow:

text 
[Middleware] Received request GET /api/tasks user_id=42 request_id=abc123
[Service]   Listing tasks user_id=42 request_id=abc123
[Middleware] Completed request GET /api/tasks 200 duration_ms=14 request_id=abc123

Notice how the same request_id appears in each log. That lets you:

Logging Strategy and Best Practices

Choose consistent formats

Decide early:

Example standard fields:

FieldRequired?Example value
timestampYes2026-08-28T12:01:05Z
levelYesINFO
messageYesOrder created
serviceYesorders-api
request_idRecommendedabc123
user_idRecommended42 or anonymous

Having a standard makes it easy to search across all your services.

Log at the right level

Although detailed log level usage is covered in the next chapter, some core guidelines:

Rule: An application with everything at INFO or DEBUG is as useless as an application with no logs. Too much noise hides real problems.

Avoid duplication

Do not log the same error in 5 places. Typical pattern:

Example:

python 
def service_get_user(user_id: int):
    user = repo_get_user(user_id)  # If this raises, we do not log here
    if not user:
        raise UserNotFoundError(user_id)
    return user
@app.get("/users/{user_id}")
def get_user_endpoint(user_id: int):
    try:
        user = service_get_user(user_id)
        return user
    except UserNotFoundError:
        logger.info("User not found", extra={"user_id": user_id})
        raise HTTPException(status_code=404, detail="User not found")
    except Exception:
        logger.exception("Unexpected error in get_user_endpoint", extra={"user_id": user_id})
        raise HTTPException(status_code=500, detail="Internal server error")

Only the endpoint logs, not every inner function. This keeps logs clean and readable.

Think about retention and volume

Every log line costs:

You should:

Example: Simple Logging Plan for a New API

Imagine you are building a simple Task Management API.

Here is a minimal logging plan:

  1. Startup logs:
    • “Starting Task API, version X.Y.”
    • “Connected to PostgreSQL” or error.
  2. Request logs (middleware):
    • On start: method, path, request_id, user_id.
    • On complete: status, duration_ms, request_id.
  3. Domain events (INFO):
    • task_created, task_completed, task_deleted with task_id, user_id.
  4. Warnings:
    • When a user hits a soft limit, for example “Too many tasks, consider upgrading.”
  5. Errors:
    • Unhandled exceptions with stack traces, status 500.
    • External service failures, for example “Email service timeout.”

Sample log lines:

text 
2026-08-28T12:00:00Z INFO  service=tasks-api "Starting Task API" version=1.0.0
2026-08-28T12:00:00Z INFO  service=tasks-api "Database connection established"
2026-08-28T12:01:05Z INFO  service=tasks-api request_id=abc123 method=POST path=/api/tasks user_id=42 "Request started"
2026-08-28T12:01:05Z INFO  service=tasks-api request_id=abc123 user_id=42 task_id=1001 "Task created"
2026-08-28T12:01:05Z INFO  service=tasks-api request_id=abc123 status=201 duration_ms=12 "Request completed"
2026-08-28T12:02:10Z ERROR service=tasks-api request_id=def456 user_id=42 "Unexpected error in /api/tasks" error="TimeoutError"

From this you can:

Summary

Later chapters will build on this and show how to define log levels, structure logs across services, and integrate them with monitoring tools.

Views: 10

Comments

Please login to add a comment.

Don't have an account? Register now!

© 2025 Kahibaro
Visitors: 3040 / 24h
Impressum / Imprint
Datenschutz / Data Policy