KAHIBARO
Discord Login Register ☰
☰

15.12. Secrets Management

Table of Contents

Why Secrets Management Matters

In backend development you constantly work with sensitive values such as database passwords, API keys, encryption keys, and tokens. Collectively these are called secrets.

If secrets leak, attackers can often bypass your authentication and authorization, or directly connect to your database or third‑party services.

Never hard‑code secrets in your source code or commit them to version control.

This chapter focuses on how to store, access, and rotate secrets safely, not on authentication or encryption theory which are covered in other chapters.

What Counts as a Secret?

A secret is any value that must be kept confidential in order to keep your system secure.

Typical examples in backend systems:

TypeExample valueWhy it is sensitive
Database credentialspostgres://user:pass@db:5432/appLets an attacker access or destroy your data
API keyssk_live_51N9... (Stripe secret key)Lets attacker charge cards, refund payments, etc.
OAuth client secretsGOOGLE_CLIENT_SECRET=abc123Lets attacker impersonate your app with Google
JWT signing keysJWT_SECRET=super-secret-keyLets attacker create valid tokens and impersonate users
Encryption keysMASTER_KEY=base64:...Lets attacker decrypt stored data
SSH keysid_rsa private keyLets attacker log into servers
SMTP credentialsSMTP_PASSWORD=mailpassLets attacker send email as your domain
Third‑party tokensSLACK_BOT_TOKEN=xoxb-...Lets attacker access external services
Cloud credentialsAWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEYLets attacker control your cloud resources

Non‑examples:

As a simple rule:

If exposing a value would let an attacker do something harmful, treat it as a secret.

Common Mistakes with Secrets

Understanding what not to do makes good practices clearer.

Hard‑Coding Secrets in Source Code

Example of what you should avoid:

python 
# config.py
DB_URL = "postgresql://user:mysecretpassword@localhost:5432/app_db"
JWT_SECRET = "super-secret-key-123"
STRIPE_SECRET_KEY = "sk_live_..."

Problems:

Committing Secret Files to Git

Example:

bash 
# .env
DATABASE_URL=postgresql://user:password@localhost:5432/app_db
JWT_SECRET=super-secret-key

If .env is tracked by Git:

Always use a .gitignore entry for local secret files:

gitignore 
# .gitignore
.env
.env.*
secrets.json

Sharing Secrets Over Insecure Channels

Risky patterns:

Better patterns:

Principles of Good Secrets Management

These principles guide the techniques and tools you use.

Principle 1: Externalize Configuration

Configuration values, especially secrets, should come from outside your code.

Typical sources:

Example pattern:

python 
import os
DATABASE_URL = os.environ["DATABASE_URL"]
JWT_SECRET = os.environ["JWT_SECRET"]

Your code knows how to read configuration, but not the actual values.

Principle 2: Least Privilege

Each component should have only the secrets it actually needs, and each secret should grant only the minimal permissions.

Examples:

Principle 3: Separation of Environments

Never reuse the same secret across different environments.

Better structure:

EnvironmentDatabase passwordJWT secretStripe key
locallocal_db_passlocal_jwt_secretTest key sk_test_...
stagingstaging_passstaging_jwtTest key sk_test_...
productionprod_passprod_jwtLive key sk_live_...

This keeps testing and experimentation from harming real users.

Principle 4: Rotate and Revoke

Secrets should be changeable and revocable without changing your code.

You should be able to:

Principle 5: Audit and Visibility

You should:

This is mainly handled through secret management tools and cloud platforms.

Environment Variables for Secrets

Environment variables are often the simplest, first step to get secrets out of code.

Basics

In a Unix shell:

bash 
export DATABASE_URL="postgresql://user:password@localhost:5432/app_db"
export JWT_SECRET="super-secret-key"

In Python:

python 
import os
DATABASE_URL = os.environ["DATABASE_URL"]
JWT_SECRET = os.environ["JWT_SECRET"]

If a variable is missing, os.environ["KEY"] raises a KeyError. You can use .get with a fallback, but for secrets you usually require them.

python 
JWT_SECRET = os.environ.get("JWT_SECRET")
if not JWT_SECRET:
    raise RuntimeError("JWT_SECRET not set")

Using .env Files Locally

Often you create a local .env file that defines environment variables:

env 
# .env (do NOT commit this)
DATABASE_URL=postgresql://user:password@localhost:5432/app_db
JWT_SECRET=local-dev-secret

Then you use a library like python-dotenv to load it:

bash 
pip install python-dotenv
python 
# main.py
from dotenv import load_dotenv
import os
load_dotenv()  # reads from .env
DATABASE_URL = os.environ["DATABASE_URL"]
JWT_SECRET = os.environ["JWT_SECRET"]

Important: Ensure .env is ignored by Git:

gitignore 
.env

You can keep a non secret example file:

env 
# .env.example (commit this)
DATABASE_URL=postgresql://user:password@localhost:5432/app_db
JWT_SECRET=change-me

This shows other developers which variables are required without exposing your real secrets.

Using Environment Variables in Docker

With Docker, you can pass secrets as environment variables.

Example docker-compose.yml:

yaml 
services:
  api:
    image: my-api
    environment:
      - DATABASE_URL=${DATABASE_URL}
      - JWT_SECRET=${JWT_SECRET}

The ${VAR} values are taken from your shell environment or a .env file next to docker-compose.yml.

Example .env file for Docker:

env 
DATABASE_URL=postgresql://user:password@db:5432/prod_db
JWT_SECRET=my-super-secret

Again, do not commit the real secrets to version control.

Storing Secrets in Configuration Files

Sometimes you store secrets in files that are not committed to Git and are loaded at runtime.

Example: secrets.json

json 
{
  "database_url": "postgresql://user:password@localhost:5432/app_db",
  "jwt_secret": "super-secret-key"
}

Python code:

python 
import json
from pathlib import Path
config_path = Path("secrets.json")
config = json.loads(config_path.read_text())
DATABASE_URL = config["database_url"]
JWT_SECRET = config["jwt_secret"]

Security notes:

This pattern is simple but best used for local development or small deployments. For production you usually want a dedicated secret management system.

Dedicated Secret Management Systems

As your application grows, you want centralized tools designed to handle secrets safely and at scale.

Common systems:

ToolTypical usage
HashiCorp VaultSelf‑hosted or managed, very flexible
AWS Secrets ManagerSecrets in AWS environments
AWS Systems Manager (SSM)Parameter store, including encrypted parameters
GCP Secret ManagerSecrets in Google Cloud
Azure Key VaultSecrets, keys, and certificates in Azure

Typical Features

Most secret managers provide:

Example: Conceptual Flow with a Secret Manager

High level example using a generic secret manager:

  1. You store a secret with a name and value:
    • Name: prod/database/url
    • Value: postgresql://user:pass@db:5432/prod_db
  2. Your application authenticates to the secret manager, usually with a non secret identity, for example:
    • Cloud instance role.
    • Kubernetes service account.
    • Short‑lived credentials.
  3. Your application fetches the secret at startup:
python 
   from my_secret_client import get_secret
   DATABASE_URL = get_secret("prod/database/url")
  1. The secret manager logs the access and returns the decrypted value.

The exact client library and configuration depend on the platform, but the core idea is always similar.

Example: Secrets with AWS (Conceptual)

You do not need AWS experience to understand the pattern, it is similar on other clouds.

Using AWS Secrets Manager (High Level)

  1. In AWS Secrets Manager you create a secret:
    • Name: prod/myapp/db-credentials
    • Value (JSON):
json 
     {
       "username": "myapp",
       "password": "very-secret",
       "host": "db.example.com",
       "port": 5432
     }
  1. You give your EC2 instance or ECS task a role that allows reading this secret.
  2. In your app code you use the AWS SDK:
python 
   import boto3
   import json
   client = boto3.client("secretsmanager")
   response = client.get_secret_value(SecretId="prod/myapp/db-credentials")
   secret_dict = json.loads(response["SecretString"])
   DATABASE_URL = (
       f"postgresql://{secret_dict['username']}:{secret_dict['password']}"
       f"@{secret_dict['host']}:{secret_dict['port']}/myapp_db"
   )
  1. No passwords are stored in source code or environment variables. The instance role identifies your application to AWS.

This pattern is similar with other providers: you create a secret, attach permissions to your compute resources, and fetch the secret in code at runtime.

Managing Different Environments

You should manage secrets per environment in a consistent way.

Naming Conventions

Conventions make your setup easier to understand.

For environment variables:

bash 
# local .env
APP_ENV=local
DATABASE_URL=postgresql://user:pass@localhost:5432/local_db
JWT_SECRET=local-secret

For staging:

bash 
# staging secrets
APP_ENV=staging
DATABASE_URL=postgresql://user:pass@staging-db:5432/staging_db
JWT_SECRET=staging-secret

In code:

python 
import os
APP_ENV = os.environ.get("APP_ENV", "local")
DATABASE_URL = os.environ["DATABASE_URL"]

For secret managers you can prefix names, for example staging/myapp/db-url and prod/myapp/db-url.

Secret Value Differences

Across environments you may:

This reduces the blast radius of mistakes.

Secret Rotation and Revocation

You need to be able to change secrets safely.

When to Rotate

You should rotate secrets when:

Basic Manual Rotation Pattern

Example: Rotating a database password.

  1. Create a new password for the same database user.
  2. Update the secret storage (env var, secret manager, config file) with the new password.
  3. Restart or reload the application so it uses the new password.
  4. Test that the app can connect with the new password.
  5. Revoke the old password or remove any old users.

If you cannot change the password for the same user, you can:

Rotating JWT Signing Keys (Conceptual)

JWT signing keys are critical. If a JWT secret leaks, an attacker can create valid tokens.

Basic approach:

  1. Introduce a key ID kid in your JWT header.
  2. Keep a map of kid → signing key in your backend.
  3. Add a new key with a new kid and start signing new tokens with it.
  4. Keep old keys for some time to verify existing tokens.
  5. After all old tokens have expired, remove the old keys.

The detailed implementation belongs to the authentication chapters, but you should see how secret rotation affects token handling.

Handling Secrets in Development

Developers still need secrets to run the app locally, but you must avoid leaks.

Using .env Files Per Developer

Typical pattern:

Example .env.example:

env 
APP_ENV=local
DATABASE_URL=postgresql://user:password@localhost:5432/app_db
JWT_SECRET=change-me
STRIPE_SECRET_KEY=sk_test_...

A new developer:

bash 
cp .env.example .env
# then edit .env with real local values

This keeps the structure visible while not sharing real secrets in Git.

Using Safer Sharing Methods

If multiple developers need the same non production secrets, you can:

These are more advanced setups, but the idea is to avoid clear text secrets in your repository.

Avoiding Secret Leaks in Logs and Errors

Even if you store secrets correctly, you can leak them at runtime.

Do Not Log Secrets

Risky code:

python 
print(f"Connecting to database: {DATABASE_URL}")

If DATABASE_URL contains the password, your logs now have it.

Better:

python 
from urllib.parse import urlparse
def safe_db_url(url):
    parsed = urlparse(url)
    return f"{parsed.scheme}://{parsed.hostname}:{parsed.port}/{parsed.path.lstrip('/')}"
print(f"Connecting to database: {safe_db_url(DATABASE_URL)}")

Or log only the database host or name, not credentials.

Be Careful with Error Messages

Avoid revealing secrets in stack traces or error pages.

Example of a bad pattern:

python 
try:
    connect_to_db(DATABASE_URL)
except Exception as exc:
    raise RuntimeError(f"Failed to connect to {DATABASE_URL}") from exc

Better:

python 
try:
    connect_to_db(DATABASE_URL)
except Exception as exc:
    raise RuntimeError("Failed to connect to database") from exc

Your logs may still contain internal connection details, so avoid including secret values.

Scanning for Leaked Secrets

Even with good practices, accidents happen. Tools can help you detect secrets in your codebase.

Types of Secret Scanning

Popular tools (conceptual, no need to learn them now):

If you ever discover a secret that has been committed, act as if it is leaked:

  1. Remove it from the code.
  2. Rotate the secret.
  3. Invalidate the old value.
  4. Consider scanning your entire repository.

Practical Patterns for Small Projects

For small projects or early learning, use patterns that are simple but not careless.

Minimal Reasonable Setup

For a simple FastAPI project:

  1. Do not put secrets in .py files.
  2. Create .env and .env.example:

.env.example:

env 
   APP_ENV=local
   DATABASE_URL=postgresql://user:password@localhost:5432/app_db
   JWT_SECRET=change-me

.env (ignored by Git):

env 
   APP_ENV=local
   DATABASE_URL=postgresql://user:localpass@localhost:5432/app_db
   JWT_SECRET=really-long-random-string
  1. Load .env in your app:
python 
   from dotenv import load_dotenv
   import os
   load_dotenv()
   APP_ENV = os.getenv("APP_ENV", "local")
   DATABASE_URL = os.environ["DATABASE_URL"]
   JWT_SECRET = os.environ["JWT_SECRET"]
  1. When you deploy, set environment variables through:
    • Your hosting provider dashboard.
    • Docker Compose environment.
    • Kubernetes secrets.

This gives you a clean separation between code and secrets, even on your first backend project.

Summary

Always externalize secrets, limit their scope, separate environments, and be ready to rotate and revoke them quickly.

With these habits your backend will be far more resilient to common security failures that come from poor secrets management.

Views: 7

Comments

Please login to add a comment.

Don't have an account? Register now!

© 2025 Kahibaro
Visitors: 3542 / 24h
Impressum / Imprint
Datenschutz / Data Policy