Best practices for configuration

Table of Contents

Principles of Good Configuration Design

Configuration in OpenShift should be:

Declarative – stored as YAML manifests in Git, not edited by hand in the cluster.
Environment-specific, but app-consistent – same app, same config shape across environments; only values differ.
Immutable at runtime (as much as possible) – changes are made via new versions, not by manual edits in a running pod.
Separation of concerns – application code, configuration data, and secrets are strictly separated.

Applied to OpenShift, this means using ConfigMap, Secret, Deployment, BuildConfig, etc. in a consistent way and avoiding “magic” configuration inside images.

Key goals:

Make configuration auditable (via Git and cluster resources).
Make configuration reproducible (same manifests deployed repeatedly give the same result).
Make configuration safe (no secrets in the wrong places; safe defaults).

What Belongs Where: ConfigMaps, Secrets, and Images

A core best practice is deciding where each piece of configuration lives.

Split configuration types clearly

Use ConfigMaps for:

Non-sensitive configuration (feature flags, timeouts, logging levels).
Templates, config files (e.g. nginx.conf, application YAML/INI).
Command-line arguments or environment variables that aren’t secret.

Use Secrets for:

Credentials (DB passwords, API keys, tokens).
TLS certificates and keys.
SSH keys, private keys, encryption keys.

Use images for:

Application code.
Static assets that don’t change per-environment.
“Base defaults” but not environment-specific values.

Anti-patterns to avoid

Secrets baked into images
Never embed credentials in container images, even if the registry is private.
Environment-specific images
Do not build separate images for dev/stage/prod just to change config.
Shared credentials across environments
Each environment must have its own secrets; never reuse prod credentials elsewhere.

Structuring Configuration in OpenShift

Prefer environment variables for simple values

For simple key-value pairs:

Use env or envFrom with ConfigMap / Secret.
Keep env names consistent across environments (e.g. always DB_HOST, DB_USER).

Advantages:

Easy to inspect via oc describe pod.
Clear mapping between app code and configuration.

Examples:

env:
  - name: APP_LOG_LEVEL
    valueFrom:
      configMapKeyRef:
        name: myapp-config
        key: log.level
  - name: DB_PASSWORD
    valueFrom:
      secretKeyRef:
        name: myapp-db-credentials
        key: password

Use mounted files for structured config

When configuration is more complex:

Use ConfigMaps or Secrets as mounted files:

Application config files (application.yaml, settings.json).
CA bundles, keystores, certificates.

Mount them read-only and at a known path.

Example pattern:

volumes:
  - name: app-config
    configMap:
      name: myapp-config
containers:
  - name: myapp
    volumeMounts:
      - name: app-config
        mountPath: /etc/myapp
        readOnly: true

Prefer this when:

The config has many keys or nested structures.
Multiple containers share the same config file.

Avoid configuration sprawl

Configuration should be organized, not scattered:

Define one primary ConfigMap per application component (e.g. myapp-config), with optional specialized ones (e.g. myapp-feature-flags).
Keep keys namespaced by logical area:

db.host, db.port, db.poolSize
logging.level, logging.format

Avoid one huge “global” ConfigMap used by many unrelated apps.

Managing Configuration Across Environments

Keep configuration shape identical

Across environments (dev, test, prod):

Keep the same keys and overall structure.
Only values differ.

This enables:

Reusing the same manifests with different overlays.
Easier promotion pipelines (same config contract, different values).

Bad:

Dev uses DB_HOST, prod uses DATABASE_URL.

Good:

All environments use DB_HOST, DB_PORT, DB_NAME, etc., with different values.

Use clear naming conventions

Common patterns:

Namespaced by app and environment:

myapp-config-dev, myapp-config-prod
myapp-db-credentials-dev, myapp-db-credentials-prod

Or same resource names per environment but separated by namespace:

myapp-config in myapp-dev
myapp-config in myapp-prod

Pick one pattern and apply it consistently.

Use Git and overlays (GitOps-friendly)

Store all configuration manifests in Git:

Base manifests defining common resources and structure.
Environment overlays or kustomizations for specific values:

Different ConfigMap/Secret data.
Different replica counts, resource limits.

Benefits:

Versioning and history of configuration changes.
Review via pull requests.
Easy to roll back to a known good state.

Even without a full GitOps setup, treat configuration as code.

Security-Focused Configuration Practices

Never store secrets in plain text sources

Avoid:

Secrets in Git (even private repos).
Secrets in CI logs, console output, or shared chat.
Secrets in OpenShift resource YAML files stored in SCM.

Preferred approaches:

Use separate secret management for source (e.g. sealed secrets, external secret stores).
Keep the actual Secret resources generated from secure storage, not hand-written.

Use least privilege and separation

Different credentials per:

Environment (dev, test, prod).
Application or team (avoid app sharing root DB passwords).

Limit secret access via:

Namespaces and RBAC.
ServiceAccounts bound to particular workloads.

Application pods should only be able to read the Secrets they actually need.

Validate that configuration doesn’t leak secrets

Practices:

Do not surface secrets in:

Metrics.
Logs.
Debug endpoints.

When troubleshooting:

Avoid oc describe secret (shows data as base64) unless truly needed.
Prefer listing references to secrets, not the content.

Use separate ConfigMaps and Secrets instead of mixing sensitive and non-sensitive values in one resource. This reduces the risk of exposing secrets by accident.

Operational Best Practices

Make configuration changes controlled and trackable

Always change configuration via:

Git commits and CI/CD pipelines, or
Well-documented oc commands/scripts.

Avoid manual oc edit in production namespaces.

Use:

Labels and annotations to document:

Owner and purpose.
Ticket or change request IDs.
Version or timestamp.

Example:

metadata:
  name: myapp-config
  labels:
    app: myapp
    component: backend
  annotations:
    owner: "team-payments"
    change-ticket: "INC-12345"

Minimize required restarts for config changes

Some configuration requires pod restarts, some not:

If config values are changed often:

Consider implementing dynamic reload in the app (e.g. file watchers, SIGHUP).
Use mounted config files and reload logic.

If dynamic reload is not supported:

Document that configuration changes cause a rollout.
Use controlled rolling updates with readiness checks.

Balance between:

Operational simplicity.
Need for fast configuration changes.

Validate configuration before applying

Avoid pushing broken configs into production:

Validate configuration in lower environments first.
Add automated checks in pipeline:

Lint YAML files.
Run unit tests using sample configuration.
Run smoke tests post-deployment.

If configuration is used by multiple tenants or teams, treat changes as backward-incompatible unless proven safe.

Use defaults and fallbacks carefully

In the application:

Provide sensible defaults for non-critical options.
Explicitly require critical values (e.g., DB credentials) to be present; fail fast if missing.

In OpenShift manifests:

Keep default values in ConfigMap when:

They are safe and non-secret.

Avoid “hidden” defaults in multiple places; centralize into:

A single ConfigMap.
Or code-level defaults with environment overrides.

Designing for Multi-Tenancy and Teams

Namespace-level isolation

Use namespaces to:

Separate configuration for different:

Applications.
Teams.
Environments or tenants.

Each namespace:

Has its own ConfigMaps and Secrets.
Can have its own resource quotas and policies.

This prevents accidental cross-usage of configuration between unrelated workloads.

Align configuration ownership and responsibilities

Define:

Who owns which ConfigMaps/Secrets (teams, roles).
Who may change production configuration.
Approval workflows for changes.

Common models:

Application team owns application ConfigMaps.
Platform team owns cluster-wide config (logging, ingress, monitoring).
Security team controls policies for secrets.

Patterns for Complex Applications

Shared vs. per-service configuration

For microservices:

Put service-specific config in that service’s own ConfigMap/Secrets.
Use shared ConfigMaps for:

Common endpoints (logging, tracing, service discovery).
Cross-cutting feature flags.

But:

Avoid huge global ConfigMaps with many unrelated entries.
Use separate shared configs per domain, e.g.:

observability-config
auth-config
platform-endpoints

Configuration for jobs and batch workloads

Batch jobs often:

Use different parameters per run (dates, input sources).

Best practices:

Use parameters as environment variables derived from:

ConfigMaps for defaults.
Job-specific overrides.

Avoid copying entire ConfigMaps per job; override only what changes.