KAHIBARO
Discord Login Register ☰
☰

28.7. High Availability

Table of Contents

Understanding High Availability

High availability means designing your backend so that it is up and serving users almost all the time, even when parts of the system fail or need maintenance. It is about reducing downtime and making failures boring and recoverable, instead of catastrophic.

High availability is closely related to fault tolerance, monitoring, and deployment, but here we focus on what makes a backend highly available in production, and how you can design for it.

Key Concepts of High Availability

High availability usually talks about three main ideas: availability, reliability, and resilience.

A simple relation for availability is:

$$
\text{Availability} = \frac{\text{MTBF}}{\text{MTBF} + \text{MTTR}}
$$

Where:

Important rule: To improve availability, you can either fail less often (increase MTBF) or recover faster (reduce MTTR). High availability focuses heavily on reducing MTTR through redundancy and automation.

Common Availability Levels

AvailabilityNicknameAllowed downtime per month
99%2 nines~7 hours 18 minutes
99.9%3 nines~43 minutes
99.99%4 nines~4 minutes 19 seconds
99.999%5 nines~26 seconds

Even moving from 99.9% to 99.99% can be very expensive in terms of architecture and operations. The right target depends on your product and users.

Single Points of Failure

A single point of failure (SPOF) is any component that, if it fails, brings the whole system down.

Examples of SPOFs in a simple backend:

To achieve high availability, your main job is to find SPOFs and remove or reduce them.

Identifying SPOFs: Concrete Examples

Imagine a basic FastAPI + PostgreSQL deployment:

SPOFs here:

ComponentSPOF?Why
FastAPI appYesIf that container dies, no app.
PostgreSQLYesIf it dies, app cannot use data.
RedisYesIf used for sessions, users break
NginxYesIf it fails, no entry point
Network linkYesIf network to server fails

Awareness is the first step. Later we will see how to add redundancy and failover around these.

Redundancy and Replication

Redundancy means having multiple instances of the same component so that if one fails, others can take over. Replication means having the same data stored in multiple places.

Both are core tools to reach high availability.

Redundant Application Servers

Instead of one application server, you run multiple instances behind a load balancer.

This does not help if:

So redundancy should go beyond just application processes.

Database Replication

For relational databases such as PostgreSQL, a common pattern is:

If the primary fails, a failover process promotes a replica to primary.

Simplified example:

  1. You have:
    • db-primary in zone A.
    • db-replica-1 in zone B.
    • db-replica-2 in zone C.
  2. Your application:
    • Sends all writes to db-primary.
    • Sends some reads to replicas (optional but common).
  3. When db-primary fails:
    • A failover tool (or managed service) automatically chooses a replica and promotes it to primary.
    • App either reconnects automatically or gets reconfigured to point to the new primary.

Types of Replication

TypeDescriptionExample use case
SynchronousPrimary waits until replica writes dataStrong consistency, smaller setups
AsynchronousPrimary does not wait for replicaHigher performance, possible lag
Multi-primaryMultiple writable nodesAdvanced setups, conflict resolution

For many backends, async replication with automatic failover is a reasonable balance.

Important rule: Do not confuse replication with backups. Replication copies current data to other nodes, including accidental deletions. You still need backups.

Health Checks and Failover

High availability requires the system to detect failures and move traffic away from unhealthy components quickly.

Health Checks

A health check is an automated test that runs repeatedly to see if a component is working.

At the application level, you might expose:

Simple FastAPI example:

python 
from fastapi import FastAPI
import asyncpg
app = FastAPI()
db_pool = None
@app.on_event("startup")
async def startup():
    global db_pool
    db_pool = await asyncpg.create_pool(dsn="postgresql://...")
@app.get("/healthz")
async def healthz():
    async with db_pool.acquire() as conn:
        await conn.execute("SELECT 1")
    return {"status": "ok"}

Your load balancer can then:

Types of Health Checks

TypeWhat it checksCostExamples
LivenessIs the process alive?Very lowUsed by orchestrators like Kubernetes
ReadinessCan this instance serve traffic now?Low / mediumDB connection, config loaded
Deep healthDeeper checks of dependenciesHigherQueue connectivity, external APIs, etc.

For high availability, readiness checks are crucial, because an app might be running but temporarily unable to serve.

Failover

Failover is the automatic switch from a failed component to a healthy one.

Examples:

Key points:

Typical configuration at a load balancer:

Load Balancing for High Availability

A load balancer sits in front of your application instances and splits traffic between them. It is also a central piece for health checks and failover.

What Load Balancers Do

Common Load Balancing Algorithms

AlgorithmDescriptionExample use case
Round robinEach request goes to the next instanceGood default
Least connectionsRequest goes to instance with fewest connectionsLong-lived connections (WebSockets)
IP hashSame client IP goes to same instanceSimple session stickiness
Weighted round robinSome instances get more trafficMixed instance sizes

For most basic HTTP APIs, round robin is enough.

Typical High-Availability Topology

Imagine a simple high availability setup:

A request flow:

  1. User hits https://api.example.com.
  2. DNS resolves api.example.com to the load balancer IP.
  3. Load balancer chooses a healthy FastAPI instance.
  4. FastAPI instance connects to the primary database.
  5. Response is returned to user.

If one FastAPI instance fails, the load balancer stops sending traffic to it. If one availability zone has issues, the remaining zones still serve users.

Statelessness and Session Management

Stateless applications are easier to scale and keep highly available.

Stateless vs Stateful

For high availability, aim to make your application layer stateless.

Sessions and High Availability

A classic problem is user sessions. If you store sessions in process memory:

Solutions:

  1. Sticky sessions at the load balancer:
    • Same user always goes to the same instance.
    • Simple but creates coupling between user and instance.
    • If the instance fails, the user loses the session.
  2. Shared session storage:
    • Store sessions in Redis, database, or another central store.
    • Any instance can read any user's session.
    • If an instance fails, user can continue on another instance.
  3. Stateless authentication:
    • Use JWTs or similar tokens.
    • The token contains all needed data, signed by the server.
    • Server does not store any session data, just validates tokens.

For high availability, 2 or 3 are preferred.

Example of storing sessions in Redis:

Important rule: To keep application instances stateless, do not store important user state only in memory of a single instance. Use shared or stateless mechanisms.

Multi-Zone and Multi-Region Setups

High availability is not only about processes, but also about infrastructure failures, such as power outages or network problems in a data center.

Cloud providers offer:

Multi-AZ (Multi-Zone) Architecture

A common step towards high availability:

Benefits:

Basic pattern:

ComponentPlacement
Load balancerPublic, spans all zones
App instancesAt least 2 zones
DB primaryZone A
DB replicasZone B and C

Multi-Region Architecture

Multi-region is more complex, but can:

Common patterns:

  1. Primary region + warm secondary:
    • All writes go to the primary region.
    • Secondary region has replicated data.
    • Traffic normally goes to primary region.
    • If primary fails, failover DNS or load balancer to secondary.
  2. Active-active regions:
    • Both regions handle traffic.
    • Database is replicated in both directions.
    • Requires careful conflict resolution and design.

Multi-region often involves:

This is advanced and often not needed for early stages, but it is central for very high availability (e.g. 4 or 5 nines).

Maintenance Without Downtime

High availability is not only about random failures. It is also about planned changes without visible downtime.

Rolling Deployments

Instead of stopping all instances and deploying new code, you:

  1. Have multiple instances running version A.
  2. Start new instances with version B.
  3. Load balancer:
    • Marks an old instance as "draining".
    • Stops sending new requests there, but lets existing requests finish.
  4. Once old instance is idle, shut it down.
  5. Repeat until all instances run version B.

During this process, users always have some healthy instances available.

Database Maintenance

Even databases need maintenance:

Approaches:

Blue-Green Deployments

Another pattern:

Deployment steps:

  1. Deploy new version to green.
  2. Test green environment thoroughly.
  3. Switch traffic from blue to green (via DNS, load balancer, etc.).
  4. Keep blue as a fallback for quick rollback for some time.

Blue-green allows quick rollback and minimal downtime during large changes.

Designing for Degradation, Not Perfection

High availability does not mean "everything always works". It means:

Examples of graceful degradation:

You should decide which features are critical and which are nice to have, and design degradation paths.

Practical Checklist for High Availability

Here is a simple checklist you can apply to a backend:

AreaQuestion
ApplicationDo we run multiple instances in different zones?
ApplicationDo we use health checks and automatic removal of bad instances?
SessionsAre sessions or auth tokens shared or stateless?
DatabaseDo we have replicas and automatic or manual failover?
DatabaseAre backups tested and restorable?
Cache / RedisIs there a replication or persistence strategy?
NetworkIs there a single critical IP or region?
DeploymentCan we deploy without full downtime?
MonitoringCan we detect failures quickly and alert humans?

You will not have everything perfect from day one. Start from the biggest risks and single points of failure, and improve step by step.

Summary

High availability is about staying up in the face of failures and maintenance. You:

These ideas combine with monitoring, fault tolerance, and incident response to form a production-grade backend that real users can depend on.

Views: 5

Comments

Please login to add a comment.

Don't have an account? Register now!

© 2025 Kahibaro
Visitors: 3096 / 24h
Impressum / Imprint
Datenschutz / Data Policy