KAHIBARO
Discord Login Register ☰
☰

16.7 Distributed Caching

Table of Contents

Why Distributed Caching?

When your application runs on a single server, a simple in‑memory cache (for example a Python dict or an in‑process LRU cache) is often enough.

When you scale your backend horizontally, and run many instances behind a load balancer, this no longer works well:

A distributed cache solves this by providing a single logical cache that is shared by multiple application instances, often on multiple servers.

Common distributed cache tools:

ToolTypeTypical Use
RedisIn‑memory storeGeneral caching, sessions, rate limiting
MemcachedCache onlySimple key value caching
HazelcastIn‑memory gridJVM apps, distributed data structures

Distributed caching is almost always built on top of a networked, shared cache server or cluster, not on local memory of each app server.

Important: A distributed cache is not your source of truth. The database or another persistent store remains the system of record, and the cache is a performance optimization only.

Basic Architecture of a Distributed Cache

Single Cache Server with Multiple App Instances

The simplest setup:

ASCII diagram:

text 
          +-------------+
          |  Load       |
          |  Balancer   |
          +------+------+ 
                 |
     +-----------+-----------+
     |           |           |
+----+----+  +---+----+  +---+----+
| App A  |  | App B |  | App C |
+----+----+  +---+----+  +---+----+
     \          |          /
      \         |         /
       \        |        /
        +-------+--------+
        |  Redis / Cache |
        +----------------+

Each app uses the same code to interact with the cache:

Cache Cluster and Sharding

As load grows, a single cache server may not be enough. Two scaling patterns appear:

  1. Replication
    Multiple nodes, each has (roughly) the same data. Focus on availability and read scalability.
  2. Sharding
    Data is split across nodes. Each node holds only a part of the key space.

With sharding, you need to decide which node holds a given key. This is often done with consistent hashing (described later).

Using a Distributed Cache in Your Backend

Typical Cache Access Pattern

The core pattern looks like this:

  1. Read flow
    1. App receives request.
    2. App computes a cache key, for example "user:123".
    3. Check cache:
      • If found, return cached value.
      • If not found, read from database, then write to cache, then return.
  2. Write flow
    1. App writes change to database (source of truth).
    2. App invalidates related cache keys or updates them.

In pseudocode:

python 
def get_user(user_id: int) -> User:
    key = f"user:{user_id}"
    cached = cache.get(key)
    if cached is not None:
        return cached
    user = db.get_user(user_id)
    if user is not None:
        cache.set(key, user, ttl=600)  # 10 minutes
    return user
def update_user(user_id: int, data: dict) -> User:
    user = db.update_user(user_id, data)
    cache.delete(f"user:{user_id}")  # or cache.set(...) with new data
    return user

This pattern by itself is similar to local caching. The "distributed" part is simply that all app instances use the same cache server, so they all see the same cache state.

Consistency and Cache Invalidation

Distributed caching adds new consistency problems. Multiple app servers may:

There is no completely perfect pattern, but there are standard strategies.

Cache Aside (Lazy Loading)

This is the most common pattern.

Read:

  1. Try cache.
  2. On miss, query database, then write to cache.

Write:

  1. Write to database.
  2. Invalidate or update cache entry.

Pros:

Cons:

Write Through

With write through, the cache is treated more like the primary interface:

Write:

  1. Write to cache.
  2. Cache writes to database before acknowledging success.

Read:

Pros:

Cons:

Write Behind (Write Back)

With write behind, writes go to the cache first, and the cache writes to the database asynchronously in the background.

Pros:

Cons:

For most web backends, cache aside is usually the best starting point.

Invalidation Strategies

The hardest part in cache design is invalidation.

Common patterns:

  1. Time‑based expiration (TTL)
    Each key has a time to live. After TTL, the key is removed or treated as expired.
  2. Key‑based invalidation
    When something changes, you explicitly delete or update affected keys.
  3. Pattern‑based invalidation
    Some caches support deletion by pattern, for example user:*. This can be expensive at scale.
  4. Versioned keys
    Part of the key includes a version or timestamp. When data changes, you bump the version, and all new reads use the new key. Old keys eventually expire.

Example of versioned key:

text 
"user:v3:123"

If user data schema or meaning changes, you switch from v2 to v3. Old entries become unused and are cleaned up by expiration.

Designing Cache Keys in a Distributed Cache

Good key design is critical, especially when many services share the same cache.

Namespacing

Use prefixes to keep keys organized:

Type of dataExample key
User objectuser:123
User permissionsuser:123:permissions
Product detailsproduct:987
Category products listcategory:42:products:page:1
Rate limitratelimit:/api/login:ip:1.2.3.4

Rules of thumb:

Rule: A cache key must uniquely represent what is cached. If two different queries use the same key, you will return incorrect data.

Key Length and Memory

Distributed caches often handle many keys. You need a balance:

A typical reasonable length is 20 to 100 characters, depending on your use case.

Replication, Sharding, and Consistent Hashing

As your distributed cache grows, a single node will not be enough.

Replication

With replication, the same data is stored on multiple nodes.

Two typical goals:

Many Redis setups use:

Writes go to primary, reads can be balanced over replicas, with some delay.

This provides eventual consistency: replicas may lag slightly behind primary.

Sharding

With sharding, the key space is split between nodes. Each key belongs to one primary node.

Example with 3 nodes:

In practice, sharding is usually based on a hash of the key, not directly on its numeric portion.

Consistent Hashing (Conceptual View)

Consistent hashing is a technique to decide which node stores a key while minimizing the impact of adding or removing nodes.

Idea:

  1. Map each cache node to a point on a circle based on a hash of its name, for example hash("nodeA").
  2. Map each key to a point on the same circle, for example hash("user:123").
  3. The key is assigned to the first node clockwise from its point on the circle.
  4. When you add or remove a node, only a fraction of keys move to new nodes.

You rarely need to implement this yourself. Many clients and systems provide this logic.

What matters for you as a backend developer:

Dealing with Stale or Incorrect Data

Distributed caches often trade strong consistency for performance and availability. You will occasionally serve stale data.

You need to understand and control:

Staleness Due to Replication Lag

With primary and replica nodes:

  1. App writes data to primary.
  2. Replication to replicas happens asynchronously.
  3. Another app instance reads from a replica before replication happens.

Solution options:

Staleness Due to TTL

If you use TTL, keys can stay in cache after the underlying data changed.

Mitigations:

Race Conditions in Invalidation

Classic problem:

  1. Request A reads from database and writes new value to cache.
  2. Request B writes to database and invalidates cache.
  3. Request A writes stale value to cache, after B updated DB.

Result: stale data in cache.

Typical mitigation:

Coordination and Locks in a Distributed Cache

Sometimes you need coordination across nodes, for example:

Distributed caches like Redis can be used as a distributed lock service.

Simple Lock Pattern

You can use a key as a lock:

  1. Try to set key lock:report:2024-01 with NX (only if not exists) and a short TTL.
  2. If set succeeds, you own the lock and can do the work.
  3. After work, delete the lock key.
  4. If set fails, someone else owns the lock, so you skip or wait.

Pseudocode:

python 
lock_key = "lock:report:2024-01"
got_lock = cache.set(lock_key, "1", nx=True, ex=60)  # 60s TTL
if got_lock:
    try:
        generate_report()
    finally:
        cache.delete(lock_key)
else:
    # another worker is already doing this
    pass

This pattern uses the cache as a coordination mechanism across multiple backend processes or servers.

Using Distributed Caching in a Multi‑Service System

In a microservices or modular architecture, different services may use the same Redis cluster or separate ones.

Shared Cache Cluster

Multiple services using one cache cluster:

Pros:

Cons:

To make this workable:

Per‑Service Cache

Each service has its own cache instance or cluster.

Pros:

Cons:

Both approaches are used in real systems. For a beginner, using one shared Redis with clear key namespaces is a good start.

When to Use Distributed Caching and When Not To

Distributed caching is powerful but introduces extra moving parts and possible bugs.

Good Use Cases

Less Suitable Use Cases

Guideline: If you cannot clearly define:

  1. What data to cache,
  2. How long it may be stale,
  3. How it will be invalidated,
    then you are not ready to add a distributed cache for that data yet.

Practical Example: Caching Product Details with Redis

Consider an e‑commerce backend with multiple API servers.

Cache Aside for Product Details

User requests product details:

  1. App receives GET /products/123.
  2. App checks Redis with key product:123.
  3. If cache hit, return data.
  4. If cache miss:
    • Read product from PostgreSQL.
    • Write to Redis with TTL 300 seconds.
    • Return response.

On admin update:

  1. Admin calls PUT /admin/products/123.
  2. Service updates product in PostgreSQL.
  3. Service deletes Redis key product:123 or writes updated product to cache.

This works across all app servers because:

Caching Product Lists

Lists are trickier. For example, products in category 10, sorted by price, page 2:

Key could be:

text 
"products:category:10:sort:price_asc:page:2"

When a product in category 10 changes price:

You have choices:

  1. Aggressive invalidation
    Invalidate all pages for that category:
text 
   products:category:10:*

This can be expensive since pattern deletes may scan many keys.

  1. Short TTL
    Use a short TTL, for example 60 seconds, and do not manually invalidate.
  2. Hybrid
    Keep TTL but still try to delete the most obviously affected keys, for example first page or a few pages.

In practice, you often start with short TTLs for complex lists and optimize later if needed.

Summary

You have seen how distributed caching extends simple caching to multiple servers:

With this understanding, you can recognize where a distributed cache fits into your backend design and how to use it without breaking correctness.

Views: 8

Comments

Please login to add a comment.

Don't have an account? Register now!

© 2025 Kahibaro
Visitors: 3159 / 24h
Impressum / Imprint
Datenschutz / Data Policy