KAHIBARO
Discord Login Register ☰
☰

16.2 Application Caching

Table of Contents

Why Application Caching Matters

Application caching is the technique of storing the result of expensive work so that you can reuse it instead of doing the work again.

In backend development this usually means:

Caching can drastically reduce:

and can often delay or avoid the need for expensive infrastructure scaling.

Key idea: Cache what is expensive to do and cheap to store.

Example situation:

In this case it is very effective to compute the report once, cache the result for 10 minutes, and serve the cached result for all other requests during that time.

Types of Application Caches

Application caching happens inside or very close to your backend application. Common types:

TypeWhere storedTypical scope
In‑process / in‑memoryInside app process RAMSingle instance
Shared cache (Redis, etc.)Separate cache serviceMultiple instances / whole cluster
Local on-disk cacheLocal disk / filesystemSingle server

HTTP caching is covered separately in the HTTP Caching chapter. Here we focus on what your application code controls directly.

In‑Process / In‑Memory Cache

This is the simplest form of application caching.

Example in Python:

python 
product_cache = {}  # simple in‑memory cache
def get_product(product_id: int):
    if product_id in product_cache:
        return product_cache[product_id]  # cached value
    product = query_product_from_database(product_id)  # expensive
    product_cache[product_id] = product
    return product

This is fast and easy, but it does not work well when you have many application instances, because each instance has its own copy of the cache.

Shared Cache (Using Redis or Similar)

A shared cache is a separate service, such as Redis, that all your application instances talk to.

Example pattern:

text 
Client -> Your app -> Redis (cache) -> Database
  1. App checks Redis for cached value
  2. If found, use that value
  3. If not found, query the database, store result in Redis, and return it

Shared caches are the most common choice in production systems.

What to Cache in Your Application

You should not cache everything. Cache the things that:

Common examples:

WhatWhy cache it?
Product catalogMany reads, few writes
User profiles (public data)Often shown, not updated on every request
Configuration / feature flagsRarely change, used in many places
Computed statisticsExpensive aggregation queries
HTML templates after renderingFor non personal content

Avoid Caching Highly Dynamic or Personal Data

Data that changes very often or is specific to each user is usually a poor cache candidate, unless you control it carefully.

Examples:

You might still cache parts of such data, for example, the list of products that is the same for everyone, and then add per user data on top of it without caching.

Cache-aside Pattern (The Most Common Pattern)

The most used pattern in application caching is called cache‑aside.

The algorithm is:

  1. Check the cache for the value
  2. If it exists, return it
  3. If it does not exist, fetch from the original source (usually database)
  4. Store the new value in the cache
  5. Return the value

Cache-aside rule:
Application reads first from cache. On miss, read from database and then write to cache.

Example of Cache-aside in Python

Imagine you want to cache user profiles by user id.

python 
import json
from typing import Optional
import redis
r = redis.Redis(host="localhost", port=6379, db=0)
def get_user_from_db(user_id: int) -> dict:
    # very simplified example
    # in a real app you'd use an ORM like SQLAlchemy
    return {"id": user_id, "name": "Alice", "age": 30}
def get_user(user_id: int) -> Optional[dict]:
    cache_key = f"user:{user_id}"
    cached = r.get(cache_key)
    if cached is not None:
        return json.loads(cached)
    # cache miss
    user = get_user_from_db(user_id)
    if user is None:
        return None
    # write to cache with expiration of 1 hour (3600 seconds)
    r.setex(cache_key, 3600, json.dumps(user))
    return user

This code:

Time-to-Live (TTL) and Expiration

Every cache entry should have a lifetime. This is how long the entry is considered valid.

TTL is usually expressed in seconds, for example:

Important rule:
Never rely on the cache as the only source of truth. Always be ready to reload from the database or original source when cache entries expire or are missing.

Example: Setting a TTL in Redis

python 
# cache for 10 minutes
r.setex("config:homepage", 600, json.dumps(config))

Choosing TTL Values

Use these rough guidelines:

Data typeSuggested TTL
Feature flags, configs that rarely change10 minutes to 24 hours
Product catalog5 to 60 minutes
Home page blocks / marketing banners1 to 15 minutes
Dashboard statistics30 seconds to 5 minutes
OAuth tokensSame as token lifetime

TTL choice is always a trade‑off between performance and freshness.

Cache Keys and Namespacing

A cache key is a string that identifies one cached value.

Typical structure:

text 
<namespace>:<resource_type>:<identifier>

Examples:

Why Namespaces Matter

Namespaces make it easy to:

For example, if all your product related keys start with product:, you can delete them using a pattern when product data changes.

Cache Invalidation Basics

Cache invalidation is the process of removing or updating cached data when the underlying data changes.

It is the hardest part of caching, because:

There are three basic strategies:

  1. TTL based
    Let the cache expire automatically after a certain time.
  2. Write-through / write-behind
    Update both cache and database when data changes.
  3. Explicit invalidation
    When updating data in the database, also delete or update related cache keys.

Example: Explicit Invalidation

Imagine you cache product:details:<id> and you have an API to update products.

python 
def update_product(product_id: int, data: dict) -> dict:
    updated = update_product_in_db(product_id, data)
    # Invalidate related cache
    cache_key = f"product:details:{product_id}"
    r.delete(cache_key)
    return updated

Next time a client asks for the product, the cache will miss, and your app will fetch the fresh data from the database and store it again in the cache.

Cache Hit and Cache Miss

Two important terms:

You want a high hit rate, but not necessarily 100 percent. Sometimes it is fine that some requests go to the database, as long as your system stays fast and stable.

You can calculate hit rate like this:

$$
\text{hit\_rate} = \frac{\text{hits}}{\text{hits} + \text{misses}}
$$

Remember:
Caching only helps when the hit rate is high enough to reduce the real load on your database or service.

Function-level Caching (Memoization)

You can cache the results of function calls based on their arguments. This pattern is called memoization.

A simple example in Python using functools.lru_cache:

python 
from functools import lru_cache
@lru_cache(maxsize=1024)
def fibonacci(n: int) -> int:
    if n < 2:
        return n
    return fibonacci(n - 1) + fibonacci(n - 2)

This is very helpful for:

In web backends, this is often used for:

Caching Database Queries

Many backend bottlenecks come from heavy or frequently used database queries.

A common pattern is:

  1. Build a cache key from the query parameters
  2. Check the cache before running the query
  3. Cache the result if it is not in the cache

Example: Caching a List of Products

Imagine an endpoint /products?category=phone&page=1.

You might build the key like this:

python 
def build_product_list_key(category: str, page: int, page_size: int) -> str:
    return f"product:list:category={category}:page={page}:size={page_size}"

And then:

python 
def list_products(category: str, page: int, page_size: int = 20):
    cache_key = build_product_list_key(category, page, page_size)
    cached = r.get(cache_key)
    if cached:
        return json.loads(cached)
    # expensive DB query
    products = query_products_from_db(category, page, page_size)
    r.setex(cache_key, 60, json.dumps(products))  # cache for 1 minute
    return products

Notice that the function includes all relevant parameters in the cache key. This is very important, otherwise you might return results for the wrong query.

Caching Partial Responses

Sometimes you do not want to cache an entire API response, because it depends on a mix of static and dynamic data.

For example, an article page might include:

A good approach is:

This gives you a large performance win while keeping dynamic parts fresh.

Dealing With Stale Data

Cached data is almost always slightly out of date. You must decide how much staleness is acceptable.

Options:

  1. Strong freshness, always fetch from the source when something changes, and immediately update or invalidate the cache.
  2. Eventual consistency, allow cached data to be a bit old for a defined period, like 1 or 5 minutes.

For many use cases, eventual consistency is fine. For example:

For very sensitive data, like balances in a banking system, you might avoid caching entirely or use very careful designs that combine transactions and cache updates.

Cache Warmup and Cold Starts

When your application or cache server starts and the cache is empty, you have a cold cache. This can cause a spike in load on your database, because every request is a cache miss at first.

To avoid this, sometimes you:

Example warmup script:

python 
def warmup_cache():
    # preload popular products
    for product_id in [1, 2, 3, 4, 5]:
        product = get_product_from_db(product_id)
        r.setex(f"product:details:{product_id}", 3600, json.dumps(product))

When Not to Use Application Caching

Caching is powerful, but not always appropriate.

Avoid or be very careful with caching when:

In many small projects, you can start without caching, measure performance, and add caching only where it clearly helps.

Practical Guidelines for Application Caching

To use application caching effectively, keep these principles in mind:

  1. Start with measurement
    Find slow endpoints or high load queries before adding caches.
  2. Cache only what matters
    Focus on the top 10 to 20 percent of operations that cause most of the load.
  3. Choose clear cache keys
    Use namespaces and include all relevant parameters in the key.
  4. Always set an expiration
    Do not store data in the cache forever, unless you know exactly why.
  5. Plan invalidation
    Decide how and when to clear or update caches when data changes.
  6. Monitor hit rate
    Track how often cache is used. If hit rate is low, your cache design might not be effective.
  7. Keep the database the source of truth
    Treat cache as a helper, not a replacement for your primary storage.

By following these guidelines, you will be able to add application caching to your backend in a safe and effective way, and you will avoid many common pitfalls such as stale data, inconsistent behavior, or wasted complexity.

Views: 7

Comments

Please login to add a comment.

Don't have an account? Register now!

© 2025 Kahibaro
Visitors: 3133 / 24h
Impressum / Imprint
Datenschutz / Data Policy