KAHIBARO
16.1. Why Caching Matters
Table of Contents
Understanding Why Caching Matters
Caching is one of the most powerful tools you have as a backend developer. It can make your application feel instant, reduce server costs, and keep your system alive under heavy load. To use it well, you first need to understand why it matters so much.
In this chapter you will not learn how to implement caching with specific tools. Instead, you will learn the reasons, effects, and trade‑offs that make caching such an important part of backend systems.
The Core Idea of Caching
At its simplest, caching means:
- Remember the result of some work
- Store it somewhere fast
- Reuse it next time instead of doing the work again
That work might be a database query, a complex calculation, or even a call to another remote API.
Example, without caching:
- User opens
/home - Server loads top 10 products from database
- Server renders the homepage
- Next user opens
/home, the server does everything again
With caching:
- First user opens
/home - Server loads top 10 products from database and stores the result in a cache
- Next user opens
/home, server reads top 10 products directly from the cache, skipping the database
The key benefit is that expensive work is done less often.
Performance Benefits
Caching is one of the easiest ways to achieve huge performance gains.
Time: Faster Responses
Accessing data from a cache is usually much faster than recomputing or refetching it.
Approximate time costs:
| Operation | Typical Time Scale |
|---|---|
| CPU operation (in-memory) | Nanoseconds to microseconds |
| Read from in-memory cache (Redis) | Microseconds |
| Read from local disk | Milliseconds |
| Database query over network | Several milliseconds |
| Remote API call | Tens to hundreds of ms |
If your API currently needs 50 ms for a database query and 30 ms for business logic, your response time might be around 80 ms. If you can cache the result and serve it in 3 ms, the same endpoint becomes more than 25 times faster.
Key rule: Cache data that is expensive to compute or fetch but cheap to store and does not change constantly.
Example: Cached Product List
Imagine an endpoint /products/popular:
- Database query takes 40 ms
- Formatting and building the JSON takes 10 ms
Total: about 50 ms per request.
If 1000 users call this endpoint in a minute:
- Without cache: 1000 queries × 40 ms = 40,000 ms of database time
- With cache: 1 query × 40 ms (first request) + 999 cache reads × 2 ms ≈ 40 + 1998 ≈ 2,038 ms
You cut the database time from 40 seconds to about 2 seconds per minute.
Scalability: Handling More Users
When your app gets more users, your database becomes a bottleneck. Caching can offload a lot of repetitive work.
Imagine you have an API that is mostly read heavy, like:
- Product details
- Blog posts
- Public user profiles
These do not change every second. With caching:
- Most read requests never touch the database
- The database is free to handle writes and complex queries
This effectively increases the number of requests your system can handle with the same hardware.
Example: Read-Heavy vs Write-Heavy
| Scenario | Description | Cache Helps? |
|---|---|---|
| Product catalog | Many reads, occasional updates | Yes, very effective |
| Social feed | Many reads, frequent new data | Yes, but more complex |
| Stock trading data | Constantly changing prices | Limited use |
| User login | Writes and security sensitive validation | Usually not cached |
Caching is most valuable in read-heavy use cases with moderately changing data.
Cost and Resource Savings
Caching does not only make things faster, it also makes them cheaper.
Reduced Database Load
Databases are expensive:
- They require strong hardware
- Scaling them vertically (more CPU, RAM) is costly
- Scaling them horizontally (sharding, replicas) is complex
By serving many read requests from cache, you:
- Reduce CPU load on the database
- Reduce disk I/O
- Reduce network traffic to the DB
This often allows you to:
- Use a smaller database server
- Delay the moment you need to scale
- Stay within cheaper hosting tiers
Cheaper External API Usage
If your application talks to paid third party APIs (for example, weather, payments, maps), each call might cost money or be limited by rate.
Caching lets you:
- Call the external API once
- Reuse the result for many users
Example: Weather API
You show current weather for a city:
- External API call costs money and rate limited to 1000 requests per day
- You have 10,000 users per day asking for the same city
Without caching: You would hit rate limits or pay a lot
With caching:
- Fetch weather every 5 minutes
- Store the result in cache
- Serve 10,000 user requests from the cache
You reduced 10,000 calls down to about 288 calls per day (12 per hour × 24 hours).
User Experience Benefits
Caching directly affects how users feel your application.
Perceived Speed
Users do not measure exact response times, they feel delays. If something responds instantly, they feel the app is "snappy." Caching helps many endpoints respond in a few milliseconds.
Examples:
- Cached homepage loads almost immediately
- Product pages appear quickly while images load
- Autocomplete suggestions appear as you type because results are cached
Smoother Load Spikes
Sometimes traffic suddenly increases:
- A product goes viral
- A blog post hits the front page of a news site
- A marketing email sends many users at once
Without caching, your database might get overwhelmed, cause timeouts, or even crash.
With caching:
- Many users request the same pages or data
- You compute it once and serve it many times
- Your system is more likely to survive traffic spikes
This improved stability is very important for a good user experience.
Reliability and Graceful Degradation
Caching can help your application behave more gracefully when other components fail.
Surviving Dependencies Going Down
Imagine:
- Your database becomes temporarily unavailable
- A third party API is down for a few minutes
If you have cached data:
- Some endpoints can still serve responses using cache
- Users might see slightly older data instead of an error
Example:
- A product listing page shows inventory counts
- Database is down for 1 minute
- You serve last known product list from cache instead of returning a 500 error
This is called graceful degradation. The app still works in a limited way.
Fallback Strategies
You can design logic like:
- Try to get fresh data from the source
- If it fails, serve cached data
- If cache is also empty, return an error
This layered approach improves your system reliability.
Where Caching Is Especially Valuable
Caching can exist at many levels, which you will explore in later chapters. For now, focus on what kind of data and operations benefit most.
Expensive Computations
Anything that is CPU heavy or complex to compute is a candidate.
Examples:
- Generating a PDF report
- Running complex analytics queries
- Rendering a large HTML page with many components
If the input parameters are the same and the output does not need to be updated all the time, caching can save a lot of CPU.
Frequently Accessed Data (Hot Data)
Some data is "hot" because many users ask for it often.
Examples:
- Homepage sections (featured products, banners)
- Top 10 lists (most liked posts, trending items)
- Configuration that many services read
If you can identify these "hot paths," caching them can give huge benefits.
Slow External Systems
If some part of your system depends on something slow, caching is very helpful.
Examples:
- Slow database queries
- Remote APIs with high latency
- Services in another region
By caching results, you hide this latency from users.
Trade‑offs and Risks
Caching is not free. It introduces complexity and possible problems. You should understand these before using caching everywhere.
Stale Data
The biggest risk is serving outdated information.
If you cache a value and the real data changes, you might still serve the old version until the cache expires or is updated.
Example problems:
- Showing an already sold out product as "in stock"
- Displaying old prices
- Showing outdated user information
How bad this is depends on your use case.
| Use Case | Can Tolerate Stale Data? | Reason |
|---|---|---|
| Product descriptions | Yes, usually | A short delay is acceptable |
| News homepage | Yes, short delays | Users expect near recent, not always instant |
| Bank account balance | No | Must be very accurate |
| Real time stock trading | No | Even small delay can cause losses |
Important: Do not cache data that must be perfectly up to date, unless you have a very careful strategy for invalidating or updating the cache.
Cache Invalidation Complexity
"Cache invalidation" means deciding when to remove or update cached data.
Two common strategies:
- Time based expiration (TTL, time to live)
- Event based invalidation (remove cache when data changes)
Both bring complexity:
- Too short TTL: less benefit, cache misses often
- Too long TTL: more stale data
- Event based: you must remember to clear or update all relevant cache entries whenever something changes
There is a famous saying:
There are only two hard things in Computer Science: cache invalidation and naming things.
You will see more on invalidation in a later chapter, but you should already know it is one of the hardest parts of caching.
Extra Memory Usage
Caches use memory or disk space.
- In memory caches (like Redis) are fast, but memory is limited and expensive
- If you store too much, you might evict useful entries or run out of memory
You must decide:
- What to cache
- How much to cache
- What to remove when the cache is full
Risk of Hidden Bugs
If you introduce caching without fully understanding your data logic, you can:
- Cache incorrect results
- Forget to clear cache after certain updates
- Debug issues that occur only when cache is warm
For example:
- You change a business rule but forget to update how cache keys are built
- Old cached values still follow old rules
- Some users see old behavior, others see new behavior
This can be tricky to debug, so logging and monitoring become important.
Caching and System Design
As you learn more about backend architecture, you will see that caching is part of a bigger picture.
Layered Caching
Caching can exist at many layers:
| Layer | Example | Purpose |
|---|---|---|
| Browser cache | Static images, CSS, JS | Avoid new network requests |
| CDN cache | Images, static pages, sometimes HTML | Serve content from edge locations |
| Reverse proxy cache | Nginx caching responses | Offload web/app servers |
| Application cache | In app memory, Redis for computed results | Speed up specific logic and queries |
| Database cache | DB internal page cache, query result cache | Faster DB queries |
Each layer can reduce load for the layer behind it.
You do not need to implement all of them at once. Start simple:
- Cache at the application level for expensive parts
- Later, add HTTP level caching and CDNs as needed
Caching as a Scaling Strategy
When people talk about scaling systems, caching is usually one of the first tools they consider, often before more complex changes like:
- Sharding databases
- Introducing microservices
- Heavy database tuning
Caching is attractive because:
- It can give big improvements with relatively small code changes
- It often works with your existing architecture
When Not to Cache
Knowing when not to use caching is as important as knowing when to use it.
You should avoid caching:
- Highly dynamic, user specific data
Example: Real time chat messages for a single user. Caching might give almost no benefit because each request is unique and messages change constantly. - Sensitive data that must be accurate
Example: Bank balances, current order payment status. A small delay or stale value can cause serious problems. - Very cheap operations
If a computation or query is already extremely fast and not called often, caching might add unnecessary complexity. - Data with strict consistency requirements
If your business logic requires all users to see the same, current value at all times, caching introduces risk.
Thinking in Terms of Trade‑offs
Every caching decision is a trade‑off between:
- Freshness of data
- Performance
- Complexity
You can think of it like this:
- If you need perfect freshness, you usually sacrifice performance
- If you want maximum performance, you accept small delays in freshness
- If you want both, you need more complex logic
A simple mental framework:
- Ask: How often does this data change?
- Ask: How often do users request this data?
- Ask: How bad is it if users see data that is 10 seconds, 1 minute, or 5 minutes old?
- Ask: How slow or expensive is it to fetch or compute this data?
If:
- Data changes rarely
- Data is requested often
- Slightly stale data is acceptable
- Computation or fetch is slow or expensive
then caching is almost always a good idea.
Summary
Caching matters because it:
- Greatly improves performance by avoiding repeated work
- Helps scalability by reducing load on your database and services
- Cuts costs by lowering resource usage and external API calls
- Improves user experience with faster and more stable responses
- Increases reliability by allowing graceful degradation when dependencies fail
But it also:
- Introduces risks of stale data
- Requires careful cache invalidation
- Consumes memory and storage
- Adds complexity to your code and system design
In the next chapters you will learn specific types of caching, technical tools like Redis, and concrete patterns for cache keys, expiration, and invalidation. Keep these motivations in mind, because understanding why caching matters will help you make smart decisions about what and how to cache in real backend systems.
Views: 8