KAHIBARO
Vertical Scaling
Table of Contents
Understanding Vertical Scaling
Vertical scaling is about making a single server more powerful so it can handle more work. Instead of adding more machines, you upgrade the one you already have.
In this chapter you will see what vertical scaling is, how it compares to horizontal scaling, when it works well, and what limits it in real applications.
What Is Vertical Scaling?
Vertical scaling means increasing the capacity of one node.
Typical ways to scale vertically:
- Add more CPU cores
- Use faster CPUs
- Add more RAM
- Use faster disks (for example move from HDD to SSD, or to faster SSDs)
- Use better network hardware
In cloud environments, vertical scaling usually means:
- Switching from a small instance type to a larger one, for example:
t3.small→t3.largedb.t3.micro→db.m6g.large- Changing configuration of a managed database to use more vCPUs and memory
You still have one application instance or one database instance, only bigger.
Vertical vs Horizontal Scaling
You learn about horizontal scaling in another chapter. Here is the focused comparison:
| Aspect | Vertical scaling | Horizontal scaling |
|---|---|---|
| Basic idea | Make one machine bigger | Add more machines |
| Number of instances | Stays 1 | Grows (2, 3, 10, 100, …) |
| Implementation effort | Usually simple (change instance size) | Requires more architecture work |
| Typical limits | Hardware limit of one machine | Coordination and network complexity |
| Good for | Early growth, databases, simple apps | Large scale, high availability, big traffic |
| Downtime risk | Often requires restart or maintenance window | New nodes can be added with less disruption |
| Cost behavior | Often grows nonlinearly with size | More linear, but adds operational overhead |
Vertical scaling is usually the first step when your application hits resource limits, especially for:
- Databases
- Caches
- Simple monolithic applications
When Vertical Scaling Works Well
Vertical scaling is very effective in several common situations.
1. Early Stage or Simple Architecture
When you have:
- One API server
- One database
- Possibly one cache
you can often handle a lot more load by just:
- Increasing CPU and RAM for the API server
- Upgrading the database instance class
- Moving to SSD if you use spinning disks
Example:
- Your task management API is slow under load
- CPU usage on the server is at 90%, memory is fine
- You move from 2 CPU cores to 4 CPU cores
- The same code now serves more requests per second
No code changes, no new infrastructure components.
2. CPU-Bound Workloads
If your backend spends most time doing CPU work, for example:
- Encoding images
- Encrypting / decrypting data
- Complex calculations
- Heavy JSON serialization
then more CPU and better single-core performance can help a lot.
Example:
- A report generator that takes ~45 seconds on a 2-core server
- On a 4-core server with faster CPU, the same report may complete in ~20 seconds
3. Memory-Bound Workloads
Some workloads are limited by RAM:
- Large in-memory caches
- Big queries that need to sort or join lots of data
- Many concurrent processes or threads
If processes are constantly killed by the OS because of out-of-memory, or your database is swapping to disk, adding RAM can dramatically increase performance.
Example:
- PostgreSQL with 1 GB RAM:
- Frequently hits disk
- Slow queries under load
- Upgrade to 8 GB RAM:
- Working set fits in RAM
- Queries become much faster without any code changes
4. Databases and State
Databases are often the hardest component to scale horizontally. Vertical scaling is a natural first strategy:
- Increase CPU for faster query execution
- Increase RAM so more data stays in buffer cache
- Move to faster disks to reduce I/O latency
For many products databases can run for a long time on a single powerful machine before horizontal scaling is necessary.
Practical Example: Scaling a Simple API
Imagine a FastAPI application running on one server.
Initial server:
- 2 vCPUs
- 4 GB RAM
- Basic SSD
Traffic grows, and you see:
- 95% CPU usage during peak hours
- Response times increase from 100 ms to 800 ms
- Error rate grows because of timeouts
Without changing your code you:
- Upgrade to 4 vCPUs
- Upgrade to 8 GB RAM
After the upgrade:
- CPU usage drops to around 50% at peak
- Response time goes back to 120 ms
- Error rate drops significantly
You did vertical scaling. The process was:
- Observe metrics
- Identify the bottleneck (CPU in this case)
- Increase the relevant resource
- Observe again
This pattern is common early in a project.
How to Decide Whether to Scale Vertically
You should always base scaling decisions on measurements, not guesses.
Useful metrics:
| Component | Metric examples | Vertical scaling signal |
|---|---|---|
| CPU | Average CPU usage, per-core usage | Sustained > 70–80% for long periods |
| Memory | Memory usage, swap usage, OOM errors | Frequent swapping, OOM kills, high page faults |
| Disk | Disk I/O wait, IOPS, disk latency | High I/O wait time, slow reads/writes |
| Network | Network throughput, dropped packets | Constantly at or near NIC limits |
| Database | Buffer cache hit ratio, slow queries | Low cache hit ratio, heavy I/O, CPU-bound queries |
A simple decision process:
- Identify where the bottleneck is
- Check if a bigger instance can reduce that bottleneck
- Estimate cost vs. benefit
- Plan and execute the upgrade, including a rollback plan
Important rule: Always profile and measure first, then scale. Do not scale vertically based on intuition alone.
Benefits of Vertical Scaling
Vertical scaling has several strong advantages:
Simplicity
- No need to add load balancers
- No need for complex stateless architectures
- Fewer moving parts, easier to reason about
Fast to Apply
In many cloud providers:
- Stop instance
- Choose bigger instance type
- Start instance
Managed databases often offer a one-click or automated upscale.
No Code Changes Required
Most of the time, you get benefits just by:
- Increasing CPU and memory
- Ensuring your application uses the hardware correctly (for example right number of worker processes)
Your application architecture can remain the same.
Good for Single-Node Stateful Systems
For:
- Databases
- Message queues
- Caches
vertical scaling avoids the complexity of clustering or sharding for as long as possible.
Limits of Vertical Scaling
Vertical scaling is powerful, but it has hard and soft limits.
Hard Limits
You cannot keep growing a single machine forever.
Examples:
- Maximum vCPUs for an instance type, for example 96 or 128
- Maximum RAM on a physical host
- Maximum network bandwidth per instance
At some point, there is simply no bigger machine.
Diminishing Returns
Even before you hit hard limits, returns can become worse.
Reasons:
- Contention on shared resources inside the machine
- Operating system overhead
- Some workloads do not parallelize well
For example:
- Adding CPUs might not help if your code is mostly single-threaded
- Adding RAM might not help if your working set already fits in memory
Downtime Risk
Changing instance size often requires:
- Reboot
- Short downtime for cutover to a new instance
For production systems, this means:
- Maintenance windows
- Careful preparation
- Potential customer impact
You can reduce downtime with techniques like blue/green deployments, but that is more complex.
Cost Behavior
Vertical scaling often has non-linear cost:
- 2x bigger instance may cost more than 2x the price
- Very large instances can be disproportionately expensive
Example:
| Instance type | vCPUs | RAM | Price (relative) |
|---|---|---|---|
| Small | 2 | 4GB | 1x |
| Medium | 4 | 8GB | 2.4x |
| Large | 8 | 16GB | 5.5x |
So 4x capacity might cost more than 4x money.
Single Point of Failure
One big machine is still one machine.
If that machine:
- Crashes
- Has hardware failure
- Has a network issue
your whole system goes down.
Horizontal scaling helps with high availability. Vertical scaling alone cannot.
Important statement: Vertical scaling alone cannot give high availability. You still have a single point of failure.
Vertical Scaling and Performance Types
You learned about CPU-bound and I/O-bound work in another chapter. Vertical scaling interacts with them in different ways.
CPU-Bound Work
Vertical scaling helps by:
- More CPU cores
- Faster single-core performance
Example:
- A synchronous image processing endpoint
- Each request uses 1 core fully for 500 ms
- With 2 cores, you can handle ~4 such requests per second
- With 8 cores, you can handle ~16 such requests per second (in ideal conditions)
If your application server can use multiple processes or threads, more cores usually mean more throughput.
I/O-Bound Work
Vertical scaling helps mostly by:
- Faster disks for I/O-bound databases or file systems
- More RAM for caching data and reducing disk access
- Better network interface for network-heavy workloads
Example:
- PostgreSQL doing many disk reads
- On slow HDD, each disk read is slow
- Moving to SSD can reduce read latency by a factor of 10 or more
Here, even if CPU is not the limit, better hardware makes each I/O operation faster.
How to Scale Vertically in Practice
Even though every provider is different, there are common patterns.
Scaling an Application Server
Steps usually include:
- Check metrics
- CPU, memory, load average, response times
- Choose a bigger instance
- More vCPUs
- More RAM
- Plan the upgrade
- Off-peak time if you expect downtime
- Backup important data (logs, configurations)
- Upgrade
- Resize instance or create a new bigger one and move traffic
- Verify
- Check metrics after upgrade
- Run smoke tests on critical endpoints
You might also adjust application settings:
- Number of worker processes for Gunicorn / Uvicorn
- Database connection pool size
- Thread pools in background workers
Scaling a Database Vertically
Typical steps:
- Analyze database performance
- Slow queries
- Buffer cache hit ratio
- CPU and disk usage
- Try optimizations first
- Indexes
- Query optimization
- Configuration tuning
- If still constrained, increase instance size
- Monitor after upgrade
- Verify that performance actually improved
It is common to:
- Combine vertical scaling with index and query improvements
- Use more RAM to allow bigger caches
Combining Vertical and Horizontal Scaling
Vertical and horizontal scaling are not enemies. You often use both over time.
Typical growth path:
- Small instance, simple app
- Vertically scale:
- Bigger app server
- Bigger database instance
- When vertical scaling becomes too expensive or reaches limits:
- Add more app servers behind a load balancer
- Use read replicas for the database
- Move some responsibilities to other services
Vertical scaling often comes first because it is:
- Cheaper to implement at the engineering level
- Faster to apply
- Good enough for many use cases
Later, you add horizontal scaling for:
- High availability
- Better fault tolerance
- Very high traffic
Rules of Thumb for Vertical Scaling
You can treat these as practical guidelines.
Rules of thumb:
- Use vertical scaling first for simple systems until costs or technical limits make it unreasonable.
- Always measure bottlenecks before scaling, and validate after scaling.
- Do not rely on vertical scaling for high availability, use redundancy and horizontal strategies for that.
- Optimize software and queries before or alongside buying bigger hardware.
Applied examples:
- If CPU is always at 90% and response times are bad, increase CPU or number of cores.
- If memory is always full and the system swaps or kills processes, add RAM.
- If database I/O wait is high, move from HDD to SSD or to faster SSDs and consider more RAM.
- If a bigger instance is significantly more expensive than adding another small instance plus a load balancer, consider transitioning to horizontal scaling.
Summary
Vertical scaling is about making your existing server more powerful by upgrading CPU, memory, disk, or network. It is:
- Very simple to implement
- Great for early stages and for databases
- Limited by hardware maximums, cost growth, and single point of failure
As a backend developer you should:
- Recognize when vertical scaling is appropriate
- Know how to read basic resource metrics
- Plan and execute vertical upgrades safely
- Understand that at some scale you will need horizontal strategies too
You will see how vertical scaling compares to horizontal scaling and how to design for large scale in the surrounding chapters on performance and scalability.
Views: 6