KAHIBARO
Discord Login Register ☰
☰

26.2. CPU-Bound vs I/O-Bound Work

Table of Contents

Understanding CPU-Bound vs I/O-Bound Work

When you design and optimize backends, you must know what kind of work your application is doing. Almost every performance decision, from choosing async to adding more CPUs, depends on whether your workload is CPU-bound or I/O-bound.

This chapter explains the difference, shows how it impacts Python and web backends, and gives you practical patterns and code examples.

Key idea:
To improve performance, first identify if your code is limited by CPU speed (CPU-bound) or by waiting for input/output (I/O-bound).
Optimizing the wrong thing can waste time and even make performance worse.

Basic Definitions

What is CPU-bound work?

CPU-bound work is work where the main bottleneck is the processor doing calculations.

Examples:

In CPU-bound code, you usually have:

What is I/O-bound work?

I/O-bound work is work where the main bottleneck is waiting on external systems.

"I/O" stands for Input/Output, like:

In I/O-bound code, you usually have:

Comparing CPU-bound and I/O-bound

AspectCPU-boundI/O-bound
Main bottleneckCPU speed / calculationsWaiting for external resources
Typical usage80–100% CPU during processingLow to medium CPU, but long waits
ExamplesImage resize, encryption, math-heavy APIDB queries, HTTP calls, file uploads/downloads
Good solutionsMore CPUs, multiprocessing, optimize codeAsync I/O, concurrency, caching, faster I/O systems
Python threads help?Usually no (GIL limits)Yes, can overlap waiting time
Async/await helps?No, CPU still the limitYes, can handle many concurrent I/O tasks efficiently

How to Recognize CPU-Bound vs I/O-Bound in Backends

You rarely see "pure" CPU-bound or "pure" I/O-bound systems. Usually, your endpoint does some combination.

A simple HTTP request in a backend might:

  1. Parse JSON from the client (CPU)
  2. Validate data (some CPU)
  3. Query the database (I/O)
  4. Process the result (CPU)
  5. Call a third-party API (I/O)
  6. Format the response (CPU)

You want to know what dominates the total time.

Rough mental checks

Ask these questions about a slow endpoint:

  1. Does CPU usage spike to 100% for your process?
    • Yes, and stays high during requests: probably CPU-bound.
    • No, CPU is low but requests are slow: probably I/O-bound.
  2. What happens if you run more concurrent requests?
    • If latency gets much worse, and CPU is maxed out: CPU-bound.
    • If latency grows slowly and CPU is low, but DB or network is busy: I/O-bound.
  3. If you optimize queries or network calls, do things improve a lot?
    • Yes: likely I/O-bound.
    • No, but optimizing CPU-heavy functions helps: likely CPU-bound.

Simple Python Examples

A CPU-bound example

This function calculates many square roots. It uses CPU but no external I/O.

python 
import math
def cpu_bound_task(n: int) -> float:
    result = 0.0
    for i in range(1, n):
        result += math.sqrt(i)
    return result
result = cpu_bound_task(10_000_000)

Characteristics:

An I/O-bound example

This function makes HTTP requests to another service.

python 
import requests
def io_bound_task(urls):
    results = []
    for url in urls:
        response = requests.get(url)  # Network I/O
        results.append(response.text)
    return results
urls = ["https://example.com"] * 10
data = io_bound_task(urls)

Characteristics:

Concurrency and the Python GIL

Python has a Global Interpreter Lock (GIL) that affects how CPU-bound and I/O-bound tasks behave with threads.

How threads behave

Important rule:

  • Use threads or async for I/O-bound tasks, to overlap waiting time.
  • Use multiprocessing or multiple processes for CPU-bound tasks, to use multiple CPU cores.

I/O-Bound Work and Async in Backends

Modern Python backends, like FastAPI, are often built around async I/O to handle many concurrent clients.

I/O-bound example with async

Using httpx and asyncio to fetch many URLs:

python 
import asyncio
import httpx
async def fetch(client: httpx.AsyncClient, url: str) -> str:
    response = await client.get(url)  # I/O, does not block event loop
    return response.text
async def main(urls):
    async with httpx.AsyncClient() as client:
        tasks = [fetch(client, url) for url in urls]
        results = await asyncio.gather(*tasks)
    return results
urls = ["https://example.com"] * 10
asyncio.run(main(urls))

Here:

This is ideal for:

CPU-Bound Work and Multiprocessing

Async and threads do not fix CPU limits. For CPU-heavy tasks you need multiple processes or external workers.

CPU-bound example with multiprocessing

python 
import math
from multiprocessing import Pool, cpu_count
def cpu_bound_task(n: int) -> float:
    result = 0.0
    for i in range(1, n):
        result += math.sqrt(i)
    return result
if __name__ == "__main__":
    n = 10_000_000
    tasks = [n] * cpu_count()
    with Pool() as pool:
        results = pool.map(cpu_bound_task, tasks)
    print(sum(results))

Here:

Web Backend Examples

Example 1: Pure I/O-bound endpoint

python 
# FastAPI-like pseudocode
from fastapi import FastAPI
import httpx
app = FastAPI()
@app.get("/weather")
async def get_weather(city: str):
    async with httpx.AsyncClient() as client:
        r = await client.get(
            "https://api.weather.example.com",
            params={"city": city}
        )
    data = r.json()
    # small CPU work to format result
    return {"city": city, "temp": data["temperature"]}

Characteristics:

Example 2: Mixed CPU-bound and I/O-bound

python 
from fastapi import FastAPI
import httpx
import hashlib
app = FastAPI()
def heavy_hash(data: bytes, rounds: int = 100_000) -> str:
    digest = data
    for _ in range(rounds):
        digest = hashlib.sha256(digest).digest()
    return digest.hex()
@app.post("/process")
async def process_file(file_url: str):
    async with httpx.AsyncClient() as client:
        r = await client.get(file_url)  # I/O-bound part
    content = r.content
    hash_value = heavy_hash(content)   # CPU-bound part
    return {"hash": hash_value}

Here:

Choosing the Right Tool

Summary table of strategies

Workload typeGood strategiesLess effective strategies
I/O-boundAsync I/O, threads, faster network/disk, cachingMultiprocessing alone, more CPU cores only
CPU-boundMultiprocessing, more CPU cores, algorithm tuningAsync/await alone, many threads in CPython
Mixed (most APIs)Async for I/O, offload CPU-heavy parts to workersDoing everything in a single request thread

Offloading CPU work in backends

For CPU-heavy operations in a web API:

Pattern:

  1. API receives request
  2. API enqueues a CPU-heavy job to a worker
  3. API responds quickly with a job ID
  4. Client checks job status or uses webhooks

This turns a CPU-bound synchronous endpoint into an I/O-bound flow, where the main service mostly deals with I/O: queue messages, job status, etc.

Measuring and Confirming the Bottleneck

Do not guess. You should measure.

Common approaches:

If:

Practical rule:
Always measure where the time goes, then label your main bottleneck as CPU-bound or I/O-bound.
Only then choose between:

  • Async/threads and I/O optimizations
  • Or multiprocessing/CPU scaling and algorithm optimizations.

How This Connects to Performance and Scalability

Later topics in performance and scalability, like:

all depend on this distinction.

In practice:

Understanding CPU-bound vs I/O-bound work is the foundation for choosing the right performance strategy for your backend.

Views: 6

Comments

Please login to add a comment.

Don't have an account? Register now!

© 2025 Kahibaro
Visitors: 3505 / 24h
Impressum / Imprint
Datenschutz / Data Policy