KAHIBARO
Discord Login Register ☰
☰

27.6. API Rate Limiting

Table of Contents

Why API Rate Limiting Matters

Modern backend systems often serve thousands or millions of requests per day. Without any control, a single buggy client or a malicious attacker could:

Rate limiting is how you control how many requests a client can make in a given time window. It is a key part of API reliability, cost control, and security.

Core idea of rate limiting

Limit the number of allowed actions (for example, requests) per subject (for example, user, IP, API key) in a given time window to protect your system.

Common goals of rate limiting:

Typical rules look like:

What Exactly Is Being Limited?

You must first decide what you are limiting and for whom.

What you limit

Examples:

You do not always rate limit every endpoint equally. Some endpoints are more expensive:

It is common to have different limits per endpoint or endpoint group.

Who you limit (the key)

The “subject” that you count against is often called the rate limit key.

Typical keys:

Key typeExample valueWhen to use
IP address203.0.113.10Public APIs without auth, anonymous users
User IDuser_123Authenticated APIs
API key / tokensk_live_abc...Third‑party client apps or partners
Device IDdevice_abc123Mobile apps, IoT devices
Account/org IDcompany_42Multi‑tenant SaaS, team‑level limits

You can also combine them:

Important rule

Always choose a rate limit key that matches your security and fairness goals. Do not rely on IP only if you really care about user‑level abuse.

Types of Rate Limiting Rules

Per second, minute, hour, day

The most common style: “X requests per Y time”.

Examples:

Short windows handle bursts. Long windows handle overall quotas.

You can combine them:

Per endpoint or operation

Examples:

Login attempts are a good example: you might allow very few attempts to reduce brute force risk.

Per plan or tier

In paid APIs, rate limits often depend on the subscription plan:

PlanLimit
Free1000 requests per day
Pro10,000 requests per day
Business100 requests per second, 1M per day

The plan is part of the rate limit key, for example: plan:pro:user_123.

Basic Rate Limiting Algorithms

Internally, rate limiting is done by algorithms that count events in time.

1. Fixed window counter

You choose a window like 1 minute and a max count like 100. For each key you store a counter per window.

Example logic:

  1. Compute the current window: for example, window_id = floor(current_timestamp / 60) for 1‑minute windows.
  2. Counter key: rate:{user_id}:{window_id}
  3. Increment the counter on each request.
  4. If counter > limit, reject.

Simple example in pseudocode:

python 
def allow_request(key: str, limit: int, window_seconds: int) -> bool:
    window_id = int(time.time()) // window_seconds
    counter_key = f"rate:{key}:{window_id}"
    count = increment_counter(counter_key, expire=window_seconds)
    return count <= limit

Pros:

Cons:

This “burst at the window boundary” can be a problem for strict systems.

2. Sliding window log

You store timestamps of each request and count only those within the last X seconds.

Algorithm idea:

  1. For each request, store timestamp in a list for the key.
  2. Remove timestamps older than window_seconds.
  3. If list length > limit, reject.

Pseudocode:

python 
def allow_request(key: str, limit: int, window_seconds: int) -> bool:
    now = time.time()
    timestamps = get_timestamps(key)
    timestamps = [t for t in timestamps if now - t <= window_seconds]
    if len(timestamps) >= limit:
        return False
    timestamps.append(now)
    save_timestamps(key, timestamps)
    return True

Pros:

Cons:

In practice, Redis sorted sets (ZADD, ZREMRANGEBYSCORE, ZCARD) can help implement this efficiently.

3. Sliding window counter / hybrid

A compromise between fixed window counter and full logs.

Idea:

This smooths out boundary spikes without storing all timestamps.

You might see it called “sliding window” in many libraries.

4. Token bucket

Very popular, especially for APIs and network traffic shaping.

Imagine a bucket that:

If the bucket has tokens, the request is allowed and a token is removed. If it is empty, the request is rejected (or delayed).

Properties:

Pseudocode:

python 
def allow_request(key: str, capacity: int, refill_rate: float) -> bool:
    state = load_state(key)  # {tokens, last_refill_timestamp}
    now = time.time()
    # Refill tokens based on elapsed time
    elapsed = now - state.last_refill_timestamp
    refill = elapsed * refill_rate
    tokens = min(capacity, state.tokens + refill)
    if tokens < 1:
        # not enough tokens
        save_state(key, tokens, now)
        return False
    tokens -= 1
    save_state(key, tokens, now)
    return True

Pros:

Cons:

5. Leaky bucket

Very similar conceptually to token bucket, but described as:

In most API use cases token bucket and leaky bucket give similar behavior.

Where to Store Rate Limiting State

Rate limiting needs to remember counts for each key. Where you store this state affects correctness and scalability.

In memory of a single server

For a small single‑instance backend, you can store counters in memory.

Example with a Python dict (simplified):

python 
from collections import defaultdict
import time
counters = defaultdict(list)  # key -> list of timestamps
def allow_request_in_memory(key: str, limit: int, window_seconds: int) -> bool:
    now = time.time()
    timestamps = [t for t in counters[key] if now - t <= window_seconds]
    counters[key] = timestamps
    if len(timestamps) >= limit:
        return False
    counters[key].append(now)
    return True

Limitations:

This is acceptable for dev, not for serious production.

Shared in-memory store (Redis)

Redis is the most common choice:

A fixed window counter with Redis:

python 
import time
import redis
r = redis.Redis()
def allow_request_redis(key: str, limit: int, window_seconds: int) -> bool:
    window_id = int(time.time()) // window_seconds
    redis_key = f"rate:{key}:{window_id}"
    # INCR and EXPIRE should be atomic, usually done with a pipeline or script
    with r.pipeline() as pipe:
        pipe.incr(redis_key, 1)
        pipe.expire(redis_key, window_seconds)
        count, _ = pipe.execute()
    return count <= limit

This works across multiple app servers as long as they all talk to Redis.

Database

You can store counters in PostgreSQL or another DB, but:

This is usually used for long‑term quotas, not for high‑frequency per second limits.

Example use:

CDN / API gateway

Sometimes you do not implement rate limiting in your application at all. Instead you:

Advantages:

Disadvantages:

In many real systems you will combine gateway‑level rate limiting with app‑level rules.

Designing Practical Rate Limits

Choosing sensible limits

You must balance:

A simple way to start:

  1. Estimate how many requests a normal user makes.
  2. Multiply by a safety factor, for example 3 or 5.
  3. Apply a short‑term limit and a long‑term quota.

Example for a small SaaS API:

Different limits per use case

Examples:

Multiple layers of limits

You can combine several rules:

A request is denied as soon as it breaks any of the applied rules.

Communicating Rate Limits to Clients

An important aspect of rate limiting is how you tell clients about their limits and how close they are to the limit.

HTTP status code

When a request is rejected due to rate limiting, you should use:

Mandatory status code for rate limiting

Use HTTP status code 429 Too Many Requests when a request is rejected because of rate limits.

Example response:

http 
HTTP/1.1 429 Too Many Requests
Content-Type: application/json
Retry-After: 60
{
  "detail": "Rate limit exceeded. Try again in 60 seconds."
}

The Retry-After header tells clients how long to wait before retrying. The value can be:

Rate limit headers

It is best practice to tell the client:

A common pattern inspired by GitHub and others:

HeaderMeaningExample
X-RateLimit-LimitMax requests allowed in the window100
X-RateLimit-RemainingRequests left in the current window42
X-RateLimit-ResetUnix timestamp when the window resets1727443200
Retry-AfterSeconds until you can retry (on 429 responses)60

Example response when under limit:

http 
HTTP/1.1 200 OK
Content-Type: application/json
X-RateLimit-Limit: 100
X-RateLimit-Remaining: 57
X-RateLimit-Reset: 1727443200
{
  "data": "..."
}

Example response when limit exceeded:

http 
HTTP/1.1 429 Too Many Requests
Content-Type: application/json
X-RateLimit-Limit: 100
X-RateLimit-Remaining: 0
X-RateLimit-Reset: 1727443200
Retry-After: 60
{
  "detail": "Rate limit exceeded. Try again in 60 seconds."
}

Clearly exposing this information helps API consumers write polite, backoff‑aware clients.

Example: Simple Token Bucket in a FastAPI Service

To make this concrete, here is a simplified example that uses:

Assume you are already comfortable with FastAPI basics.

python 
import time
from typing import Optional
import redis
from fastapi import FastAPI, Request, HTTPException, Depends
app = FastAPI()
r = redis.Redis(host="localhost", port=6379, db=0)
class RateLimitState:
    def __init__(self, tokens: float, last_refill: float):
        self.tokens = tokens
        self.last_refill = last_refill
def get_api_key(request: Request) -> str:
    api_key = request.headers.get("X-API-Key")
    if not api_key:
        # In a real app, you might allow some endpoints without a key
        raise HTTPException(status_code=401, detail="Missing API key")
    return api_key
def rate_limiter(
    api_key: str = Depends(get_api_key),
    capacity: int = 5,
    refill_interval_seconds: int = 10,
):
    """
    Token bucket:
    - Max 5 tokens
    - Bucket refills to full every 10 seconds (so 0.5 tokens/second)
    """
    bucket_key = f"bucket:{api_key}"
    now = time.time()
    # Load existing state from Redis (simple JSON-encoded string for example)
    data: Optional[bytes] = r.get(bucket_key)
    if data is None:
        # First request: full bucket
        state = RateLimitState(tokens=capacity, last_refill=now)
    else:
        tokens_str, last_refill_str = data.decode().split(",")
        state = RateLimitState(tokens=float(tokens_str), last_refill=float(last_refill_str))
    # Refill tokens
    elapsed = now - state.last_refill
    refill_rate = capacity / refill_interval_seconds  # tokens per second
    new_tokens = state.tokens + elapsed * refill_rate
    state.tokens = min(capacity, new_tokens)
    state.last_refill = now
    if state.tokens < 1:
        # No tokens available
        # Calculate wait time until next token is available
        missing = 1 - state.tokens
        wait_seconds = int(missing / refill_rate) + 1
        headers = {"Retry-After": str(wait_seconds)}
        raise HTTPException(
            status_code=429,
            detail=f"Rate limit exceeded. Try again in {wait_seconds} seconds.",
            headers=headers,
        )
    # Consume a token and save state
    state.tokens -= 1
    r.setex(bucket_key, refill_interval_seconds * 2, f"{state.tokens},{state.last_refill}")
    # Optionally you can return info for headers
    remaining = int(state.tokens)
    return {
        "limit": capacity,
        "remaining": remaining,
        "reset_in": refill_interval_seconds,
    }
@app.get("/data")
def get_data(limit_info=Depends(rate_limiter)):
    # You could expose rate info in headers, or just ignore it here
    return {"message": "OK", "rate_limit": limit_info}

Example of using it:

  1. Client sends a request with X-API-Key: test123.
  2. First 5 requests in a 10 second period succeed.
  3. 6th request during that period returns 429 Too Many Requests with Retry-After header.

This is simplified. In production you would:

Handling Distributed Systems

When your backend runs on multiple instances, rate limiting must work across all of them.

Key considerations:

Use a shared store

All instances must read and update the same counters. That is why Redis is used so often.

Bad pattern:

Good pattern:

Atomic operations

When many instances try to update the same counter at the same time, you must avoid race conditions.

Approaches:

Handling network and store failures

If Redis is down:

There is no perfect answer. Common strategies:

Whatever you choose, document it clearly.

Rate Limiting and Client Behavior

Rate limiting only works well if clients behave reasonably.

Backoff strategies

Clients should not:

Instead they should:

A simple algorithm for a client:

  1. Make a request.
  2. If response status is 429:
    • If Retry-After header exists, wait that many seconds.
    • Else, wait an increasing backoff delay.
  3. Try again up to some max number of retries.

Idempotency and retries

When clients retry due to rate limits or timeouts, you want to avoid doing the same side‑effect twice, for example creating two orders.

This is where idempotency (covered in a separate chapter) and rate limiting interact:

Special Use Cases

Protecting login endpoints

Login is a classic target for brute force attacks.

Typical strategy:

If a user gets blocked, you might show:

json 
{
  "detail": "Too many login attempts. Please wait a few minutes before trying again."
}

And send status code 429.

Protecting expensive resources

Some endpoints cost more:

You can apply very low limits for these:

Here the rate limit protects CPU, memory, and third‑party services, not only your API endpoint.

Free vs paid tier

When you monetize your API, rate limiting becomes part of your business model.

You might store limits per plan:

text 
plan:free   -> 1000 requests/day
plan:pro    -> 10000 requests/day
plan:biz    -> 100000 requests/day

In your app, after authenticating the user and loading their plan, you select a different limit.

Pseudocode:

python 
def get_limits_for_plan(plan: str) -> tuple[int, int]:
    if plan == "free":
        return (10, 1000)   # 10 req/min, 1000 req/day
    if plan == "pro":
        return (60, 10000)
    if plan == "business":
        return (200, 100000)
    return (5, 500)         # default fallback

Common Pitfalls and Best Practices

Pitfalls

  1. Only per IP limits
    • Users behind NAT or large organizations may share one IP, causing unfair blocking.
    • Attackers can rotate IPs.
  2. Too strict default limits
    • Perfectly valid workloads get blocked.
    • Clients build hacks to spread traffic across tokens or IPs.
  3. Not documenting rate limits
    • Clients are surprised by 429 errors.
    • Hard to debug integration issues.
  4. No observability
    • No metrics or logs about how often limits are hit.
    • Hard to know if limits are too high, too low, or abused.
  5. Ignoring time synchronization
    • Fixed windows rely on accurate server time.
    • If servers disagree on time, behavior can be unpredictable.

Best practices

Summary

API rate limiting is about controlling how often clients can perform actions so that your backend stays reliable, fair, and secure.

Key points:

With these concepts, you can design and implement effective rate limiting in your own backend APIs and understand how to integrate with APIs that expose rate limits to you.

Views: 7

Comments

Please login to add a comment.

Don't have an account? Register now!

© 2025 Kahibaro
Visitors: 3442 / 24h
Impressum / Imprint
Datenschutz / Data Policy