KAHIBARO
Discord Login Register ☰
☰

27.5. Idempotency

Table of Contents

Why Idempotency Matters in APIs

Idempotency is about making repeated API calls safe. In real systems, clients can retry requests if:

If your API is not idempotent, a single user action might accidentally create multiple records or perform the same operation multiple times, such as:

Idempotency helps you guarantee that repeating the same operation has the same effect as doing it once.

Idempotency rule
An operation is idempotent if performing it once or many times produces the same effect on the system state and the same kind of response to the client.

This is about the effect on the server, not about how many times the client sends the request.

Idempotent vs Non‑Idempotent HTTP Methods

HTTP itself classifies methods by whether they are supposed to be idempotent.

HTTP MethodIntended IdempotencyTypical Use
GETIdempotentRead a resource
HEADIdempotentRead only headers of a resource
OPTIONSIdempotentDiscover server capabilities
PUTIdempotentReplace or create a resource at a URL
DELETEIdempotentRemove a resource
POSTNon‑idempotentCreate or perform actions / side effects
PATCHNot specifiedPartially update a resource

Examples of naturally idempotent methods

GET

http 
GET /users/123 HTTP/1.1
Host: api.example.com

Calling this 1 time or 100 times does not change the user. It just reads data.
If implemented correctly, GET is idempotent.

PUT

http 
PUT /users/123 HTTP/1.1
Content-Type: application/json
{
  "name": "Alice",
  "email": "alice@example.com"
}

If your PUT logic is:

then calling the same PUT multiple times leaves the user in the same final state. So PUT is idempotent by design.

DELETE

http 
DELETE /users/123 HTTP/1.1

If you implement:

The resource state (user 123 is gone) is the same after each call, so DELETE is idempotent, even if the responses differ.

POST is not idempotent by default

http 
POST /orders HTTP/1.1
Content-Type: application/json
{
  "product_id": 10,
  "quantity": 1
}

If your logic is: “create a new order,” then calling POST /orders twice creates two orders. This is not idempotent, because the state changes each time.

Most payment APIs, order creation APIs, and “do this action” endpoints use POST and must be carefully designed if you want safe retries.

Idempotency in Practice: Real‑World Scenarios

Example: Payment or charge endpoint

Imagine an endpoint:

http 
POST /payments
{
  "user_id": 123,
  "amount": 50.00
}

If the client sends the request, gets a timeout, and retries, you might charge the card twice. This is a serious problem.

With idempotency, you want this behavior:

  1. First POST /payments with some unique idempotency key: charge is created.
  2. Client retries the exact same request with the same idempotency key:
    • The server does not create a second charge,
    • It returns the same result as the first time.

Example: Idempotent update of a resource

For an update endpoint:

http 
PUT /users/123/email
{
  "email": "new@example.com"
}

Even without a special key, if you always set the email to this value, this is naturally idempotent.
Retries are safe, because the email is the same after each call.

Implementing Idempotency with Idempotency Keys

Idempotency keys are the most common way to make non‑idempotent operations (usually POSTs) behave idempotently.

Basic idea

http 
  Idempotency-Key: 8a0a0b16-49a3-4cfc-bf43-785f9c330123

Idempotency key rule
For a given idempotency key, your server must:

  1. Perform the operation at most once, and
  2. Return consistent responses for every request that uses this key, as long as the request details match.

Where to put the key

Common approaches:

LocationExample
HTTP headerIdempotency-Key: 123e4567-e89b-12d3-a456-426614174000
Request body field{ "idempotency_key": "..." }
Query parameterPOST /payments?idempotency_key=...

Headers are the most common choice.

Server‑side storage

To support idempotency, the server needs a data store that maps:

FieldDescription
idempotency_keyUnique key provided by client
request_hashHash or snapshot of the request data
response_statusHTTP status code returned the first time
response_bodyResponse body returned the first time
created_atWhen this record was stored
statee.g. processing, succeeded, failed

You can store this in:

Example in a relational table:

sql 
CREATE TABLE idempotency_keys (
    key            TEXT PRIMARY KEY,
    request_hash   TEXT NOT NULL,
    response_status INTEGER,
    response_body   JSONB,
    state          TEXT NOT NULL,
    created_at     TIMESTAMPTZ DEFAULT NOW()
);

Basic processing flow

  1. Receive request with Idempotency-Key K and body B.
  2. Compute a simple request hash H from B (for example SHA256(B) or a stable JSON hash).
  3. Look up K in the idempotency store.

Case 1: No record found

You have never seen this key.

Case 2: Record exists and `state = "succeeded"`

Example:

http 
  HTTP/1.1 409 Conflict
  Content-Type: application/json
  {
    "error": "Idempotency key reuse with different request body"
  }

Case 3: Record exists and `state = "processing"`

This may happen if:

You have options:

Examples of Idempotent Implementations

Example: Create order endpoint with idempotency key

Assume this endpoint:

http 
POST /orders
Idempotency-Key: 50b3d1f4-07b0-4fbb-9c4c-4e1f7f4a0a10
Content-Type: application/json
{
  "user_id": 123,
  "items": [
    { "product_id": 10, "quantity": 1 }
  ]
}

First request

Second identical request (retry)

http 
  HTTP/1.1 201 Created
  Content-Type: application/json
  {
    "order_id": 101,
    "status": "pending",
    ...
  }

To the client this looks as if the operation was processed once, even though it was retried.

Idempotency, Concurrency, and Database Design

Idempotency is tied closely to database constraints and atomic operations.

Using unique constraints

Sometimes you can enforce idempotency without explicit idempotency keys, by using unique fields.

For example:

In SQL:

sql 
CREATE TABLE payments (
    id SERIAL PRIMARY KEY,
    client_transaction_id TEXT UNIQUE NOT NULL,
    user_id INTEGER NOT NULL,
    amount NUMERIC(10,2) NOT NULL
);

Then, in your API:

  1. Client sends:
http 
   POST /payments
   {
     "client_transaction_id": "abc-123",
     "user_id": 42,
     "amount": 50.00
   }
  1. Server tries to INSERT into payments.
  2. If the same request is retried:
    • The INSERT fails because client_transaction_id is already used.
    • You can then SELECT and return the existing payment instead of creating a new one.

This pattern uses database uniqueness to avoid duplicates.

Atomic check‑then‑insert

To avoid race conditions you must:

Otherwise two concurrent requests can both think the record does not exist and both insert, which breaks idempotency.

Use:

Idempotent APIs vs Idempotent Handlers

There is a subtle distinction:

You can have:

So, you must think about behavior, not just the HTTP verb.

Idempotency and Response Semantics

Idempotent operations do not have to:

They must:

For example, a DELETE endpoint:

The resource is deleted in both cases. The operation is idempotent, even though the status and body differ.

For a POST with idempotency keys you usually design so that:

Choosing When to Use Idempotency

You do not need idempotency keys everywhere. Focus on:

For simple reads or simple updates that are already idempotent (like PUT that overwrites data), you may not need special keys.

Common Pitfalls and Best Practices

Pitfalls

  1. Ignoring request differences for the same key
    • If you accept a different body for the same idempotency key, behavior becomes undefined.
    • Always compare a hash and reject mismatches.
  2. Not persisting idempotency data long enough
    • If you delete idempotency records too quickly, retries may arrive after deletion, and you will reprocess the operation.
  3. Only checking at the application layer
    • Without database constraints, a restart or race condition can still create duplicates.
    • Combine app logic with database uniqueness where possible.
  4. Side effects outside your database
    • For example, calling an external payment provider twice, or sending emails twice, even if your local DB is idempotent.
    • You may need idempotency keys when talking to external systems too.

Best practices

Summary

Views: 8

Comments

Please login to add a comment.

Don't have an account? Register now!

© 2025 Kahibaro
Visitors: 3699 / 24h
Impressum / Imprint
Datenschutz / Data Policy