Idempotency — Handling Duplicate Requests
An operation is idempotent if applying it multiple times produces the same result as applying it once. DELETE /resource/123 is idempotent — the resource is gone whether you call it once or ten times. POST /charge is not — every call charges the user again.
The gap between these two behaviors is where bugs live in distributed systems.
The Problem
Networks are unreliable. Clients implement retry logic to handle timeouts and dropped connections. The sequence that causes a duplicate operation:
1. Client sends POST /charge $100
2. Server charges the card and saves the order
3. Server sends 200 OK
4. Response is lost in transit (network blip)
5. Client times out, retries POST /charge $100
6. Server sees a new request — charges the card again
The user is double-charged. The server behaved correctly both times. The problem is that the server had no way to know the second request was a retry of the first.
This pattern affects any non-idempotent operation: payments, order creation, email sending, inventory deductions. Retries are necessary for reliability; without a deduplication mechanism, retries cause data corruption.
Idempotency Keys
The standard solution is to have the client generate a unique key for each logical operation and attach it to every request — including retries. The server uses this key to detect and deduplicate repeated requests.
Workflow:
Client generates: key = "a1b2-c3d4-..." (UUID, generated once per operation)
Attempt 1:
→ POST /charge Idempotency-Key: a1b2-c3d4
← 200 OK (response stored under key a1b2-c3d4)
Network drops. Client retries:
Attempt 2:
→ POST /charge Idempotency-Key: a1b2-c3d4 (same key)
← 200 OK (returned from cache — charge not re-executed)
The key point: the client generates the key once before the first attempt and reuses it for every retry of that same operation. A new operation gets a new key.
Server Implementation
Storage
On receiving a request with an idempotency key, the server checks a fast key-value store (Redis is standard):
- Key not found: process the operation, store the response payload under the key, return the response.
- Key found: skip the operation entirely, return the stored response.
The stored entry needs a TTL. Stripe uses 24 hours — long enough to cover any realistic retry window, short enough to not grow indefinitely.
Atomicity
The check-and-store must be atomic. A race condition exists if two requests with the same key arrive simultaneously (network retry racing with the original):
Request A: checks key → not found
Request B: checks key → not found
Request A: processes charge, stores result
Request B: processes charge, stores result ← duplicate
The fix is a distributed lock or an atomic compare-and-set before processing begins. Redis supports this with SET key value NX (set only if not exists). The first request acquires the lock; the second waits and then reads the result written by the first.
Response consistency
The server must return the exact same response on a replay — same status code, same body. If the original request returned a 402 Payment Required because the card was declined, the replay must also return 402, not re-attempt the charge with a different outcome.
This means storing the full response, not just a flag indicating the key was seen.
HTTP Methods and Idempotency
HTTP defines idempotency at the method level, but this is a protocol contract — enforcement depends on implementation:
| Method | Idempotent | Safe | Notes |
|---|---|---|---|
| GET | Yes | Yes | Read-only, no side effects |
| PUT | Yes | No | Replaces the resource; same result each time |
| DELETE | Yes | No | Resource is absent whether deleted once or ten times |
| POST | No | No | Creates or triggers — requires explicit deduplication |
| PATCH | No | No | Partial update; depends on operation semantics |
POST and PATCH operations that mutate state require idempotency keys if the client needs retry safety.
Edge Cases
Key collision. UUIDs have collision probability low enough to ignore in practice (~1 in 10³⁶). Using anything shorter (sequential IDs, hashes of request fields) reintroduces risk.
Different payloads, same key. If a client sends Idempotency-Key: a1b2 with amount=100 and then sends the same key with amount=200, the server should reject the second request with 422 Unprocessable Entity. The key is bound to the first request's parameters; a different payload is a client bug, not a retry.
Key missing on retry. If the client doesn't send a key, the server cannot deduplicate. Some APIs (Stripe, Braintree) require idempotency keys on all mutating requests and reject requests that omit them.
Concurrent requests, same key. Handled via the distributed lock described above. The second concurrent request blocks until the first completes, then returns the cached result.
In Practice
Stripe requires an Idempotency-Key header on every POST request and surfaces the key in their dashboard for debugging. Their keys expire after 24 hours. Replayed requests return a cached response with an additional Idempotent-Replayed: true header so clients can distinguish original responses from replays.
PayPal, Adyen, and most payment processors follow the same pattern. Outside of payments, the same mechanism applies anywhere a retry could cause a side effect: sending a transactional email, provisioning infrastructure, deducting inventory.
The mechanism is simple. The failure modes when it is absent — duplicate charges, duplicate emails, over-allocated inventory — are not.