Idempotency Keys: Making a Retry Safe
A retry can duplicate a payment. Learn the idempotency key, the storage, the race, and how the response is replayed.
By Velloreth
The problem
A client sends a payment request.
The network drops the response.
The client retries.
The server charges twice.
The idempotency key prevents the second charge.
The mechanism
The client sends a unique key.
POST /payments
Idempotency-Key: 8f14e45f-ceea-467a-9c1e-1a2b3c4d5e6f
The server:
- Looks up the key
- If it is new, performs the operation and stores the result
- If it exists, returns the stored result
The second request is a replay, not a new operation.
The key
Generated by the client, as a random UUID.
Once per logical operation, not per attempt.
A retry reuses the same key.
Not derived from the body alone, because two identical requests may be two legitimate operations.
A key per attempt is the common mistake, because it defeats the purpose.
The storage
The record:
| Field | Purpose |
|---|---|
| The key | the lookup |
| The status | in progress, done, failed |
| The response | the replay |
| A request hash | detect a key reused with a different body |
| A timestamp | the expiry |
A unique constraint on the key is required, because it is what makes the race safe.
The expiry is often 24 hours.
The race
Two requests with the same key arrive at once.
Without a guard, both perform the operation.
The fix: a unique constraint.
INSERT INTO idempotency (key, status) VALUES ($1, 'in_progress');
-- A duplicate key error means another request is in flight
The second request either waits for the first to finish or returns a conflict.
A lock with a timeout is the alternative.
The response
The stored response is replayed, including the status code.
A 201 is replayed as a 201, not as a 200.
A header can indicate the replay.
Idempotency-Replayed: true
The body must be identical, or the client sees a different result.
The request hash
A key reused with a different body is an error.
Key: abc
Request 1: { "amount": 100 }
Request 2: { "amount": 200 } <- the same key, a different body
The server compares the hash and returns a 422.
Without the check, the second request silently returns the first result, which is a wrong outcome.
The methods
| Method | Idempotent |
|---|---|
| GET | yes |
| HEAD | yes |
| PUT | yes |
| DELETE | yes |
| POST | no |
| PATCH | no |
A POST that creates a resource needs an idempotency key.
A PUT is idempotent by definition, because it replaces the resource.
The client
The client must:
- Generate the key once per operation
- Reuse it on every retry
- Store it until the operation is confirmed
- Handle a replay as a success
A client that generates a new key per retry has no protection.
A client that does not store the key cannot retry after a restart.
The expiry
A key needs an expiry, or the store grows forever.
The window must be longer than the client’s maximum retry period.
24 hours is a common choice.
A cleanup job removes the expired records.
A shorter window risks a duplicate after the record expires.
The key in a real client
A client that uses an idempotency key correctly has three responsibilities, and each one is a place where a naive implementation fails. The first is to generate the key at the moment the user acts, not at the moment the request is sent, because a retry must reuse the same key and a key generated per attempt provides no protection. The second is to persist the key before the request, in the same transaction as the local record of the operation, because a client that crashes between the request and the response must be able to retry with the same key after a restart. The third is to treat a replayed response as a success rather than as an error, because the operation did happen, and a client that shows an error to the user after a successful payment is a worse outcome than a duplicate. A client that follows those three rules can retry aggressively, which is what makes the whole mechanism worth the server-side cost.
The storage in a real system
The store that holds the keys is a small table with a large write rate, and its design matters more than it appears. The key is the primary key, so the lookup is a single index hit, and the unique constraint is what makes the concurrent case safe. The response body is stored as a blob or a JSON column, and it is replayed verbatim, which means the stored response must include the status code and the headers that matter. The request hash is a short digest, and it is compared on a repeat to detect a key reused with a different body. The expiry is a timestamp, and a cleanup job removes the expired rows, because a table that grows without bound is a table that eventually slows every write. A store that is a cache rather than a database is a common choice, and it must be durable enough that a restart does not lose a key that a client is about to retry.
Common mistakes
A key per attempt. No protection.
No unique constraint. A race.
No request hash. A key reused with a different body.
No expiry. An unbounded store.
A key derived from the body. Two legitimate requests collide.
A response that is not replayed. The client sees a different result.
A status code that is not replayed. A 201 becomes a 200.
A key stored in memory only. It is lost on a restart.
A key that is not documented. The client does not send one.
A retry on a non-idempotent call without a key. A duplicate.
A quotable summary
An idempotency key is a unique value the client sends with a request so the server can recognise a repeat, store the result, and replay it rather than performing the operation again. The key is generated once per logical operation and reused on every retry, the server stores the key, the status, the response and a request hash, and a unique constraint on the key is what makes a concurrent request safe. A key needs an expiry, or the store grows forever.
Related reading
For the related topics, see rest-api-design-principles and api-error-responses-guide. For the broader context, background-jobs-guide and rest-api-rate-limiting.