Your API Accepts an Idempotency Key. That Does Not Make It Idempotent.

I will use an enterprise expense management platform as the running example in this article. An employee creates a claim, adds expenses, and sends it for review. The specific code below is reconstructed, but the failure cases and design decisions come from building that kind of system.

Imagine the browser sends this request:

POST /api/claims HTTP/1.1
Content-Type: application/json
Idempotency-Key: 2ce09a4b-8d9d-4fcb-a6c4-7568d04f5ef2

{
  "period_id": "0ea8...",
  "claim_type": "NORMAL",
  "currency": "INR"
}

PostgreSQL creates the claim and commits it. The response starts travelling back to the browser, but the connection drops before it arrives.

The employee sees an error. The server has a new claim.

The browser retries.

This is the ambiguity I described in A Timeout Doesn't Tell You Whether the Write Happened. That article deals mainly with writes to systems we do not own. This time, I own the API and the database, so I can make a stronger promise.

There is just one catch: accepting an Idempotency-Key header is not that promise.

The header only gives the request a name. The server still has to make every attempt with that name converge on one committed result.

The header is only a label

It is easy to add an idempotency key to a FastAPI route:

@app.post("/claims", status_code=201)
async def create_claim(
    command: CreateClaim,
    idempotency_key: Annotated[
        str,
        Header(alias="Idempotency-Key"),
    ],
) -> ClaimResponse:
    return await claim_service.create(command)

The route now requires the header. OpenAPI documents it. Every request log can include it.

The route is still not idempotent.

Two requests with the same key still call claim_service.create() twice. If the service inserts twice, the header has changed nothing.

An in-memory dictionary is not a real fix either. It disappears on restart, exists on only one application instance, and can let two concurrent requests pass the check before either stores a result.

Idempotency is a property of the effect, not of the route signature. RFC 9110 defines an idempotent method by whether repeated identical requests have the same intended effect as one request. POST does not receive that property automatically. The application has to provide it.

For this API, the useful guarantee is:

Repeating one logical command may return more than one HTTP response, but it must not commit more than one claim.

That is narrower than “exactly once”, and it is honest. Requests can arrive many times. Logs and metrics can record every attempt. What must converge is the business effect.

First define one logical command

The client should generate a random key once, before the user action is sent. It should reuse that key only when retrying the same action.

The key should not be the entire identity by itself. A browser can generate the same UUID for two users by accident, even if that is extremely unlikely. More importantly, the same text can legitimately appear against two different operations.

I scope a key like this:

(actor_id, operation, idempotency_key)

actor_id stops one user's key from colliding with another user's key. operation stops a key used for claim.create from colliding with the same key used for claim.submit. If actor identifiers are only unique inside a tenant, the tenant identifier belongs in this scope too.

The operation name should describe the command, not merely copy today's URL. Routes and API versions change. A stable name such as claim.create can survive those changes.

That tuple answers who and what the key belongs to. It does not yet prove that a retry carries the same command.

For that, I bind the key to a request fingerprint.

The same key must mean the same input

Suppose the first request uses the key 2ce09a4b... for an INR claim. The client then reuses it with currency: "USD".

Returning the first INR response would be surprising. Executing the USD request would make the key meaningless. The safe answer is a conflict.

The server therefore stores a cryptographic hash of the validated command. A replay with the same hash can receive the stored result. The same key with a different hash receives a 409 response such as IDEMPOTENCY_CONFLICT.

I do not hash the raw HTTP bytes. These two JSON documents mean the same thing:

{"currency":"INR","claim_type":"NORMAL"}

{"claim_type": "NORMAL", "currency": "INR"}

Whitespace and field order should not turn a retry into a different command. I validate the request first, construct a canonical representation, and hash that instead.

def fingerprint(command: CreateClaim) -> str:
    canonical = {
        "schema_version": 1,
        "period_id": str(command.period_id),
        "claim_type": command.claim_type.value,
        "currency": command.currency,
    }
    encoded = json.dumps(
        canonical,
        sort_keys=True,
        separators=(",", ":"),
        ensure_ascii=False,
    ).encode("utf-8")
    return hashlib.sha256(encoded).hexdigest()

This is simplified sample code. A real canonical form must make deliberate choices about defaults, omitted fields, null, decimals, dates, and schema versions. Those choices are part of the API contract.

The random key and the request hash do different jobs. The key identifies one user action. The hash stops that identity from being reused for a changed action. Deriving the key from the body would collapse two separate but identical user actions into one, which is not always correct.

Put the guarantee in PostgreSQL

The database needs a durable record of the command and its outcome. A compact version can look like this:

CREATE TABLE api_idempotency (
    actor_id        uuid        NOT NULL,
    operation       text        NOT NULL,
    key             text        NOT NULL,
    request_hash    char(64)    NOT NULL,
    state           text        NOT NULL
        CHECK (state IN ('IN_PROGRESS', 'SUCCEEDED', 'FAILED')),
    lease_token     uuid        NOT NULL,
    resource_id     uuid,
    response_status smallint,
    response_body   jsonb,
    created_at      timestamptz NOT NULL DEFAULT now(),
    expires_at      timestamptz NOT NULL,
    completed_at    timestamptz,

    PRIMARY KEY (actor_id, operation, key)
);

The primary key is the concurrency control. PostgreSQL allows only one row for that actor, operation, and key. Its unique-constraint documentation explains the database property underneath this design.

The response should contain only what is needed for a safe replay. Storing every response body blindly can duplicate personal or financial data into a table with a different retention policy. In many cases, a resource identifier, status code, and small allowlisted response snapshot are enough.

The first request tries to reserve the key:

INSERT INTO api_idempotency (
    actor_id,
    operation,
    key,
    request_hash,
    state,
    lease_token,
    expires_at
)
VALUES (
    :actor_id,
    :operation,
    :key,
    :request_hash,
    'IN_PROGRESS',
    :lease_token,
    now() + interval '24 hours'
)
ON CONFLICT (actor_id, operation, key) DO NOTHING
RETURNING lease_token;

PostgreSQL's INSERT ... ON CONFLICT makes the insert and conflict decision one database operation. That matters. A SELECT followed by an INSERT leaves a gap where two requests can both decide that no record exists.

If the statement returns the lease token, this request owns the command. If it returns no row, another request has already reserved or completed it. The loser reads that existing record and follows the replay rules.

This works across threads, processes, restarts, and application replicas because they all contend on the same database constraint.

In this version of the pattern, the reservation insert runs in a short transaction and commits before the business work begins. That makes IN_PROGRESS visible to another application instance immediately. The successful result is finalized differently, because it must share a transaction with the effect it describes.

The business result and idempotency result must commit together

Reserving the key prevents another request from starting the same command. It does not prove that the claim was created.

For a short local command, I keep the business mutation and the successful idempotency result in one PostgreSQL transaction:

reservation = await reserve_command(
    actor_id=principal.user_id,
    operation="claim.create",
    key=idempotency_key,
    request_hash=fingerprint(command),
)

if reservation.is_replay:
    return replay(reservation)

async with session.begin():
    await lock_owned_reservation(
        session,
        key=idempotency_key,
        lease_token=reservation.lease_token,
    )

    claim = await claims.create(session, principal, command)

    session.add(AuditEvent.for_claim_created(claim, principal))
    session.add(OutboxEvent.for_claim_created(claim))

    await mark_succeeded(
        session,
        lease_token=reservation.lease_token,
        resource_id=claim.id,
        response_status=201,
        response_body=ClaimResponse.from_claim(claim),
    )

Again, this is reconstructed pseudocode rather than code copied from a private repository.

The important line is the transaction boundary. The claim, audit evidence, outbox obligation, and successful idempotency result commit together.

If the transaction rolls back, none of them exists. If it commits, the replay record points to the result that actually committed. There is no state where PostgreSQL contains the claim but the idempotency row still says that the operation never succeeded.

A definitive business rejection needs an explicit outcome too. The server can store a FAILED result and replay the same safe error for the same key and body. It can instead release the reservation when the contract proves that execution never began. What it must not do is label an ambiguous timeout or unexpected failure as safely failed. I leave that reservation in progress until a recovery rule can prove that no business transaction committed.

The outbox row is present because sending a notification inside this transaction would create a second failure boundary. Your Database Commit Succeeded. Your Message Did Not. covers that part of the design. The idempotency record protects this API command. The outbox protects the later hand-off to a worker.

Some systems avoid a separately committed IN_PROGRESS reservation and do everything in one database transaction. A competing request then waits on the unique key until the winner commits or rolls back. That is often the simplest design for very short operations.

I prefer a visible reservation when waiting would tie up request capacity or when the client benefits from an explicit “still running” response. The trade-off is that abandoned reservations now need a safe recovery rule.

Two requests can arrive at the same time

Sequential retries are the easy test. Concurrency is the real one.

Two application instances can receive the same request before either has returned a response. Both carry the same key and body. Both try to reserve the same primary key.

PostgreSQL chooses one winner. The other insert conflicts. Application code does not get to choose based on timing.

The loser then sees one of three useful states:

• SUCCEEDED: return the previously committed result.
• FAILED: return the previously recorded definitive rejection.
• IN_PROGRESS: return a documented conflict such as 409 IDEMPOTENCY_IN_PROGRESS, so the client waits and retries with the same key.

It must not start a second claim because the first one is taking longer than expected.

An expiry time does not change that rule by itself. If an expired reservation can be reclaimed, the new owner must receive a new lease_token. The old owner must include its original token when completing the command. If ownership changed, its completion update affects no row and its whole business transaction must roll back.

Without that fencing check, expiry can create the exact race the idempotency layer was meant to prevent: the old process wakes up after the new process has taken over, and both commit.

A bounded window, such as 24 hours, can be reasonable for ordinary API retries. It is still a product contract. After a completed record is removed, the same key may execute again. Operations that must remain unique forever need a business constraint as well.

Make every replay outcome explicit

The handler should not hide all duplicate cases behind “record already exists.” The body hash and state tell it what happened.

The successful replay status is an API decision, not a universal rule. Stripe's idempotency contract stores the first status and body and returns that result again. In one API I worked on, create operations returned 201 Created for the first execution and 200 OK for a replay. Both approaches can work if clients can rely on the documented behaviour.

Whatever the response policy is, authorization still runs before replay. An idempotency record is not a capability token. A user who has lost access should not receive an old sensitive response merely because they still know its key.

The client owns half of the guarantee

The server cannot recognise a retry if the client generates a new key for every attempt.

The browser needs to treat the key as part of the pending command:

new user action
    -> generate key once
    -> send key + body

timeout / lost connection / still in progress
    -> keep the same key
    -> keep the same body
    -> retry after backoff

definitive success or definitive rejection
    -> close the command

user changes the body and tries again
    -> generate a new key

This distinction sounds small, but it changes retry behaviour across the frontend.

For one project, I treated network errors, timeouts, 408, 429, and the server's explicit in-progress response as non-definitive. The client kept the original key because none of those outcomes proved that the command had not begun. A definitive validation or domain rejection allowed the corrected action to receive a fresh key.

The exact status list depends on where gateways, rate limits, and validation run in your system. The important part is that the API contract classifies outcomes. The browser should not guess from “4xx” or “5xx” alone.

Do not put user or business data inside the key. A random UUID is easier to store safely in logs and support tools. If the browser must survive a refresh while an outcome is unknown, persist the pending key and the exact command long enough to reconcile it.

Do not require a key everywhere

Idempotency keys are valuable for commands that create a new effect: create, submit, publish, approve, export, pay, or start a durable job.

They are not a ritual for every endpoint.

A GET does not need an application idempotency record. A pure preview or simulation changes nothing, even if it uses POST because the request body is large. A PUT that replaces one named resource may already have idempotent semantics. Some delete and mark-as-read operations naturally converge on the same state.

The decision should follow the business effect, not a blanket rule about HTTP verbs.

Requiring a key for a preview adds noise. Omitting it from a payment export is dangerous. Those two endpoints can both use POST and still need different contracts.

Four protections that answer different questions

An idempotency key is one database invariant, not a replacement for all the others.

The expense platform can use more than one at the same time.

The idempotency record makes a retry with the same key return the same claim. A unique constraint on the employee, period, and claim type can stop two different keys from creating a business duplicate. An ETag can stop an old browser tab from overwriting a newer claim version. The outbox can preserve the obligation to notify a reviewer.

These mechanisms overlap at the edges, but they do not solve the same failure.

The same is true outside the database. Our API key does not make an email provider, payment system, or another company's API idempotent. The downstream worker still needs the provider's idempotency support, a discoverable business reference, or an honest reconciliation path.

Test the failure boundaries, not the happy path

A test that sends one request and receives 201 proves almost nothing about idempotency.

The useful test suite tries to break the invariant:

• send the same key and body twice, then assert that both responses identify one claim;
• send many identical requests concurrently, then assert that PostgreSQL contains one business effect;
• reuse the key with a changed body and expect a conflict;
• use the same key for two actors and prove that their commands do not collide;
• use the same key for two operation names and prove that their commands do not collide;
• commit the transaction but drop the response, then verify that a retry replays the committed result;
• stop after reservation but before the business transaction, then exercise expiry and fenced recovery;
• fail inside the business transaction and verify that the claim, audit event, outbox row, and success result all roll back;
• revoke the actor's permission between the first request and replay, then verify that authorization still wins;
• expire the idempotency record and confirm that the permanent business constraint still rejects an impossible duplicate.

The concurrency test must use the real database. A mock cannot reproduce PostgreSQL's unique-index wait and conflict behaviour.

I also inspect the stored response. It should contain enough information to replay safely and no more. Idempotency is a reliability feature, but a careless response snapshot can quietly become a second store of sensitive data.

The standard I use

I do not call an API idempotent because its documentation lists Idempotency-Key.

I call the command idempotent when:

• the client keeps one key for one logical action;
• the server scopes that key to an actor and operation;
• the key is bound to a canonical request fingerprint;
• a database constraint chooses one owner under concurrency;
• business state and the successful replay result commit together;
• changed input, definitive failure, in-progress work, and successful replay have distinct responses;
• expiry cannot let an old owner commit after a new owner takes over;
• authorization, business uniqueness, optimistic concurrency, and downstream delivery remain separate controls;
• crash and concurrency tests prove the property against the real database.

The header is useful. It gives both sides a shared name for the command.

The guarantee lives behind it: in the unique constraint, the transaction boundary, the stored result, and the client's decision to reuse the same key when the outcome is unknown.

Without those pieces, the API does not support idempotency.

It supports a string called an idempotency key.