Spurlock Studios
Contact
Idempotency Keys in n8n: Stop Double-Charging, Double-Emailing, Double-Everything

If a webhook can fire twice — and it can — your workflow must be safe to run twice. That is all idempotency means in practice: same event in, same business outcome out, no extra charges, emails, or CRM rows.

This post is the production pattern Spurlock Studios uses in n8n to prevent duplicate side effects. It sits inside the broader Production n8n handbook.

Why duplicates happen (even when “nothing is wrong”)

Providers deliver webhooks at least once. Common causes:

  • Your endpoint was slow; the provider timed out and retried after you already succeeded.
  • A network blip dropped the 200 response.
  • You replayed an execution while debugging.
  • A queue consumer redelivered after a crash.

None of these require a bug in your graph. They require a gate before irreversible work.

What an idempotency key is

An idempotency key is a stable string that uniquely identifies a business event, not a workflow execution. You compute it from the payload, store it when you start (or finish) processing, and skip side effects if you have seen it before.

Good keys:

  • stripe_evt_12345
  • typeform_response_abc:submitted
  • hubspot_deal_99:stage_won:2026-06-01T12:00:00Z (when stage transitions can repeat meaningfully)

Bad keys:

  • n8n execution ID (new every run — useless for dedupe)
  • Timestamp-only keys (collisions and replay pain)
  • Entire raw body hashed without knowing which fields are identity vs noise

The n8n pattern that works

Minimal spine:

  1. Webhook / Trigger receives the event.
  2. Verify signature (see Webhook Security).
  3. Code node builds the key from identity fields.
  4. Data store / DB / Redis lookup — if key exists, return 200 and stop.
  5. Mark key as in-flight or completed (with TTL).
  6. Only then run CRM / email / payment nodes.
  7. On success, ensure the key is marked complete; on poison failure, send to DLQ without deleting the key if a side effect may have landed.

Example key construction

const crypto = require("crypto");
const id = $json.id || $json.data?.id;
const version = $json.updated_at || $json.data?.updated_at || "v1";
if (!id) throw new Error("Missing event id for idempotency key");
const key = crypto.createHash("sha256").update(`${id}:${version}`).digest("hex");
return [{ json: { ...($json), idempotencyKey: key } }];

Use the provider’s event ID when they give you one. Fall back to a hash of stable identity fields when they do not.

Where to store the keys

Pick storage you already operate:

StoreProsCons
n8n Static Data / Data StoreSimple for low volumeNot ideal as the only store at high scale
Postgres / Supabase tableQueryable, auditableYou manage schema and TTL cleanup
RedisFast TTL natural fitAnother moving part
Downstream “create if not exists”Best when the API supports idempotency keys nativelyNot all APIs do

Prefer native idempotency headers when Stripe-like APIs offer them and still keep your own key for the rest of the workflow. Defense in depth beats faith in one vendor.

Prevent duplicate webhook runs without breaking retries

Important nuance: when you detect a duplicate, still return success to the provider (HTTP 200) if you already processed the event. Returning 500 invites more retries and more noise.

Flow logic:

  • Key unseen → process → store → 200
  • Key seen + prior success → no-op → 200
  • Key seen + prior in-flight → either wait/no-op carefully or 200 with “already accepted” if your design allows
  • Key unseen but processing fails before any side effect → do not store as complete; allow retry
  • Side effect maybe applied, then crash → store as needs-review, send to DLQ, do not blindly re-run creates

This is why idempotency and dead-letter queues travel together. Keys stop doubles. DLQs handle the ambiguous middle.

Side effects that always need a key check

Put the check before:

  • Payment capture or invoice send
  • Outbound email / SMS / Slack to customers
  • CRM create (not always update — know your upsert rules)
  • Spreadsheet append rows
  • “Increment counter” style analytics writes
  • Ticket creation

Updates that are naturally idempotent (set stage to won with the same payload) are safer, but still benefit from dedupe to cut noise and rate-limit burn.

Testing duplicates on purpose

Before go-live:

  1. Fire the same webhook payload twice in sixty seconds.
  2. Confirm one set of side effects.
  3. Confirm the provider-facing response stays 200.
  4. Replay an old execution in n8n and confirm the gate holds.
  5. Fail the workflow after the key is stored and confirm your DLQ path is sane.

If you have never tested a double delivery, you do not have idempotency. You have a comment in a Notion doc.

Common mistakes

  • Keying on the whole body including volatile fields (timestamps that change, request IDs) — duplicates miss.
  • Storing the key only at the end — a crash after the charge but before the store guarantees a double on retry.
  • Shared keys across different event types — a created and updated event collide and skip real work.
  • No TTL — store grows forever; use retention that matches replay windows (often 7–30 days).
  • Dedupe after the email node — entertaining, useless.

How this fits the production spine

Idempotency is structure #1 in our handbook. Pair it with schema validation so garbage payloads never get a key reserved for real events, and with approvals for high-risk first writes.

Key design patterns by event type

Create events (customer.created, form submitted)
Key = provider event ID. If the provider retries the exact event, you no-op. If a new create happens for the same email, that is a different business question (dedupe/upsert), not the same idempotency key.

Update events (deal.updated)
Key = objectId + updated_at or objectId + version or objectId + hash(meaningful fields). Pure object ID is wrong if updates should apply more than once.

Action events (invoice.paid)
Key = provider event ID. Money events almost always ship with stable IDs — use them.

Synthetic events (cron sweeps)
Key = jobName + partition + date (e.g., renewal-reminders:2026-06-22). Prevents overlapping cron runs from double-sending.

Write the key formula in a sticky note on the canvas. Future you will forget it.

In-flight vs completed states

A boolean “seen” flag is not enough for every system. Prefer a small state machine:

StateMeaningOn retry
processingWork started, not confirmed doneWait / DLQ if stuck too long
completedSide effects committedNo-op 200
failed_cleanFailed before side effectsAllow retry
needs_reviewAmbiguous partial applyHuman only

Implement with a row in Postgres/Airtable: key, state, executionId, updatedAt. TTL or janitor job clears old completed rows.

If you only store keys at the end, crashes after charges create doubles. If you only store at the start without states, crashes block legitimate retries forever. States fix both.

Coordination with downstream idempotency

Some APIs accept Idempotency-Key headers. Use the same string you store locally when possible. That way a network timeout after the provider accepted the request still safe-retries.

When the API has no such header, make creates into upserts keyed by natural identity (email, external ID). Idempotency at the workflow layer plus upsert at the app layer is the usual production pair.

Multi-workflow and fan-out cases

One inbound event sometimes triggers multiple workflows. Options:

  1. Single intake workflow that fans out internally after the key gate (simplest).
  2. Per-workflow keys with suffix: evt_123:crm, evt_123:email — if each side effect must be independently replayable.
  3. Shared key, shared gate service — a tiny sub-workflow that all graphs call first.

Avoid two workflows both checking different incomplete keys against the same charge API. That is how you get “we deduped Slack but double-charged.”

Observability for duplicates

Track metrics weekly:

  • Duplicate hits (key seen → no-op)
  • First-time processes
  • needs_review volume
  • Time stuck in processing

A sudden spike in duplicates often means a provider retry storm or your endpoint latency got worse. A spike in needs_review means partial applies — fix those graphs before you scale traffic.

Worked example: form → CRM → Slack

Imagine a Typeform webhook that creates a HubSpot contact and pings Slack.

Without idempotency: provider retries after a slow HubSpot create → second contact → second Slack ping → sales thinks two leads arrived.

With idempotency:

  1. Verify signature.
  2. Key = typeformResponseId.
  3. Insert processing row (unique constraint on key). If conflict and state=completed, return 200. If conflict and state=processing older than 10 minutes, mark needs_review.
  4. Upsert HubSpot by email with external ID.
  5. Slack notify once.
  6. Mark completed.

Even if step 5 fails after step 4, replay uses HubSpot upsert and a Slack notify keyed by typeformResponseId:slack so the CRM does not fork.

Write this story into the runbook. New teammates ship safer graphs when they can see a concrete path.

Storage schema you can copy

create table automation_idempotency (
  key text primary key,
  workflow text not null,
  state text not null,
  execution_id text,
  business_id text,
  created_at timestamptz default now(),
  updated_at timestamptz default now()
);
create index on automation_idempotency (updated_at);

Janitor: delete completed rows older than 30 days; alert on processing older than 15 minutes; never auto-delete needs_review.

When “exactly once” is a myth

You cannot get mathematically perfect exactly-once across arbitrary SaaS APIs. You can get:

  • At-least-once delivery from the provider
  • Effectively-once business outcomes via keys + upserts + careful replay

Anyone selling “exactly once webhooks” without those pieces is selling a slogan. Design for the myth; implement for the outcome.

Checklist before enabling a new trigger

  • Identity field documented
  • Key formula written on canvas
  • Store supports unique constraint
  • Duplicate test passed
  • Partial-fail path defined
  • TTL / janitor exists
  • Metrics for duplicate hits enabled

Ship the checklist with the workflow. Skip it and you will meet the duplicate in production first.

FAQ

What is n8n idempotency?

It is the practice of giving each business event a stable key and skipping irreversible nodes when that key was already processed. n8n does not do this for you automatically on every trigger. You build the check.

How do I prevent duplicate webhook runs?

Verify the webhook, compute a key from the provider event identity, look it up in a store, and exit successfully if it exists. Only then run creates, charges, or outbound messages. Test by sending the same payload twice.

Should I use the n8n execution ID as the key?

No. Every execution gets a new ID. That cannot detect provider retries. Use the external event ID or a hash of stable business identity fields.

What if the API already supports idempotency keys?

Use them. Also keep your workflow-level key for nodes that do not support native idempotency (email, Slack, spreadsheets, secondary CRMs).

How long should I keep keys?

Long enough to cover provider retry windows and your own manual replays — commonly 7 to 30 days. Finance-adjacent events may need longer audit retention even if the hot dedupe TTL is shorter.

What about partial failures after a key is stored?

Send the item to a dead-letter queue with the original payload and mark it for human review. Do not delete the key and blindly replay if a charge or create may have succeeded. See the DLQ spoke.

CTA

Duplicate side effects are not an edge case. They are a calendar event.

Build the key gate into your next n8n workflow, read the production handbook, and if you want a production review, use the automation lane or book a call.

Book the audit