New-client launch offer: talk to sales about a focused integration pilot using MuleSoft or n8n. See what's included →
Home / Insights / Idempotency: handling duplicate messages safely
Integrations

Idempotency: handling duplicate messages safely

Explore stable keys, duplicate detection and processing decisions that make integration retries safer.

A retry is not an error. Networks drop packets, brokers redeliver, and a consumer that acknowledged a message a moment before it crashed will see that message again. Every integration platform in existence guarantees at-least-once delivery, and almost none guarantee exactly-once. So the question is never whether a message will arrive twice. It is what your system does the second time.

The failure looks like a business problem

This rarely surfaces as a technical alert. It surfaces as a customer charged twice, an order shipped twice, or a ledger that will not reconcile at month end. By the time finance notices, the run has scrolled out of the logs and nobody can prove what happened.

If your flow cannot safely run twice, it is not finished — it is a demo that happens to be in production.

What idempotency actually requires

Three things, and the first is the one teams skip:

  • A stable key from the source. Not a timestamp, not a generated UUID at the consumer end. Something the producer controls and will repeat when it repeats the message.
  • A record of what has been processed. A table, a cache, a dedupe window. It needs to outlive a restart.
  • A decision about the second arrival. Ignore it, overwrite, or reconcile. All three are valid. Not choosing is what causes the incident.
// the shape of it, whatever the platform
key = message.sourceId + ":" + message.version
seen = store.get(key)

// second arrival: decided, not accidental
if (seen) return ack("duplicate", key)

result = process(message)
store.put(key, result, ttl: 7.days)
ack("processed", key)

Where the argument usually happens

Someone will point out that the dedupe store is itself a dependency, and that a sufficiently unlucky failure between processing and storing the key reopens the hole. That is true. It is also a much smaller hole than the one you started with, and closing it entirely means distributed transactions that cost more than the problem.

The honest position is that idempotency reduces duplicate processing from a routine occurrence to a rare one, and that rare one is caught by reconciliation. Anyone promising you exactly-once semantics over a network is selling something.

What we do on every flow

A stable key agreed with the source system before any code is written. A dedupe store with a TTL longer than the longest plausible retry window. An explicit written decision for the second arrival, in the runbook, so the person on call at 2am does not have to infer it. And a reconciliation job that reports rather than fixes, because silent auto-correction hides the very problem you need to see.

Building something where this matters?

Discuss a focused integration pilot covering the agreed workflow, error handling, testing and handover. Talk to our team about available new-client launch benefits.

See the scope

More on this

Let’s Talk

Your request has been accepted

Thank you for contacting BraidArc. We typically respond within one business day.

Reference: