Mercur

Idempotency on a Marketplace: Why One Order Becomes Three

Technology and integrations~15 min

Discuss with AI

Idempotency on a Marketplace: Why One Order Becomes Three

Idempotency is the property of an operation that gives the same result whether you perform it once or many times. Setting a stock level to 7 is idempotent, and decreasing it by 1 isn't.

On a marketplace, the same message often reaches your system twice. If the operation behind it isn't idempotent, one order becomes two or three. Each copy carries its own commission and its own seller payout.

You rarely spot a duplicate yourself. You hear about it from a buyer charged twice, from a seller who sent two parcels, or from your accounting team at the month-end close.

If you decide before launch how your system recognizes a repeat, it costs you one database column. If you decide after launch, you pay the yearly bill calculated below.

This article breaks down:

  • Why does the same message reach your marketplace more than once?
  • What is idempotency, and how does an idempotency key stop a repeat?
  • How long should a marketplace remember an idempotency key?
  • Which marketplace operations cost the most when a duplicate gets through?

Key insights

  • The same message reaches your marketplace more than once because queues and event channels promise at-least-once delivery. A marketplace with 10,000 orders and 200,000 offers handles about 680,000 messages a month, and some of them will arrive twice.
  • Idempotency means that a repeated operation leaves the same result as the first one. An idempotency key is a marker the sender attaches to an operation, and the receiver uses it to return the first result when the same key comes back.
  • A marketplace should remember an idempotency key for longer than the sender's longest retry. If the sender retries for four days and you remember the key for one day, a repeat gets through as a new operation for three days.
  • The marketplace operations that cost the most when a duplicate gets through are orders and payouts. At one duplicate per 1,000 operations, duplicates cost about €234,000 a year, or 1.6% of your commission revenue. Protect payouts and refunds first, because you can't reverse them on your own.

Why does the same message reach your marketplace more than once?

Order events 80,000 + offer updates 600,000 = 680,000 messages a month, around 70× the order count.

This series uses one hypothetical platform as its standard example: 10,000 orders a month, 300 sellers, and 200,000 active offers. The cart averages €1,000, and your commission is 12%. That gives you €10 million of GMV a month and €120 million a year.

Count messages instead of orders. One order lives for a couple of weeks and sets off events along the way: created, accepted, sent, delivered, closed, and settled.

About eight events per order give you 80,000 messages a month. Each of the 200,000 offers also changes its price or stock level three times a month on average. That adds another 600,000 messages.

You handle about 680,000 messages a month, almost 70 times more than you have orders.

Some of those messages will arrive twice, and that's how the infrastructure is designed. Every queue and event channel you buy promises at-least-once delivery. Exactly-once delivery is a different promise.

The Amazon SQS documentation says this in plain words. A standard queue may deliver more than one copy of a message and in a different order. The receiver has to survive the repeat.

Some products offer an exactly-once mode, but it works only inside that one product. It ends at the border between your platform and a seller's warehouse system.

A sender that gets no confirmation has two choices: send again or give up. If a channel never repeats a message, it drops every message that gets no confirmation.

You'd rather receive an order twice than lose it. Your system then has to recognize that the second copy is the same operation.

The chapter on webhook integration covers delivery and retry ladders. This chapter starts when a message arrives for the second time.

What is idempotency, and how does an idempotency key stop a repeat?

Three operations compared: setting stock to 7 is idempotent, decreasing by 1 and creating an order are not.

Some operations are idempotent by nature. Setting a stock level to 7 leaves it at 7, however many copies arrive. Decreasing stock by 1 takes one more unit with every copy.

Creating an order isn't idempotent either. It fails in the most expensive way, because every copy becomes a new order with its own commission.

Idempotency in REST API design follows the HTTP methods. GET, PUT, and DELETE are idempotent by definition, and POST isn't. Creating an order is usually a POST request, and it needs protection that the method itself doesn't provide.

1. Idempotency key returns the first answer

With an idempotency key two attempts return order #10412; without a key three attempts create three orders.

An idempotency key is a marker attached to an operation, and the receiver uses it to recognize an operation it has seen before. Stripe's API documentation describes the common method. The sender attaches its own unique key, and the receiver stores the result of the first execution.

When a request with the same key comes back, the receiver returns that stored result. It doesn't perform the operation again.

The receiver doesn't answer with a "duplicate, rejected" error. It returns the response it gave the first time.

The sender gets an order number instead of an error. It has no reason to try a third time.

2. Who issues the idempotency key

Only the sender knows what "the same operation" means. The receiver sees two requests and can't tell a buyer who ordered the same cart twice in a minute from a buyer whose connection stalled.

If the sender issues the key, every system that integrates with you has to produce one. That duty belongs in the integration contract.

The reverse setup is weaker, because the receiver has to guess at similarity. It doesn't ask anything of 300 outside teams, though.

Either setup works. Pick one before launch and write it down for every integrator.

3. Products that leave the key to you

Some products in this class have an idempotency key. Their own manual warns that a create-order call repeated three times without the key creates three orders.

Other products have no such key at all. Then you have to build all of the protection against duplicates yourself.

How long should a marketplace remember an idempotency key?

Timeline: sender retries for four days, you remember the key for one — three days in which a repeat arrives as a new operation.

Deduplication means recognizing repeats and discarding them. It works only while you remember the keys.

You can't remember them forever. You'd store over 8 million keys a year from 680,000 messages a month.

Deduplication therefore comes with a window: the time after which you forget a key. Stripe's API documentation says it can remove a key once it's at least 24 hours old.

Your window has to be longer than the longest retry on the sender's side. Event channels retry delivery for hours and sometimes for several days.

Say the sender retries for four days and you remember the key for one. For three days, a repeat gets through as a new operation. You see no outage and no error message.

A longer window costs less than you'd expect. You get more stored keys to search, and you have to decide how long you may keep them.

A one-day window needs less storage. A one-week window spares you from explaining a duplicate order to the buyer.

People repeat operations much later than machines do. A machine repeats within seconds or hours, and a person uploads the same file next Tuesday.

Which marketplace operations cost the most when a duplicate gets through?

Cost of duplicates across order creation, refund, payout and settlement correction — €234,000 a year, about 1.6% of commission revenue.

Assume one repeat per 1,000 operations. You won't know your own rate until you count it, and 0.1% is a round number that's easy to swap for yours.

1. Order creation

You create 10,000 orders a month. One duplicate per 1,000 gives you 10 duplicates a month and 120 a year. At €1,000 each, that's €120,000.

2. Refunds

Refunds come to 8% of orders, which is 800 operations a month. That gives you about 10 duplicates a year at €1,000 each, or €10,000.

3. Payouts to sellers

Your 300 sellers get paid in two cycles a month. That's 600 transfers a month and about 7 duplicates a year.

The average seller sells €33,300 a month and keeps €29,300 after commission. The payout comes in two halves of about €14,700 each. Seven duplicate transfers cost about €103,000 a year.

4. Settlement corrections

Your 800 refunds plus about 100 manual fixes make 900 corrections a month. That gives you 11 duplicates a year at €120 of commission each, or about €1,300.

5. Total cost of duplicates

Duplicates cost you about €234,000 a year in total. Your commission revenue on €120 million of GMV is €14.4 million.

Duplicates take about 1.6% of that revenue. A repeat rate of 0.1% would still pass for very good.

The last column of the table matters more than the annual cost. An order duplicate happens about 17 times as often as a payout duplicate, and you can usually stop it before the parcel leaves.

A payout duplicate happens a few times a year and costs about 15 times more. You get the money back only by deducting it from the seller's future balances, and only if the seller still sells with you. Protect the operations you can't undo on your own first.

A correction duplicate costs the least in euros, but it ends up in your books. The chapter on the marketplace ledger explains why you reverse an entry with a second one and leave the first unchanged.

Why does a natural identifier fail as an idempotency key?

Four natural keys and the scope each is actually unique in: label code, seller document number, email address, external identifier field.

The operation already carries some identifier, and it's tempting to use it. The seller's document number, the code on the product label, and the buyer's email address all look like free keys.

A natural identifier is free to use, but it can stop being unique at any time. An identifier works as a key only when someone named guarantees that it's unique and states the scope.

Ask if the identifier is unique within the seller, within the instance, or within the year. If no one can answer that, the identifier is only a string of characters.

A key you assign yourself costs one column and one naming rule. A natural identifier can fail in four ways.

1. Code on the product label

The code on the label describes a product model. It doesn't identify a single unit or a single operation.

Two offers of the same product carry the same code, and a manufacturer can reassign a code to a different product years later. That makes a barcode a poor identifier for updates.

The chapter on product matching covers how you match offers by these codes.

2. Seller's document number

The seller's document number is unique only at that seller. Your 300 sellers each number their documents from one.

The first collision comes on the first day, and it's silent. Both numbers are formally correct.

3. Buyer's email address

An email address identifies a mailbox. One address can serve a whole purchasing department, and one person can keep three addresses.

If you build the key from personal data, you also have to decide how long you may keep it. The chapter on marketplace GDPR covers that question.

4. Shared external identifier field

Some products offer an "external identifier" field that several tools write to. The documentation may state that the field doesn't enforce uniqueness, while built-in integrations already use it.

Two systems then write two different meanings into one field.

Which marketplace duplicates come from people instead of machines?

Machine duplicates arrive in seconds to hours, human duplicates in hours to weeks — four human repeat channels.

People repeat operations too, and no one counts those repeats. A machine sends a duplicate within seconds. A person repeats an operation hours or weeks later, after any sensible window has closed.

Operations that people trigger need different tools:

  • A batch identifier on every uploaded file.
  • A button lock that shows the state of the previous operation.
  • A record of who triggered what. The chapter on the audit log covers that record.

1. Double click

An "approve payout" button that doesn't lock after the first press gets pressed twice when the screen is slow. It happens in good faith, usually in the last hour of the payout cycle.

2. Same file uploaded twice

A large share of sellers work through the panel and file imports. Only a minority integrate through an API.

If you protect only the API, the panel and file imports stay open. They carry most of your traffic.

3. Repeated import after a failed attempt

Someone who doesn't know whether the last attempt went through will try again. They're right to try, because the alternative is having no data at all.

An idempotency key can't help here. People don't issue keys, and a visible status of the last attempt does the job instead.

4. Same operation in two places

A team member issues a refund manually at the payment provider. Then they issue it again on the platform, because the first refund doesn't show there.

What does idempotency change in your marketplace?

The idempotency key belongs to whoever owns the object. If the order is created in your store, the store issues the key, and the platform respects it.

The chapter on single source of truth covers which system owns which object. Agree on that map before you discuss who issues the keys.

Hundreds of outside teams will build against your naming convention for idempotency keys. Changing its format after launch changes the contract with every integrator. The chapter on API versioning covers how you change an interface that 300 parties depend on.

Keys from the test environment can collide with production keys. The chapter on configuration management covers how you keep environments apart.

A duplicate in your data differs from a duplicate operation. The chapter on duplicate products covers how you merge two product pages for the same product.

Every repeat you discard also needs a record. You need that count to decide how much to spend on protection.

How do you check idempotency with a vendor?

Ask for a demonstration of each answer. All six questions fit into one demo:

  1. Can you send the same create-order request twice and show me what got created? One order and the same response mean the protection works. Two orders mean it doesn't.
  2. Who issues the idempotency key, your system or mine? What happens if my system doesn't send one?
  3. How long do you remember a key, and how long do you retry delivery? You want two numbers, and the first one has to be larger.
  4. What do you do when the same key arrives with different content? The right answer is a refusal with a clear message. Overwriting the record or creating a second one is the wrong answer.
  5. Which money operations are protected: payouts, refunds, or corrections? Protection on orders alone covers you only where a duplicate is cheap.
  6. What happens when a seller uploads the same file twice? If duplicates appear and the vendor cleans them up afterwards, the channel most of your sellers use has no protection.

Ask yourself two more questions. Who by name owns the key convention, and how many duplicates did you have last quarter?

If no one can answer the second question, your rate isn't zero. No one is counting it.

Which mistakes do operators make with idempotency?

All four mistakes pass a normal test. They show up later, in payouts and refunds.

1. Natural key taken from someone else's data

The label code, the seller's document number, or the email address works until the first collision. The collision gives you a silent overwrite or a second record, with no error.

2. Deduplication window shorter than the sender's retries

This mistake is the hardest to spot. Both sides work correctly, and no one reports a problem.

3. Protection on orders only

The cheapest operation is protected, and the most expensive one stays exposed. You can't reverse a payout or a refund by your own decision.

4. Silent deduplication without a trace

If you discard repeats without recording them, you lose the count you need to plan your protection. It also erases the difference between a repeat and a buyer who really ordered the same cart twice.

What do you still have to settle about idempotency?

This chapter doesn't cover how you build the delivery channel. The chapter on webhook integration covers event subscriptions, retries, and the choice between webhooks and polling.

This chapter also leaves these topics to their own chapters:

This chapter isn't legal or accounting advice. Confirm two questions with your lawyer and your accounting team:

  • How long you may store a key built from data that identifies a person. The chapter on marketplace GDPR covers the data protection side.
  • Whether a duplicated sales document needs a corrective procedure in your case. The chapter on order documents covers how you produce and keep them.

Every number above is hypothetical. Put in your own volume of operations, your own share of refunds, and your own repeat rate. Use the same calculation to get your own result.

Summary: Why does one marketplace order become three?

One order becomes three when the same message arrives three times, and the operation behind it isn't idempotent. Queues and event channels promise at-least-once delivery, and repeats are part of normal traffic.

An idempotency key lets your system return the first result for every repeat. Remember each key for longer than the sender's longest retry, and protect payouts and refunds before anything else.

At one duplicate per 1,000 operations, duplicates cost about €234,000 a year on €120 million of GMV. Orders and payouts carry almost all of it, and only order duplicates are easy to stop.

Building a marketplace where a repeated order or payout returns the first result instead of creating a second one?

Frequently asked questions on idempotency

What is idempotency?

Idempotency means that repeating an operation leaves the same result as performing it once. Setting a stock level is idempotent, and decreasing it isn't. On a marketplace, idempotency decides whether a repeated message creates a second order.

What is an idempotency key?

An idempotency key is a unique marker that the sender attaches to an operation. The receiver stores the result of the first request with that key. When the same key comes back, the receiver returns the stored result and doesn't perform the operation again.

What does idempotent mean in a REST API?

In a REST API, an idempotent request is one you can send many times with the same effect as sending it once. GET, PUT, and DELETE are idempotent by definition, and POST isn't. APIs that create orders or payments with POST add an idempotency key to close that gap.

Ready to build?

We build marketplaces that recognize a repeated order or payout and return the first result instead of creating a second one.