The Outbox Pattern: how to save data and tell other services about it, without losing either
A microservice that writes to its database and then publishes an event can lose one of the two in a crash. The Outbox Pattern fixes that, and here's how it works.
Let me introduce you to a really cool microservice design pattern today: the Outbox Pattern! 📤
It’s a small idea that solves a very real problem. When a service changes its own data and also needs to tell other services about that change, it has to do two writes: one to its database and one to a message broker. Those two writes can’t share a transaction, and that’s where things go wrong.
The problem: one change, two writes
Say a customer places an order. The order service saves the order to its database, and then publishes an OrderPlaced event so inventory, billing and email can do their part.
Now crash the service, or lose the network, between those two steps. You get one of these:

- Save, then publish: the order exists, but nobody downstream ever hears about it. No stock reserved, no invoice, no email.
- Publish, then save: everyone hears about an order that was never saved. Inventory reserves stock for a ghost.
Retries don’t fully save you here either, because the crash can happen right after a write succeeded and before you recorded that it did. This is usually called the dual write problem, and in a microservices setup, where each service has its own database, you hit it all the time.
The fix: put the message in the database too
The Outbox Pattern’s trick is to stop doing two writes in the moment. Instead, the service writes the message into an outbox table in its own database, in the same transaction as the business data. Either both rows are committed, or neither is. A separate process then reads the outbox and does the publishing.

Here’s how that plays out for an order sale:
- When an order is placed, the order data is stored in the database.
- In the same transaction, a message with the order data goes into the
outboxtable. - A background process (the relay) periodically polls the outbox table for unsent messages.
- When it finds one, it publishes it to the other microservices that need to know about the order.
- It marks the message as sent, and the other microservices update their own databases with the order data.
In SQL, the write side is just a normal transaction:
BEGIN;
INSERT INTO orders (id, customer_id, total)
VALUES (1042, 'c-311', 1499.00);
INSERT INTO outbox (id, aggregate_id, type, payload)
VALUES ('e-77', 1042, 'OrderPlaced', '{"orderId": 1042, "items": 2, "total": 1499.00}');
COMMIT;
And the relay is a loop along these lines:
SELECT id, type, payload FROM outbox
WHERE sent_at IS NULL
ORDER BY created_at
LIMIT 100;
-- publish each one to the broker, then:
UPDATE outbox SET sent_at = now() WHERE id = 'e-77';
If the service crashes or loses connectivity, nothing is lost: the message is sitting in the outbox, and the relay picks it up when things come back.
The catch: “at least once”, not “exactly once”
One thing worth being precise about. What the outbox makes atomic is saving the order and saving its message. The delivery itself is at least once.
Look at the relay again. If it publishes e-77 and then crashes before it marks it as sent, it will publish e-77 again after a restart. The broker can’t tell that it’s a duplicate.

So the consumers have to be idempotent: processing the same event twice must have the same effect as processing it once. The simplest way is to remember which event IDs you’ve already handled, in the same transaction as the work itself:
BEGIN;
INSERT INTO processed_events (event_id) VALUES ('e-77')
ON CONFLICT DO NOTHING;
-- if that inserted nothing, we've seen e-77 already: stop here.
INSERT INTO invoices (order_id, amount) VALUES (1042, 1499.00);
COMMIT;
At-least-once delivery plus an idempotent consumer gets you “effectively once”, which is what you actually wanted for something like order sale transactions, where it’s essential that the numbers come out right.
Polling, or tailing the log
The relay in my example polls the table. That’s simple and works with any database, but it adds a little latency and some load from the queries.
The other option is change data capture (CDC): instead of polling, a tool like Debezium reads the database’s own transaction log (the Postgres WAL or the MySQL binlog) and turns new outbox rows into events. It’s faster and puts no query load on the database, but it’s one more piece of infrastructure to run. Debezium even ships an outbox event router for exactly this.
A few things to keep in mind
- Ordering: if consumers care about order, publish events for the same entity in order. Using the entity ID (here, the order ID) as the message key keeps them on the same partition.
- Clean up the outbox: sent rows pile up fast. Delete or archive them on a schedule.
- Keep messages self-contained: put what consumers need in the payload, so they don’t have to call back into your service to make sense of the event.
- You may not need it: if everything lives in one database, or nobody downstream cares about a lost event, a plain transaction is enough. The outbox earns its keep when other services depend on hearing about every change.
That’s the Outbox Pattern: write the message where your data lives, in the same transaction, and let a relay deliver it. Your service never ends up with “saved but never announced” or “announced but never saved” again. 🙌
Keep reading
Data Portability in Enterprise Cloud: my DevFest 2023 New Delhi talk, written up
The written version of my DevFest 2023 New Delhi talk: what enterprise data portability means, data residency, backups vs exports, why a pigeon beats your fibre, and BYOK.
Elevating Trust: Atlassian's Journey to HIPAA Eligibility Expansion!
How Atlassian expanded HIPAA eligibility by replacing a manual BAA process with a self-serve flow and a new service integrating IronClad with Admin Hub.
Our Journey with Multi-region Cloud Architecture and Data Residency
How Atlassian built its multi-region cloud architecture for data residency, covering region creation, service hydration, tenant migration and self-serve moves.
Comments
No comments yet. Start the conversation.
Comments couldn't load right now.