Event-driven systems often need to do two things that belong to one business action: change local data and tell other systems about that change.
Create an order and publish OrderCreated. Confirm a payment and publish PaymentConfirmed. Change a customer record and notify downstream systems.
The problem is technical but the consequence is very business-like: the database transaction and message publication are still two separate operations. If the first succeeds and the second fails, different parts of the company start working with different realities.
What is the Outbox Pattern?
The Outbox Pattern writes two records inside the same local database transaction:
- the actual business change;
- a record describing the event that must be published later.
That second record is stored in an outbox table. A separate publisher reads pending outbox records and sends them to the message broker. If publication fails, the record remains available for another attempt.
The important point is not the table itself. The important point is that the business change and the obligation to publish an event are committed together.
Why not save to the database and then publish normally?
Because there is always a failure window between the two operations.
If the database commits and the process crashes before publishing, the event is lost. If the event is published first and the database transaction later fails, other systems may react to a business fact that never actually existed.
Trying to make the database and broker participate in one distributed transaction can introduce a different kind of complexity. The Outbox Pattern keeps the atomic part local and moves message delivery into a retryable asynchronous process.
How the Outbox Pattern works step by step
- The application begins a local database transaction.
- It writes or updates the business data.
- It inserts an event record into the outbox table.
- The transaction commits both changes together.
- A publisher finds the pending outbox record.
- The publisher sends the event to the broker.
- The system marks the record as published or otherwise makes sure it will not be treated as new forever.
If the broker is unavailable at step six, the business transaction is still safe. The publisher can try again later.
What does an outbox table contain?
There is no single mandatory schema. The table needs enough information to publish the event reliably and operate the process.
A practical outbox record often needs concepts such as:
- a unique event identifier;
- the aggregate or business object the event belongs to;
- the event type;
- the serialized event payload or the information needed to build it;
- creation time;
- publication state or enough information to determine whether the record still needs processing.
The exact fields depend on the delivery mechanism. The mistake is to treat the outbox table as an ordinary business table and forget that it is part of a delivery protocol.
Polling or change data capture?
The publisher needs a way to discover new outbox records.
Polling is conceptually simple: a worker periodically queries pending rows, publishes them, and updates their state. It is easy to understand and often enough.
Change data capture uses the database change stream or transaction log to detect new records. It can reduce custom polling logic and latency, but introduces its own infrastructure and operational model.
The better choice depends on existing platform capabilities, expected throughput, operational maturity, and how much additional infrastructure the problem justifies.
Duplicate delivery is normal
Outbox delivery is commonly designed around at-least-once behavior.
Imagine the publisher successfully sends an event but crashes before recording that success. On restart, it may publish the same event again.
That means consumers should be idempotent where duplicate processing would create damage. An event identifier can help a consumer recognize that it has already handled a specific message.
The Outbox Pattern protects against lost publication. It does not magically create exactly-once business behavior across every downstream system.
What about event ordering?
Ordering becomes important when several events describe changes to the same business object.
If OrderCreated, OrderPaid, and OrderCancelled can be processed out of sequence, consumers need enough information to detect or tolerate that situation.
The architecture should define where ordering matters: globally, per customer, per order, per partition, or not at all. Global ordering is a much stronger requirement than ordering changes for one business object.
Retries need limits and visibility
A retry loop that silently fails forever is not reliability.
The team needs to see publishing delay, repeated failures, stuck records, unusual backlog growth, and events that cannot be delivered because of invalid data or a persistent downstream problem.
Some failures should be retried. Others need investigation or a dead-letter process. The important business requirement is that a missing downstream action does not stay invisible.
Outbox cleanup is part of the design
Published records accumulate. Keeping them forever may increase table size and slow operational queries; deleting them too aggressively may remove useful audit or recovery information.
Retention should therefore be an explicit decision. The team should know how long records remain, how cleanup works, and what happens if a publisher is delayed longer than expected.
What does the business gain?
The business gets fewer invisible inconsistencies between systems.
An order, payment, shipment, or document action should not disappear from the next process just because infrastructure failed for a moment.
That means fewer manual reconciliations and fewer cases where one department believes an action happened while another system never received it.
What does the team pay for?
The pattern adds an outbox table, a publication process, monitoring, retries, duplicate handling, retention, and possibly ordering logic.
There is also a small delay between the local transaction and downstream processing. The architecture is intentionally trading immediate publication for more reliable publication.
When is the Outbox Pattern unnecessary?
If no important business process depends on publication after a database change, the pattern may be unnecessary.
It can also be excessive when an occasional lost notification has negligible impact and a simpler retry at the application boundary is enough.
Do not add an outbox because event-driven architecture sounds mature. Add it when losing the message after a successful business transaction creates a real problem.
Questions to ask before deciding
- What happens if data is saved but the event is not published?
- Can consumers safely handle duplicates?
- Where does ordering matter?
- How quickly must events reach downstream systems?
- How will we detect and recover stuck records?
- How long do we keep published outbox records?
- Is polling sufficient, or does the platform already support reliable change capture?
In the end
The Outbox Pattern is useful when a database change and the event representing it need to survive failures together.
Its business value is preventing an important process from disappearing in the tiny technical gap between a successful transaction and a message broker.