2026-10-27 · Architecture
Transactional outbox in a messenger: seven phases, not one changed frame
Write to the database first, then send the event: that works until the process crashes right in between. So in a messenger I moved the side effects into an outbox.
A messenger built with Go, PostgreSQL, Redis and WebSocket. After saving a message, the HTTP handler itself sent the event to the sockets, triggered push and webhooks, and wrote system messages such as “X joined”. If the process crashed between commit and sending, the event was lost: the other person saw the message only after a reload, and the webhook never went out. There were no retries, and the same logic lived in several handlers.
Event and data in one transaction
The event now goes into an outbox_events table, in the same transaction as the data change. After the commit, fast consumers such as socket delivery run immediately; slow ones such as push and webhooks run in a worker pool. A background pass picks up, every second, events left behind by a crashed process. Failed consumers are retried with exponential backoff, at most eight times.
A redelivery reads the current state, not an old snapshot. A webhook’s delivery ID is deterministic, so the receiver can drop duplicates. When a message is edited or deleted, its text is also removed from the stored events.
The edges are where it gets dangerous
I built it in seven phases, each in its own branch, each checked by an independent review before merging. The trickiest spots were found by review, not by tests: in a long transaction the delivery deadline counted from insert rather than from commit; a transaction’s connection was reused after the commit; a delete could race with an already scheduled delivery.
To make sure clients would notice nothing, before starting I recorded every WebSocket frame of a scenario with two users, groups and calls: 60 steps, 241 frames. Every phase ran against that reference without a single format change.
On the test environment, data and its consequences no longer drift apart: either the event is stored with the data and will be delivered, or neither happened. Channel joins and call starts, which used to be logged and lost when something failed, are now retried. You need an outbox as soon as a side effect outlives the commit.