feat: outbox pattern for reliable integrations #235

New issue

Open

opened 2026-03-10 10:14:24 +00:00 by ash · 0 comments

ash commented

2026-03-10 10:14:24 +00:00

Owner

Problem

When an event is appended and an external side effect is needed (webhook, email, payment API), there is no transactional guarantee. The event may be stored but the side effect lost (crash between append and HTTP call), or vice versa.

Automations help but they replay on crash — causing duplicate side effects unless the external system is idempotent.

Solution

Outbox pattern: store events + outbox entry in the same DB transaction. A background relay reads the outbox and delivers to external systems.

type OutboxEntry struct {
    ID          int64
    StreamID    string
    EventType   string
    Payload     []byte
    Destination string  // webhook URL, queue name, etc.
    CreatedAt   time.Time
    DeliveredAt *time.Time
    Attempts    int
    LastError   string
}

Components:

OutboxWriter — called within Append transaction
OutboxRelay — background goroutine, polls outbox, delivers, marks done
WithOutbox(dest, filter) — option on store to auto-populate outbox
Retry with backoff, DLQ after max attempts

Supported for pgstore and sqlitestore (same DB transaction guarantee).

Pillar: Security (no lost events), Developer Experience

## Problem When an event is appended and an external side effect is needed (webhook, email, payment API), there is no transactional guarantee. The event may be stored but the side effect lost (crash between append and HTTP call), or vice versa. Automations help but they replay on crash — causing duplicate side effects unless the external system is idempotent. ## Solution Outbox pattern: store events + outbox entry in the same DB transaction. A background relay reads the outbox and delivers to external systems. ```go type OutboxEntry struct { ID int64 StreamID string EventType string Payload []byte Destination string // webhook URL, queue name, etc. CreatedAt time.Time DeliveredAt *time.Time Attempts int LastError string } ``` **Components:** - `OutboxWriter` — called within Append transaction - `OutboxRelay` — background goroutine, polls outbox, delivers, marks done - `WithOutbox(dest, filter)` — option on store to auto-populate outbox - Retry with backoff, DLQ after max attempts Supported for pgstore and sqlitestore (same DB transaction guarantee). ## Pillar: Security (no lost events), Developer Experience