feat: event store compaction for long-lived streams #238

New issue

Open

opened 2026-03-10 10:14:58 +00:00 by ash · 0 comments

ash commented

2026-03-10 10:14:58 +00:00

Owner

Problem

Long-lived streams (e.g., a user account active for years) accumulate thousands of events. Reading the full stream to rebuild state gets slower over time. Snapshots help but old events still consume storage.

Solution

Stream compaction: take a snapshot at version N, then delete events 1..N.

type Compactor[S any] struct {
    Store         EventStore
    SnapshotStore SnapshotStore[S]
    Decider       Decider[S, any, any]
}

func (c *Compactor[S]) Compact(ctx context.Context, streamType, streamID string) (CompactionResult, error)

Safety:

Snapshot MUST succeed before any deletion
Verify snapshot can rebuild correct state before deleting
Configurable retention: keep last N events even after compaction (audit trail)
Dry-run mode
Different from archival (#148) — archival moves to cold storage, compaction deletes permanently after snapshotting

Pillar: Performance, Memory Efficiency

## Problem Long-lived streams (e.g., a user account active for years) accumulate thousands of events. Reading the full stream to rebuild state gets slower over time. Snapshots help but old events still consume storage. ## Solution Stream compaction: take a snapshot at version N, then delete events 1..N. ```go type Compactor[S any] struct { Store EventStore SnapshotStore SnapshotStore[S] Decider Decider[S, any, any] } func (c *Compactor[S]) Compact(ctx context.Context, streamType, streamID string) (CompactionResult, error) ``` **Safety:** - Snapshot MUST succeed before any deletion - Verify snapshot can rebuild correct state before deleting - Configurable retention: keep last N events even after compaction (audit trail) - Dry-run mode - Different from archival (#148) — archival moves to cold storage, compaction deletes permanently after snapshotting ## Pillar: Performance, Memory Efficiency