Event type filtering on store loads (LoadOptions) #54

New issue

Closed

opened 2026-02-21 22:42:08 +00:00 by ash · 0 comments

ash commented

2026-02-21 22:42:08 +00:00

Owner

Problem

Projections and views declare interest in specific event types but the store loads ALL events and deserializes them. The handler then skips irrelevant ones. For a stream with 50 event types where a projection cares about 2, we deserialize 48 events for nothing.

Inspired by Martin Dilger's projection pattern — his handler switches on 2 event types out of 60+ in the system.

Proposal

Add LoadOptions to both EventStore and DCBStore interfaces:

type LoadOptions struct {
    EventTypes   []string  // SQL WHERE event_type IN (...)
    FromVersion  int       // resume from checkpoint
    Limit        int       // batch size
}

Store implementations add WHERE event_type IN (...) clause. Works for:

Standard ES: filter within a stream
DCB: already filters by type, but LoadOptions unifies the interface

Benchmark Target

Load 10K events, projection cares about 2 of 50 types
Current: deserialize all 10K → measure baseline
With filtering: deserialize ~400 → measure improvement
Expect 10-20x reduction in allocs

Acceptance

LoadOptions struct with EventTypes, FromVersion, Limit
SQLite store implements filtered query
Postgres store implements filtered query
Memory store implements in-memory filter
Benchmarks proving improvement
No breaking changes to existing Load/LoadFrom

## Problem Projections and views declare interest in specific event types but the store loads ALL events and deserializes them. The handler then skips irrelevant ones. For a stream with 50 event types where a projection cares about 2, we deserialize 48 events for nothing. Inspired by Martin Dilger's projection pattern — his handler switches on 2 event types out of 60+ in the system. ## Proposal Add `LoadOptions` to both `EventStore` and `DCBStore` interfaces: ```go type LoadOptions struct { EventTypes []string // SQL WHERE event_type IN (...) FromVersion int // resume from checkpoint Limit int // batch size } ``` Store implementations add `WHERE event_type IN (...)` clause. Works for: - **Standard ES**: filter within a stream - **DCB**: already filters by type, but LoadOptions unifies the interface ## Benchmark Target - Load 10K events, projection cares about 2 of 50 types - Current: deserialize all 10K → measure baseline - With filtering: deserialize ~400 → measure improvement - Expect 10-20x reduction in allocs ## Acceptance - [ ] `LoadOptions` struct with EventTypes, FromVersion, Limit - [ ] SQLite store implements filtered query - [ ] Postgres store implements filtered query - [ ] Memory store implements in-memory filter - [ ] Benchmarks proving improvement - [ ] No breaking changes to existing Load/LoadFrom