subscription: lag monitoring with configurable alert callback #146

New issue

Closed

opened 2026-03-01 11:45:06 +00:00 by ash · 0 comments

ash commented

2026-03-01 11:45:06 +00:00

Owner

Problem

If projections fall behind (slow Evolve, high write throughput), there is no signal. Projection lag grows silently until users notice stale data.

Solution

Add lag monitoring to subscription:

subscription.Config{
    OnLagThreshold: func(lag uint64, consumer string) {
        slog.Warn("projection falling behind", "lag", lag, "consumer", consumer)
        metrics.Record("projection.lag", lag)
    },
    LagThreshold: 1000, // fire when lag exceeds this
    LagCheckInterval: 10 * time.Second,
}

The subscription already has Lag() — just need periodic checking and callback.

Requirements

Configurable threshold and check interval
Callback fires when lag exceeds threshold
Callback fires again when lag recovers (with recovered=true)
Test: simulate slow handler, verify callback fires
Zero overhead when not configured

## Problem If projections fall behind (slow Evolve, high write throughput), there is no signal. Projection lag grows silently until users notice stale data. ## Solution Add lag monitoring to subscription: ```go subscription.Config{ OnLagThreshold: func(lag uint64, consumer string) { slog.Warn("projection falling behind", "lag", lag, "consumer", consumer) metrics.Record("projection.lag", lag) }, LagThreshold: 1000, // fire when lag exceeds this LagCheckInterval: 10 * time.Second, } ``` The subscription already has `Lag()` — just need periodic checking and callback. ## Requirements - Configurable threshold and check interval - Callback fires when lag exceeds threshold - Callback fires again when lag recovers (with recovered=true) - Test: simulate slow handler, verify callback fires - Zero overhead when not configured