ash/eskit
1
0
Fork 0
Code Issues 25 Pull requests Projects Releases Packages Wiki Activity Actions

perf: pre-allocate slices + event_type index + partitioning guide #169

Merged
ash merged 1 commit from feat/perf-insane-traffic into main 2026-03-02 22:53:27 +00:00
Conversation 0 Commits 1 Files changed 3 +119 -3
ash commented 2026-03-02 22:53:22 +00:00
Owner
Copy link

Performance optimizations for high-traffic deployments:

  • Pre-allocate event slices in ReadFrom/ReadFromWithOptions/GlobalReader
  • Add idx_events_type composite index for filtered subscriptions
  • Table partitioning guide for 100M+ events
  • Monitoring queries for production

Benchmarks (AMD EPYC 4 CPU, Postgres 16):

  • AppendBatch100: 18.8K events/sec
  • ReadStream 1000: 322K events/sec
  • GlobalRead 1000: 333K events/sec
  • ConcurrentAppend 10 writers: 11.7K events/sec
Performance optimizations for high-traffic deployments: - Pre-allocate event slices in ReadFrom/ReadFromWithOptions/GlobalReader - Add idx_events_type composite index for filtered subscriptions - Table partitioning guide for 100M+ events - Monitoring queries for production Benchmarks (AMD EPYC 4 CPU, Postgres 16): - AppendBatch100: 18.8K events/sec - ReadStream 1000: 322K events/sec - GlobalRead 1000: 333K events/sec - ConcurrentAppend 10 writers: 11.7K events/sec
perf: pre-allocate slices + event_type index + partitioning guide
Some checks failed
CI / test-sqlitestore (push) Failing after 3s
Details
CI / test-pgstore (push) Failing after 3s
Details
CI / test-nats (push) Failing after 6s
Details
CI / test-integration (push) Failing after 5s
Details
CI / benchmarks (push) Failing after 3s
Details
CI / test-codec-otelkit (push) Successful in 12s
Details
CI / test-core (push) Failing after 1m10s
Details
CI / test-sqlitestore (pull_request) Failing after 3s
Details
CI / test-pgstore (pull_request) Failing after 3s
Details
CI / test-nats (pull_request) Failing after 4s
Details
CI / test-integration (pull_request) Failing after 4s
Details
CI / benchmarks (pull_request) Failing after 4s
Details
CI / test-codec-otelkit (pull_request) Successful in 12s
Details
CI / test-core (pull_request) Failing after 1m15s
Details
c7074e1c18 
- Pre-allocate event slices in ReadFrom/ReadFromWithOptions/GlobalReader
  with make([]E, 0, limit) to avoid append growth during batch reads
- Add idx_events_type index on (event_type, id) for filtered subscriptions
- Table partitioning guide for 100M+ events
- Monitoring queries for production
ash merged commit a453ad1160 into main 2026-03-02 22:53:27 +00:00
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels   
bug

   
documentation

   
enhancement

   
investigation

   
nice-to-have

   
performance

   
production-ready

   
testing
No labels
bug
documentation
enhancement
investigation
nice-to-have
performance
production-ready
testing
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
ash/eskit!169
No description provided.