Problem
internal/api/stream_sse.go:51:
ch := make(chan []byte, 64) // TODO: need to test how many are actually needed, as this is ~1.6KB per subscriber channel...
The SSE per-subscriber channel buffer is hardcoded to 64. At ~1.6KB per slot per subscriber that's ~100KB per stream connection — fine for hundreds of subscribers, painful for thousands. More importantly: 64 is a guess, not a measurement, and the comment admits as much.
If the buffer is undersized for a real workload, the Hub drops messages to slow subscribers (existing Hub behavior); if it's oversized, memory grows linearly with concurrent subscribers.
Proposed Solution
Two parts:
- Make it configurable via
WH_SSE_CHANNEL_BUFFER / server.sse_channel_buffer (default current value of 64) so operators can tune without recompiling.
- Load-test to set a defensible default. Drive
make dev-obs with N=10/100/1000 concurrent SSE subscribers and a realistic ingest rate, measure (a) drop rate at the Hub, (b) per-subscriber memory, (c) tail latency from publish → flush. Pick the smallest buffer that keeps drop-rate at 0 under expected alpha workloads. Document the experiment in docs/deployment.md so operators can re-run it.
Acceptance criteria
Related
Problem
internal/api/stream_sse.go:51:The SSE per-subscriber channel buffer is hardcoded to 64. At ~1.6KB per slot per subscriber that's ~100KB per stream connection — fine for hundreds of subscribers, painful for thousands. More importantly: 64 is a guess, not a measurement, and the comment admits as much.
If the buffer is undersized for a real workload, the Hub drops messages to slow subscribers (existing Hub behavior); if it's oversized, memory grows linearly with concurrent subscribers.
Proposed Solution
Two parts:
WH_SSE_CHANNEL_BUFFER/server.sse_channel_buffer(default current value of 64) so operators can tune without recompiling.make dev-obswith N=10/100/1000 concurrent SSE subscribers and a realistic ingest rate, measure (a) drop rate at the Hub, (b) per-subscriber memory, (c) tail latency from publish → flush. Pick the smallest buffer that keeps drop-rate at 0 under expected alpha workloads. Document the experiment indocs/deployment.mdso operators can re-run it.Acceptance criteria
stream_sse.go:51updated or removed.Related
stream_ws.go) likely has the same parameter; check while you're in there.