Decision stream: move to chunked transfer by default#4413
Conversation
blotus
commented
Apr 7, 2026
- Always send the decisions list in chunk in the decisions stream.
- Avoid some useless copies to intermediate structs
- Use a single JSON encoder to encode each decision
|
@blotus: There are no 'kind' label on this PR. You need a 'kind' label to generate the release automatically.
DetailsI am a bot created to help the crowdsecurity developers manage community feedback and contributions. You can check out my manifest file to understand my behavior and what I can do. If you want to use this for your project, you can check out the BirthdayResearch/oss-governance-bot repository. |
|
@blotus: There are no area labels on this PR. You can add as many areas as you see fit.
DetailsI am a bot created to help the crowdsecurity developers manage community feedback and contributions. You can check out my manifest file to understand my behavior and what I can do. If you want to use this for your project, you can check out the BirthdayResearch/oss-governance-bot repository. |
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #4413 +/- ##
==========================================
+ Coverage 63.54% 63.77% +0.22%
==========================================
Files 476 476
Lines 33841 33821 -20
==========================================
+ Hits 21505 21568 +63
+ Misses 10201 10109 -92
- Partials 2135 2144 +9
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
|
/kind enhancement |
There was a problem hiding this comment.
Pull request overview
Moves the v1 decisions streaming endpoint to always use a chunked/streaming JSON response and reduces per-decision allocations while querying/encoding decisions.
Changes:
- Deprecates the
chunked_decisions_streamfeature flag and removes the non-chunked streaming path. - Updates decision DB query helpers to accept a shared
nowtimestamp and selects only required fields. - Streams decisions using a reusable JSON encoder and avoids intermediate slice/marshal copies.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
| pkg/fflag/crowdsec.go | Deprecates the chunked decisions stream feature flag. |
| pkg/database/decisions.go | Adds now time.Time to decision-query methods and narrows selected columns. |
| pkg/apiserver/controllers/v1/decisions.go | Implements always-streaming decisions response and per-decision encoding. |
Comments suppressed due to low confidence (1)
pkg/apiserver/controllers/v1/decisions.go:295
- Setting the
Transfer-Encoding: chunkedheader explicitly is unnecessary (net/http will choose the right framing) and can be problematic under HTTP/2 whereTransfer-Encodingis not valid. Consider removing this header and relying on streaming writes/flushes to trigger chunked transfer on HTTP/1.1.
gctx.Writer.Header().Set("Content-Type", "application/json")
gctx.Writer.Header().Set("Transfer-Encoding", "chunked")
gctx.Writer.WriteHeader(http.StatusOK)
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.