Skip to content

Decision stream: move to chunked transfer by default#4413

Merged
blotus merged 7 commits intomasterfrom
lapi-decisions-stream-optimization
Apr 27, 2026
Merged

Decision stream: move to chunked transfer by default#4413
blotus merged 7 commits intomasterfrom
lapi-decisions-stream-optimization

Conversation

@blotus
Copy link
Copy Markdown
Member

@blotus blotus commented Apr 7, 2026

  • Always send the decisions list in chunk in the decisions stream.
  • Avoid some useless copies to intermediate structs
  • Use a single JSON encoder to encode each decision

@github-actions
Copy link
Copy Markdown

github-actions Bot commented Apr 7, 2026

@blotus: There are no 'kind' label on this PR. You need a 'kind' label to generate the release automatically.

  • /kind feature
  • /kind enhancement
  • /kind refactoring
  • /kind fix
  • /kind chore
  • /kind dependencies
Details

I am a bot created to help the crowdsecurity developers manage community feedback and contributions. You can check out my manifest file to understand my behavior and what I can do. If you want to use this for your project, you can check out the BirthdayResearch/oss-governance-bot repository.

@github-actions
Copy link
Copy Markdown

github-actions Bot commented Apr 7, 2026

@blotus: There are no area labels on this PR. You can add as many areas as you see fit.

  • /area agent
  • /area local-api
  • /area cscli
  • /area appsec
  • /area security
  • /area configuration
Details

I am a bot created to help the crowdsecurity developers manage community feedback and contributions. You can check out my manifest file to understand my behavior and what I can do. If you want to use this for your project, you can check out the BirthdayResearch/oss-governance-bot repository.

@codecov
Copy link
Copy Markdown

codecov Bot commented Apr 7, 2026

Codecov Report

❌ Patch coverage is 81.92771% with 15 lines in your changes missing coverage. Please review.
✅ Project coverage is 63.77%. Comparing base (7d4daef) to head (1008f00).
⚠️ Report is 1 commits behind head on master.

Files with missing lines Patch % Lines
pkg/apiserver/controllers/v1/decisions.go 77.27% 9 Missing and 6 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #4413      +/-   ##
==========================================
+ Coverage   63.54%   63.77%   +0.22%     
==========================================
  Files         476      476              
  Lines       33841    33821      -20     
==========================================
+ Hits        21505    21568      +63     
+ Misses      10201    10109      -92     
- Partials     2135     2144       +9     
Flag Coverage Δ
bats 46.99% <81.92%> (+0.21%) ⬆️
unit-linux 36.67% <39.75%> (+0.13%) ⬆️
unit-windows 25.35% <0.00%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@blotus
Copy link
Copy Markdown
Member Author

blotus commented Apr 8, 2026

/kind enhancement

@github-actions github-actions Bot added kind/enhancement New feature or request and removed needs/kind labels Apr 8, 2026
@buixor buixor added this to the 1.7.8 milestone Apr 13, 2026
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Moves the v1 decisions streaming endpoint to always use a chunked/streaming JSON response and reduces per-decision allocations while querying/encoding decisions.

Changes:

  • Deprecates the chunked_decisions_stream feature flag and removes the non-chunked streaming path.
  • Updates decision DB query helpers to accept a shared now timestamp and selects only required fields.
  • Streams decisions using a reusable JSON encoder and avoids intermediate slice/marshal copies.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.

File Description
pkg/fflag/crowdsec.go Deprecates the chunked decisions stream feature flag.
pkg/database/decisions.go Adds now time.Time to decision-query methods and narrows selected columns.
pkg/apiserver/controllers/v1/decisions.go Implements always-streaming decisions response and per-decision encoding.
Comments suppressed due to low confidence (1)

pkg/apiserver/controllers/v1/decisions.go:295

  • Setting the Transfer-Encoding: chunked header explicitly is unnecessary (net/http will choose the right framing) and can be problematic under HTTP/2 where Transfer-Encoding is not valid. Consider removing this header and relying on streaming writes/flushes to trigger chunked transfer on HTTP/1.1.
	gctx.Writer.Header().Set("Content-Type", "application/json")
	gctx.Writer.Header().Set("Transfer-Encoding", "chunked")
	gctx.Writer.WriteHeader(http.StatusOK)

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread pkg/apiserver/controllers/v1/decisions.go
Comment thread pkg/apiserver/controllers/v1/decisions.go
Comment thread pkg/apiserver/controllers/v1/decisions.go
@blotus blotus merged commit ef8e8c5 into master Apr 27, 2026
20 of 22 checks passed
@blotus blotus deleted the lapi-decisions-stream-optimization branch April 27, 2026 09:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

kind/enhancement New feature or request needs/area

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants