Backport security audit fixes (V01-V07), snappy gRPC fix, and stream push panic fix to release-1.21 by friedrichg · Pull Request #7574 · cortexproject/cortex

friedrichg · 2026-05-29T23:38:50Z

Summary

Prepares v1.21.1-rc.0 by backporting all seven Quarkslab security-audit fixes (V01-V07), PR #7459 (snappy gRPC panic), and PR #7541 (stream push panic) onto the release-1.21 branch.

Backport status

#	PR	Finding	Conflicts
1	#7473	V03 config leakage	CHANGELOG only
2	#7474	V07 gossip integrity	none
3	#7475	V01 tenant impersonation	CHANGELOG only
4	#7512	V02 stored XSS	CHANGELOG only
5	#7515	V04 gzip DoS	none
6	#7518	V06 gossip TCP limits	CHANGELOG only
7	#7570	V05 histogram allocation	CHANGELOG + 4 code files
8	#7459	snappy gRPC panic	CHANGELOG + workflow
9	#7541	stream push slice-out-of-range panic	CHANGELOG only

Plus a final commit bumping VERSION to 1.21.1-rc.0 per RELEASE.md.

V05 (#7570) conflict resolution notes

CHANGELOG: reclassified the upstream [ENHANCEMENT] line as [BUGFIX] to match the patch-release tone. All nine new entries appear under ## 1.21.1 in progress.
integration/remote_write_v2_test.go: dropped three incoming tests (TestIngest_StartTimestamp, TestIngest_CreatedTimestampFallback, TestIngest_StartAndCreatedTimestampIgnoredWhenDisabled); they belong to Support Start TimeStamp (ST) & Created Timestamp (CT) on PRW2 #7371 which isn't on release-1.21.
pkg/cortexpb/cortex.pb.go: took the incoming fileDescriptor blob; it matches the cherry-picked .proto.
pkg/cortexpb/timeseriesv2_test.go: kept the Histograms []Histogram -> []WrappedHistogram swap and the MetadataV2 reuse-clearing assertions; dropped the CreatedTimestamp setup + assert.Zero because ReuseTimeseriesV2 on release-1.21 doesn't clear CreatedTimestamp, so the upstream assertion would fail. The MetadataV2 field exists on release-1.21 and is reset by ReuseTimeseriesV2, so its coverage is preserved.
pkg/ingester/ingester_test.go: dropped TestIngester_Push_StartTimestamp and TestIngester_Push_StartTimestampAppendFailureMetrics; both depend on StartTimestamp.
pkg/util/push/push_test.go: dropped Test_convertV2RequestToV1_PreservesStartTimestamp; same reason.

Test plan

go build ./...
go test -timeout 2400s -tags "netgo slicelabels" ./...
CI green on this PR
Sanity test in a staging cluster after rc.0 is tagged

Signed-off-by: Friedrich Gonzalez <1517449+friedrichg@users.noreply.github.com>

* drop digest verification fail packets Signed-off-by: SungJin1212 <tjdwls1201@gmail.com> * fix lint Signed-off-by: SungJin1212 <tjdwls1201@gmail.com> --------- Signed-off-by: SungJin1212 <tjdwls1201@gmail.com> Signed-off-by: Friedrich Gonzalez <1517449+friedrichg@users.noreply.github.com>

* add checking logic if not distirbutor side request Signed-off-by: SungJin1212 <tjdwls1201@gmail.com> * Add SignWriteRequestsKeys Signed-off-by: SungJin1212 <tjdwls1201@gmail.com> * change addr Signed-off-by: SungJin1212 <tjdwls1201@gmail.com> * fix flakiness Signed-off-by: SungJin1212 <tjdwls1201@gmail.com> * fix test message Signed-off-by: SungJin1212 <tjdwls1201@gmail.com> * Add SecretStringSliceCSV type Signed-off-by: SungJin1212 <tjdwls1201@gmail.com> --------- Signed-off-by: SungJin1212 <tjdwls1201@gmail.com> Signed-off-by: Friedrich Gonzalez <1517449+friedrichg@users.noreply.github.com>

#7512) * security: fix stored XSS in alertmanager and storegateway status pages Replace text/template with html/template in alertmanager_http.go and gateway_http.go to auto-escape HTML special characters. This prevents stored XSS via crafted gossip member names rendered on status pages. The html/template package has an identical API to text/template but automatically escapes HTML, JS, and URI contexts. Add TestStatusHandler_HTMLEscaping to verify XSS payloads are escaped. Fixes #22 Signed-off-by: Daniel Blando <ddeluigg@amazon.com> * Fix lint Signed-off-by: Friedrich Gonzalez <1517449+friedrichg@users.noreply.github.com> * changelog Signed-off-by: Daniel Blando <ddeluigg@amazon.com> --------- Signed-off-by: Daniel Blando <ddeluigg@amazon.com> Signed-off-by: Friedrich Gonzalez <1517449+friedrichg@users.noreply.github.com> Co-authored-by: Friedrich Gonzalez <1517449+friedrichg@users.noreply.github.com> Signed-off-by: Friedrich Gonzalez <1517449+friedrichg@users.noreply.github.com>

… ingestion path (#7515) Wrap gzip.Reader with io.LimitReader(maxSize+1) before reading decompressed bytes in both pkg/util/http.go and pkg/util/push/otlp.go. Signed-off-by: Daniel Blando <ddeluigg@amazon.com> Signed-off-by: Friedrich Gonzalez <1517449+friedrichg@users.noreply.github.com>

* add flags for TCP connection limits and timeouts Signed-off-by: SungJin1212 <tjdwls1201@gmail.com> * ensure semaphore slot is held for the lifetime of stream connections Signed-off-by: SungJin1212 <tjdwls1201@gmail.com> * modernize Signed-off-by: SungJin1212 <tjdwls1201@gmail.com> * change oversize packet log level to debug Signed-off-by: SungJin1212 <tjdwls1201@gmail.com> * add metrics Signed-off-by: SungJin1212 <tjdwls1201@gmail.com> * add stream to activeConnections to make accurate tracking Signed-off-by: SungJin1212 <tjdwls1201@gmail.com> * add histogram for tracking inbound packet sizes in bytes Signed-off-by: SungJin1212 <tjdwls1201@gmail.com> --------- Signed-off-by: SungJin1212 <tjdwls1201@gmail.com> Signed-off-by: Friedrich Gonzalez <1517449+friedrichg@users.noreply.github.com>

Signed-off-by: Paurush Garg <paurushg@amazon.com> Signed-off-by: Friedrich Gonzalez <1517449+friedrichg@users.noreply.github.com>

Signed-off-by: Friedrich Gonzalez <1517449+friedrichg@users.noreply.github.com>

* Triggers issue in #7456 Signed-off-by: Friedrich Gonzalez <1517449+friedrichg@users.noreply.github.com> * Including integration tests Signed-off-by: Friedrich Gonzalez <1517449+friedrichg@users.noreply.github.com> * Fix snappy register in grpcclient.go Signed-off-by: Friedrich Gonzalez <1517449+friedrichg@users.noreply.github.com> * Improve integration test Signed-off-by: Friedrich Gonzalez <1517449+friedrichg@users.noreply.github.com> * Update changelog and spawn grpc integration test for this Signed-off-by: Friedrich Gonzalez <1517449+friedrichg@users.noreply.github.com> --------- Signed-off-by: Friedrich Gonzalez <1517449+friedrichg@users.noreply.github.com>

Signed-off-by: Friedrich Gonzalez <1517449+friedrichg@users.noreply.github.com>

SungJin1212 · 2026-05-30T03:05:49Z

@friedrichg
Can you add this PR #7541?

sandy2008

Took a careful pass over the backport — diffed each cherry-pick against its upstream PR and built + vetted the branch under -tags 'netgo slicelabels'. Overall this is a clean, faithful backport: every production hunk for V01–V07 + the snappy fix is byte-identical to upstream, the []Histogram → []WrappedHistogram swap and the regenerated cortex.pb.go build green, and the dropped tests are the right ones (they depend on StartTimestamp/CreatedTimestamp from #7371, which isn't on 1.21).

A few small things inline — mostly test-quality (a couple of the security tests currently pass without actually exercising what they guard), one CHANGELOG/rc nit, and one empty-signing-key edge case carried over from #7475 (flagging for a follow-up, not a blocker). Thanks for putting this together!

Signed-off-by: Friedrich Gonzalez <1517449+friedrichg@users.noreply.github.com>

… marshalling (#7541) * fix panic in stream push when context deadline expires during request marshalling Signed-off-by: SungJin1212 <tjdwls1201@gmail.com> * fix error handling Signed-off-by: SungJin1212 <tjdwls1201@gmail.com> * extract job processing Signed-off-by: SungJin1212 <tjdwls1201@gmail.com> --------- Signed-off-by: SungJin1212 <tjdwls1201@gmail.com> Signed-off-by: Friedrich Gonzalez <1517449+friedrichg@users.noreply.github.com>

sandy2008

LGTM!

SungJin1212

thanks

friedrichg and others added 7 commits May 29, 2026 15:48

Mask Swift, etcd, Redis, and HTTP basic-auth credentials (#7473)

540b773

Signed-off-by: Friedrich Gonzalez <1517449+friedrichg@users.noreply.github.com>

add WrappedHistogram with size limit before unmarshal (#7570)

0aa625b

Signed-off-by: Paurush Garg <paurushg@amazon.com> Signed-off-by: Friedrich Gonzalez <1517449+friedrichg@users.noreply.github.com>

dosubot Bot added the type/security label May 29, 2026

pull-request-size Bot added the size/XXL label May 29, 2026

friedrichg marked this pull request as draft May 29, 2026 23:43

friedrichg added 5 commits May 29, 2026 17:05

Regenerate cortex.pb.go

6c8ec08

Signed-off-by: Friedrich Gonzalez <1517449+friedrichg@users.noreply.github.com>

Prepare release candidate 0 for v1.21.1

15288b4

Signed-off-by: Friedrich Gonzalez <1517449+friedrichg@users.noreply.github.com>

Adapt tests for go 1.25

22b8052

Signed-off-by: Friedrich Gonzalez <1517449+friedrichg@users.noreply.github.com>

Set up target date for release

f70e08f

Signed-off-by: Friedrich Gonzalez <1517449+friedrichg@users.noreply.github.com>

friedrichg force-pushed the backport-security-audit-v01-v07 branch from 27bfcec to f70e08f Compare May 30, 2026 00:07

friedrichg marked this pull request as ready for review May 30, 2026 00:45

friedrichg requested review from SungJin1212, Copilot and danielblando May 30, 2026 00:45

Copilot started reviewing on behalf of friedrichg May 30, 2026 00:45 View session

friedrichg requested review from sandy2008 and yeya24 May 30, 2026 00:45

This comment was marked as off-topic.

Sign in to view

sandy2008 reviewed May 30, 2026

View reviewed changes

friedrichg added 2 commits May 30, 2026 18:12

Set release in progress

6ce3ad7

Signed-off-by: Friedrich Gonzalez <1517449+friedrichg@users.noreply.github.com>

Re add Metadatav2 to test

2866b21

Signed-off-by: Friedrich Gonzalez <1517449+friedrichg@users.noreply.github.com>

This was referenced May 31, 2026

Distributor signing keys: empty entry in -distributor.sign-write-requests-keys silently downgrades V01 stream auth to forgeable #7579

Open

Stream-push security test in integration/ingester_stream_push_test.go is effectively a no-op when signing fails #7580

Open

friedrichg changed the title ~~Backport security audit fixes (V01-V07) and snappy gRPC fix to release-1.21~~ Backport security audit fixes (V01-V07), snappy gRPC fix, and stream push panic fix to release-1.21 May 31, 2026

friedrichg requested a review from sandy2008 May 31, 2026 02:16

sandy2008 approved these changes May 31, 2026

View reviewed changes

SungJin1212 approved these changes May 31, 2026

View reviewed changes

dosubot Bot added the lgtm This PR has been approved by a maintainer label May 31, 2026

friedrichg merged commit 5f5273f into release-1.21 May 31, 2026
37 checks passed

friedrichg deleted the backport-security-audit-v01-v07 branch May 31, 2026 03:46

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Backport security audit fixes (V01-V07), snappy gRPC fix, and stream push panic fix to release-1.21#7574

Backport security audit fixes (V01-V07), snappy gRPC fix, and stream push panic fix to release-1.21#7574
friedrichg merged 15 commits into
release-1.21from
backport-security-audit-v01-v07

friedrichg commented May 29, 2026 •

edited

Loading

Uh oh!

This comment was marked as off-topic.

Uh oh!

SungJin1212 commented May 30, 2026

Uh oh!

sandy2008 left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

sandy2008 left a comment

Uh oh!

SungJin1212 left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Conversation

friedrichg commented May 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Backport status

V05 (#7570) conflict resolution notes

Test plan

Uh oh!

This comment was marked as off-topic.

Uh oh!

SungJin1212 commented May 30, 2026

Uh oh!

sandy2008 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

sandy2008 left a comment

Choose a reason for hiding this comment

Uh oh!

SungJin1212 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

friedrichg commented May 29, 2026 •

edited

Loading