Skip to content

Conversation

@alanprot
Copy link
Member

@alanprot alanprot commented Nov 5, 2025

What this PR does:
Follow up of #6808

  • Extended QueryRequest message with buffer reference management capabilities
  • Added defer req.Free() calls in QueryStream methods to properly release memory buffers
  • Added new BenchmarkGrpcCalls benchmark function to measure gRPC call performance and memory allocations

Upon high query load, we can see a bunch of memory being allocated by those requests.

Screenshot 2025-11-05 at 10 47 01 AM

Looking at the code, we can see that is we turn on compression, the min size of the byte[] allocated to decompress is 32kb (which can add up quite fast).

https://github.com/grpc/grpc-go/blob/7472d578b15f718cbe8ca0f5f5a3713093c47b03/rpc_util.go#L878
https://github.com/grpc/grpc-go/blob/7472d578b15f718cbe8ca0f5f5a3713093c47b03/rpc_util.go#L922
https://github.com/grpc/grpc-go/blob/7472d578b15f718cbe8ca0f5f5a3713093c47b03/mem/buffer_slice.go#L262
https://github.com/grpc/grpc-go/blob/7472d578b15f718cbe8ca0f5f5a3713093c47b03/mem/buffer_slice.go#L28

The benchmark before and after shows exactly the 32K less bytes being allocated per request.

Before:

BenchmarkGrpcCalls-10    	    4092	    277700 ns/op	  105624 B/op	     520 allocs/op

After:

BenchmarkGrpcCalls-10    	    4204	    248313 ns/op	   72962 B/op	     516 allocs/op

Which issue(s) this PR fixes:
Fixes #

Checklist

  • Tests updated
  • [NA] Documentation added
  • [NA] CHANGELOG.md updated - the order of entries should be [CHANGE], [FEATURE], [ENHANCEMENT], [BUGFIX]

Signed-off-by: alanprot <alanprot@gmail.com>
Copy link
Contributor

@yeya24 yeya24 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@yeya24 yeya24 merged commit f3113c9 into cortexproject:master Nov 5, 2025
33 of 34 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants