Skip to content

fix(objectstore): Fix proxy in siloed mode#106172

Merged
lcian merged 17 commits intomasterfrom
lcian/fix/objectstore-headers
Jan 16, 2026
Merged

fix(objectstore): Fix proxy in siloed mode#106172
lcian merged 17 commits intomasterfrom
lcian/fix/objectstore-headers

Conversation

@lcian
Copy link
Member

@lcian lcian commented Jan 13, 2026

Fixes an issue where PUT and POST requests to the Objectstore proxy endpoint would send requests with an empty request body to the actual Objectstore service.

The issue was due to the usage of _body_with_length in src/sentry/hybridcloud/apigateway/proxy.py, which doesn't work for our endpoint.
I would love to understand the full reason behind that, but admittedly I don't, this was more of a trial and error process, where this solution appears to work, at least locally.

With this PR, we add a special case to the hybrid cloud proxy to use the same logic to stream the request body as we use in the Objectstore endpoint itself when the request would hit that endpoint.
That logic has been refactored into a function and slightly adapted to handle all cases.

This PR also adds a test that verifies that the endpoint works in control + region mode.
In addition, I've also tested this locally running both silos with granian and it works.

@github-actions github-actions bot added the Scope: Backend Automatically applied to PRs that change backend components label Jan 13, 2026
@lcian lcian changed the title wip fix(objectstore): Fix proxy in control + region silo mode Jan 14, 2026
Comment on lines +134 to +137
def tearDown(self) -> None:
for conn in connections.all():
conn.close()
super().tearDown()
Copy link
Member Author

@lcian lcian Jan 14, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adding this because I was getting

E   psycopg2.errors.ObjectInUse: database "test_test_test_test_test_test_region" is being accessed by other users
E   DETAIL:  There are 2 other sessions using the database.

in a previous CI run.
It feels that this is just putting a bandaid and not addressing the actual issue, but I'm also not familiar enough with our testing setup to understand what would be the proper fix.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is likely being caused by the the live_server that runs in a thread.

@lcian lcian changed the title fix(objectstore): Fix proxy in control + region silo mode fix(objectstore): Fix proxy in siloed mode Jan 14, 2026
@lcian lcian marked this pull request as ready for review January 14, 2026 16:58
@lcian lcian requested review from a team as code owners January 14, 2026 16:58
Comment on lines +134 to +137
def tearDown(self) -> None:
for conn in connections.all():
conn.close()
super().tearDown()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is likely being caused by the the live_server that runs in a thread.

Copy link
Member

@markstory markstory left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me.

@lcian lcian merged commit 11a75f3 into master Jan 16, 2026
97 of 99 checks passed
@lcian lcian deleted the lcian/fix/objectstore-headers branch January 16, 2026 10:06
@github-actions github-actions bot locked and limited conversation to collaborators Jan 31, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

Scope: Backend Automatically applied to PRs that change backend components

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants