Skip to content

Conversation

@nora-shap
Copy link
Member

The CommitFileChange query in _get_commit_file_changes is timing out (SENTRY-4596) due to:

  • Multiple LIKE clauses with leading wildcards (cannot use indexes)
  • Large IN clauses with 1000+ commit IDs
  • Complex combined query that the optimizer struggles to plan efficiently

Changes:

  • Batch commit IDs into groups of 100 to limit IN clause size
  • Split filename queries to eliminate complex OR conditions
  • Use set operations for efficient deduplication
  • Fetch IDs first, then bulk fetch objects to avoid large IN clauses

This reduces query complexity from a single query with 1000+ commit IDs and multiple OR conditions to multiple simpler queries that the PostgreSQL optimizer can handle efficiently.

Fixes SENTRY-4596

@nora-shap nora-shap requested a review from a team November 11, 2025 23:46
@github-actions github-actions bot added the Scope: Backend Automatically applied to PRs that change backend components label Nov 11, 2025
@JoshFerge
Copy link
Member

is this the same as #103170?

@nora-shap
Copy link
Member Author

going with #103170 instead

@nora-shap nora-shap closed this Nov 12, 2025
@codecov
Copy link

codecov bot commented Nov 12, 2025

❌ 1 Tests Failed:

Tests completed Failed Passed Skipped
29633 1 29632 243
View the top 1 failed test(s) by shortest run time
tests.relay_integration.lang.java.test_plugin.BasicResolvingIntegrationTest::test_resolving_inline
Stack Traces | 121s run time
#x1B[1m#x1B[31m.venv/lib/python3.13.../docker/api/client.py#x1B[0m:275: in _raise_for_status
    response.raise_for_status()
#x1B[1m#x1B[31m.venv/lib/python3.13....../site-packages/requests/models.py#x1B[0m:1026: in raise_for_status
    raise HTTPError(http_error_msg, response=self)
#x1B[1m#x1B[31mE   requests.exceptions.HTTPError: 404 Client Error: Not Found for url: http+docker:.../localhost/v1.48/containers/create?name=sentry_test_relay_server#x1B[0m

#x1B[33mThe above exception was the direct cause of the following exception:#x1B[0m
#x1B[1m#x1B[31m.venv/lib/python3.13.../docker/models/containers.py#x1B[0m:876: in run
    container = self.create(image=image, command=command,
#x1B[1m#x1B[31m.venv/lib/python3.13.../docker/models/containers.py#x1B[0m:935: in create
    resp = self.client.api.create_container(**create_kwargs)
#x1B[1m#x1B[31m.venv/lib/python3.13.../docker/api/container.py#x1B[0m:440: in create_container
    return self.create_container_from_config(config, name, platform)
#x1B[1m#x1B[31m.venv/lib/python3.13.../docker/api/container.py#x1B[0m:457: in create_container_from_config
    return self._result(res, True)
#x1B[1m#x1B[31m.venv/lib/python3.13.../docker/api/client.py#x1B[0m:281: in _result
    self._raise_for_status(response)
#x1B[1m#x1B[31m.venv/lib/python3.13.../docker/api/client.py#x1B[0m:277: in _raise_for_status
    raise create_api_error_from_http_exception(e) from e
#x1B[1m#x1B[31m.venv/lib/python3.13....../site-packages/docker/errors.py#x1B[0m:39: in create_api_error_from_http_exception
    raise cls(e, response=response, explanation=explanation) from e
#x1B[1m#x1B[31mE   docker.errors.ImageNotFound: 404 Client Error for http+docker:.../localhost/v1.48/containers/create?name=sentry_test_relay_server: Not Found ("No such image: ghcr.io/getsentry/relay:nightly")#x1B[0m

#x1B[33mDuring handling of the above exception, another exception occurred:#x1B[0m
#x1B[1m#x1B[31m.venv/lib/python3.13.../docker/api/client.py#x1B[0m:275: in _raise_for_status
    response.raise_for_status()
#x1B[1m#x1B[31m.venv/lib/python3.13....../site-packages/requests/models.py#x1B[0m:1026: in raise_for_status
    raise HTTPError(http_error_msg, response=self)
#x1B[1m#x1B[31mE   requests.exceptions.HTTPError: 500 Server Error: Internal Server Error for url: http+docker:.../localhost/v1.48/images/create?tag=nightly&fromImage=ghcr.io%2Fgetsentry%2Frelay#x1B[0m

#x1B[33mThe above exception was the direct cause of the following exception:#x1B[0m
#x1B[1m#x1B[.../testutils/pytest/relay.py#x1B[0m:137: in relay_server
    container = docker_client.containers.run(**options)
#x1B[1m#x1B[31m.venv/lib/python3.13.../docker/models/containers.py#x1B[0m:879: in run
    self.client.images.pull(image, platform=platform)
#x1B[1m#x1B[31m.venv/lib/python3.13.../docker/models/images.py#x1B[0m:464: in pull
    pull_log = self.client.api.pull(
#x1B[1m#x1B[31m.venv/lib/python3.13.../docker/api/image.py#x1B[0m:429: in pull
    self._raise_for_status(response)
#x1B[1m#x1B[31m.venv/lib/python3.13.../docker/api/client.py#x1B[0m:277: in _raise_for_status
    raise create_api_error_from_http_exception(e) from e
#x1B[1m#x1B[31m.venv/lib/python3.13....../site-packages/docker/errors.py#x1B[0m:39: in create_api_error_from_http_exception
    raise cls(e, response=response, explanation=explanation) from e
#x1B[1m#x1B[31mE   docker.errors.APIError: 500 Server Error for http+docker:.../localhost/v1.48/images/create?tag=nightly&fromImage=ghcr.io%2Fgetsentry%2Frelay: Internal Server Error ("Head "https://ghcr..../relay/manifests/nightly": dial tcp 140.82.113.34:443: i/o timeout")#x1B[0m

To view more test analytics, go to the Test Analytics Dashboard
📋 Got 3 mins? Take this short survey to help us improve Test Analytics.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Scope: Backend Automatically applied to PRs that change backend components

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants