Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix the SQL SELECT query in _paginate_room_events_txn #6340

Merged
merged 6 commits into from Nov 8, 2019

Conversation

@babolivier
Copy link
Member

babolivier commented Nov 7, 2019

Doing a SELECT DISTINCT when paginating is quite expensive, because it requires the engine to do sorting on the entire events table. However, we only need to run it if we're filtering on 2+ labels, so this PR is changing the request so that DISTINCT is only used then.

Fixes #6337

babolivier added 2 commits Nov 7, 2019
Doing a SELECT DISTINCT when paginating is quite expensive, because it requires the engine to do sorting on the entire events table. However, we only need to run it if we're filtering on 2+ labels, so this PR is changing the request so that DISTINCT is only used then.
@babolivier babolivier requested a review from matrix-org/synapse-core Nov 7, 2019
babolivier added 3 commits Nov 7, 2019
@richvdh

This comment has been minimized.

Copy link
Member

richvdh commented Nov 7, 2019

fixes #6337?

@babolivier babolivier mentioned this pull request Nov 7, 2019
4 of 5 tasks complete
@babolivier

This comment has been minimized.

Copy link
Member Author

babolivier commented Nov 7, 2019

fixes #6337?

Ah, yes, I missed the issue

Copy link
Member

erikjohnston left a comment

Let's get this merged to unbreak develop

synapse/storage/data_stores/main/stream.py Outdated Show resolved Hide resolved
synapse/storage/data_stores/main/stream.py Outdated Show resolved Hide resolved
@babolivier babolivier merged commit 963ffb6 into develop Nov 8, 2019
20 checks passed
20 checks passed
buildkite/synapse Build #5484 passed (20 minutes, 55 seconds)
Details
buildkite/synapse/check-sample-config Passed (1 minute, 31 seconds)
Details
buildkite/synapse/check-style Passed (1 minute, 56 seconds)
Details
buildkite/synapse/isort Passed (17 seconds)
Details
buildkite/synapse/mypy Passed (26 seconds)
Details
buildkite/synapse/newspaper-newsfile Passed (15 seconds)
Details
buildkite/synapse/packaging Passed (18 seconds)
Details
buildkite/synapse/pipeline Passed (3 seconds)
Details
buildkite/synapse/python-3-dot-5-slash-postgres-9-dot-5 Passed (18 minutes, 49 seconds)
Details
buildkite/synapse/python-3-dot-5-slash-sqlite Passed (7 minutes, 13 seconds)
Details
buildkite/synapse/python-3-dot-5-slash-sqlite-slash-old-deps Passed (9 minutes, 15 seconds)
Details
buildkite/synapse/python-3-dot-6-slash-sqlite Passed (7 minutes, 3 seconds)
Details
buildkite/synapse/python-3-dot-7-slash-postgres-11 Passed (18 minutes, 21 seconds)
Details
buildkite/synapse/python-3-dot-7-slash-postgres-9-dot-5 Passed (18 minutes, 34 seconds)
Details
buildkite/synapse/python-3-dot-7-slash-sqlite Passed (6 minutes, 26 seconds)
Details
buildkite/synapse/synapse-port-db-slash-python-3-dot-5-slash-postgres-9-dot-5 Passed (1 minute, 42 seconds)
Details
buildkite/synapse/synapse-port-db-slash-python-3-dot-7-slash-postgres-11 Passed (1 minute, 38 seconds)
Details
buildkite/synapse/sytest-python-3-dot-5-slash-postgres-9-dot-6-slash-monolith Passed (14 minutes, 29 seconds)
Details
buildkite/synapse/sytest-python-3-dot-5-slash-postgres-9-dot-6-slash-workers Passed (15 minutes, 57 seconds)
Details
buildkite/synapse/sytest-python-3-dot-5-slash-sqlite-slash-monolith Passed (15 minutes, 38 seconds)
Details
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants
You can’t perform that action at this time.