Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix races in room stats (and other) updates. #6187

Merged
merged 5 commits into from Oct 10, 2019

Conversation

@richvdh
Copy link
Member

richvdh commented Oct 9, 2019

Hopefully this will fix the occasional failures we were seeing in the room directory.

The problem was that events are not necessarily persisted (and current_state_delta_stream updated) in the same order as their stream_id. So for instance current_state_delta 9 might be persisted before current_state_delta 8. Then, when the room stats saw stream_id 9, it assumed it had done everything up to 9, and never came back to do stream_id 8.

We can solve this easily by only processing up to the stream_id where we know all events have been persisted.

richvdh added 2 commits Oct 9, 2019
Hopefully this will fix the occasional failures we were seeing in the room
directory.

The problem was that events are not necessarily persisted (and
current_state_delta_stream updated) in the same order as their stream_id. So
for instance current_state_delta 9 might be persisted *before*
current_state_delta 8. Then, when the room stats saw stream_id 9, it assumed it
had done everything up to 9, and never came back to do stream_id 8.

We can solve this easily by only processing up to the stream_id where we know
all events have been persisted.
@richvdh richvdh requested a review from matrix-org/synapse-core Oct 9, 2019
@richvdh richvdh added this to In progress in Homeserver Task Board via automation Oct 9, 2019
@richvdh richvdh moved this from In progress to Review in Homeserver Task Board Oct 9, 2019
richvdh added 3 commits Oct 9, 2019
@richvdh richvdh merged commit a139420 into develop Oct 10, 2019
18 checks passed
18 checks passed
buildkite/synapse Build #4875 passed (19 minutes, 29 seconds)
Details
buildkite/synapse/check-sample-config Passed (1 minute, 45 seconds)
Details
buildkite/synapse/check-style Passed (1 minute, 44 seconds)
Details
buildkite/synapse/isort Passed (17 seconds)
Details
buildkite/synapse/mypy Passed (56 seconds)
Details
buildkite/synapse/newspaper-newsfile Passed (15 seconds)
Details
buildkite/synapse/packaging Passed (19 seconds)
Details
buildkite/synapse/pipeline Passed (3 seconds)
Details
buildkite/synapse/python-3-dot-5-slash-postgres-9-dot-5 Passed (17 minutes, 34 seconds)
Details
buildkite/synapse/python-3-dot-5-slash-sqlite Passed (6 minutes, 14 seconds)
Details
buildkite/synapse/python-3-dot-5-slash-sqlite-slash-old-deps Passed (7 minutes, 57 seconds)
Details
buildkite/synapse/python-3-dot-6-slash-sqlite Passed (6 minutes, 43 seconds)
Details
buildkite/synapse/python-3-dot-7-slash-postgres-11 Passed (17 minutes, 12 seconds)
Details
buildkite/synapse/python-3-dot-7-slash-postgres-9-dot-5 Passed (17 minutes, 21 seconds)
Details
buildkite/synapse/python-3-dot-7-slash-sqlite Passed (5 minutes, 12 seconds)
Details
buildkite/synapse/sytest-python-3-dot-5-slash-postgres-9-dot-6-slash-monolith Passed (11 minutes, 26 seconds)
Details
buildkite/synapse/sytest-python-3-dot-5-slash-postgres-9-dot-6-slash-workers Passed (11 minutes, 43 seconds)
Details
buildkite/synapse/sytest-python-3-dot-5-slash-sqlite-slash-monolith Passed (10 minutes, 19 seconds)
Details
Homeserver Task Board automation moved this from Review to Done Oct 10, 2019
@richvdh richvdh deleted the rav/fix_room_stats branch Oct 10, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
2 participants
You can’t perform that action at this time.