Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix backfill storing incorrect state for events #4718

Merged
merged 7 commits into from Feb 25, 2019

Conversation

Projects
None yet
3 participants
@erikjohnston
Copy link
Member

commented Feb 22, 2019

Broadly, we were going through marking all fetched "auth events" as outliers, even if they appeared in the main backfill chunk of events. This led to state getting very confused when we tried to persist events that referenced outliers.

@codecov-io

This comment has been minimized.

Copy link

commented Feb 22, 2019

Codecov Report

Merging #4718 into develop will increase coverage by 0.09%.
The diff coverage is 75%.

@@             Coverage Diff             @@
##           develop    #4718      +/-   ##
===========================================
+ Coverage    75.15%   75.25%   +0.09%     
===========================================
  Files          340      340              
  Lines        34823    34829       +6     
  Branches      5704     5704              
===========================================
+ Hits         26171    26209      +38     
+ Misses        7042     7019      -23     
+ Partials      1610     1601       -9

erikjohnston added some commits Feb 25, 2019

@erikjohnston erikjohnston requested a review from matrix-org/synapse-core Feb 25, 2019

@@ -786,13 +801,18 @@ def backfill(self, dest, room_id, limit, extremities):
})

for e_id in events_to_state:
# For paranoia we ensure that these events are marked as

This comment has been minimized.

Copy link
@richvdh

richvdh Feb 25, 2019

Member

this seems to be in conflict with the comment at line 775 that says "... state get persisted as outliers". What am I missing?

This comment has been minimized.

Copy link
@erikjohnston

erikjohnston Feb 25, 2019

Author Member

This needs more comments, obviously. The keys of events_to_state are the new backwards extremities, as those are the events that we fetched state for

This comment has been minimized.

Copy link
@richvdh

richvdh Feb 25, 2019

Member

it would seem a whole lot less opaque to iterate over edges rather than the keys of events_to_state, but ok.

# For paranoia we ensure that these events are marked as
# non-outliers
ev = event_map[e_id]
ev.internal_metadata.outlier = False

This comment has been minimized.

Copy link
@richvdh

richvdh Feb 25, 2019

Member

this sort of thing always alarms me slightly; if outlier was set doesn't it mean that something else has gone wrong? shouldn't we just raise an AssertionError or something?

This comment has been minimized.

Copy link
@erikjohnston

erikjohnston Feb 25, 2019

Author Member

Assert is actually probably what we want here, yes

@richvdh
Copy link
Member

left a comment

lgtm

@richvdh

This comment has been minimized.

Copy link
Member

commented Feb 25, 2019

[sad pep8 is sad]

@@ -0,0 +1 @@
Fix paginating over federation persisting incorrect state

This comment has been minimized.

Copy link
@richvdh

richvdh Feb 25, 2019

Member

needs a full-stop

erikjohnston added some commits Feb 25, 2019

@erikjohnston erikjohnston merged commit 69efe6f into develop Feb 25, 2019

5 checks passed

ci/circleci: sytestpy2merged Your tests passed on CircleCI!
Details
ci/circleci: sytestpy2postgresmerged Your tests passed on CircleCI!
Details
ci/circleci: sytestpy3merged Your tests passed on CircleCI!
Details
ci/circleci: sytestpy3postgresmerged Your tests passed on CircleCI!
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details

@erikjohnston erikjohnston deleted the erikj/fix_backfill_state_shred branch Mar 5, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.