Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use event streams to calculate presence #4942

Merged
merged 5 commits into from Mar 28, 2019

Conversation

Projects
None yet
2 participants
@erikjohnston
Copy link
Member

commented Mar 26, 2019

Primarily this fixes a bug in the handling of remote users joining a
room where the server sent out the presence for all local users in the
room to all servers in the room.

We also change to using the state delta stream, rather than the
distributor, as it will make it easier to split processing out of the
master process (as well as being more flexible).

Finally, when sending presence states to newly joined servers we filter
out old presence states to reduce the number sent. Initially we filter
out states that are offline and have a last active more than a week ago,
though this can be changed down the line.

Fixes #3962


Note that there a few TODO items, but I think they can be solved separately. Certainly I think what we have now is strictly better than what was there before.

@erikjohnston erikjohnston force-pushed the erikj/fix_presence branch from 0a63923 to b02b719 Mar 26, 2019

@codecov

This comment has been minimized.

Copy link

commented Mar 26, 2019

Codecov Report

Merging #4942 into develop will increase coverage by 0.07%.
The diff coverage is 66.3%.

@@             Coverage Diff             @@
##           develop    #4942      +/-   ##
===========================================
+ Coverage    60.44%   60.52%   +0.07%     
===========================================
  Files          331      328       -3     
  Lines        34100    34173      +73     
  Branches      5627     5647      +20     
===========================================
+ Hits         20612    20683      +71     
+ Misses       12017    12011       -6     
- Partials      1471     1479       +8

@erikjohnston erikjohnston requested a review from matrix-org/synapse-core Mar 26, 2019

@erikjohnston

This comment has been minimized.

Copy link
Member Author

commented Mar 26, 2019

I'm still figuring out quite how to test this fixes the bug, but at least we have sytests to check that the correct behaviour happens

erikjohnston added some commits Mar 26, 2019

Use event streams to calculate presence
Primarily this fixes a bug in the handling of remote users joining a
room where the server sent out the presence for all local users in the
room to all servers in the room.

We also change to using the state delta stream, rather than the
distributor, as it will make it easier to split processing out of the
master process (as well as being more flexible).

Finally, when sending presence states to newly joined servers we filter
out old presence states to reduce the number sent. Initially we filter
out states that are offline and have a last active more than a week ago,
though this can be changed down the line.

Fixes #3962

@erikjohnston erikjohnston force-pushed the erikj/fix_presence branch from f1be5cb to d5a5d1c Mar 27, 2019

@richvdh
Copy link
Member

left a comment

generally looks sane. a few nits.

Show resolved Hide resolved synapse/handlers/presence.py
Show resolved Hide resolved synapse/handlers/presence.py Outdated
Show resolved Hide resolved tests/handlers/test_presence.py Outdated
Show resolved Hide resolved tests/handlers/test_presence.py Outdated
Show resolved Hide resolved tests/handlers/test_presence.py Outdated
@@ -220,6 +220,15 @@ def __init__(self, hs):
LaterGauge("synapse_handlers_presence_wheel_timer_size", "", [],
lambda: len(self.wheel_timer))

# Used to handle sending of presence to newly joined users/servers
if hs.config.use_presence:
self.notifier.add_replication_callback(self.notify_new_event)

This comment has been minimized.

Copy link
@richvdh

richvdh Mar 28, 2019

Member

I'd love it if the stuff in the notifier could be renamed to be less about "replication", since we seem to be using it for things which aren't replication. Guess we can leave that for another time, though.

@erikjohnston erikjohnston requested a review from richvdh Mar 28, 2019

@richvdh
Copy link
Member

left a comment

lgtm otherwise

Show resolved Hide resolved synapse/handlers/presence.py Outdated

@erikjohnston erikjohnston force-pushed the erikj/fix_presence branch from 50a9e39 to 4e5f0f7 Mar 28, 2019

@erikjohnston erikjohnston merged commit 2480143 into develop Mar 28, 2019

22 checks passed

buildkite/synapse Build #665 passed (19 minutes, 44 seconds)
Details
buildkite/synapse/check-sample-config Passed (1 minute, 5 seconds)
Details
buildkite/synapse/isort Passed (15 seconds)
Details
buildkite/synapse/newspaper-newsfile Passed (14 seconds)
Details
buildkite/synapse/packaging Passed (16 seconds)
Details
buildkite/synapse/pep-8 Passed (54 seconds)
Details
buildkite/synapse/pipeline Passed (2 seconds)
Details
buildkite/synapse/python-2-dot-7-slash-postgres-9-dot-4 Passed (11 minutes, 7 seconds)
Details
buildkite/synapse/python-2-dot-7-slash-postgres-9-dot-5 Passed (10 minutes, 54 seconds)
Details
buildkite/synapse/python-2-dot-7-slash-sqlite Passed (6 minutes, 12 seconds)
Details
buildkite/synapse/python-2-dot-7-slash-sqlite-slash-old-deps Passed (8 minutes, 51 seconds)
Details
buildkite/synapse/python-3-dot-5-slash-postgres-9-dot-4 Passed (11 minutes, 45 seconds)
Details
buildkite/synapse/python-3-dot-5-slash-postgres-9-dot-5 Passed (12 minutes, 13 seconds)
Details
buildkite/synapse/python-3-dot-5-slash-sqlite Passed (7 minutes, 17 seconds)
Details
buildkite/synapse/python-3-dot-6-slash-sqlite Passed (6 minutes, 50 seconds)
Details
buildkite/synapse/python-3-dot-7-slash-postgres-11 Passed (12 minutes, 5 seconds)
Details
buildkite/synapse/python-3-dot-7-slash-postgres-9-dot-5 Passed (11 minutes, 46 seconds)
Details
buildkite/synapse/python-3-dot-7-slash-sqlite Passed (7 minutes, 14 seconds)
Details
ci/circleci: sytestpy2merged Your tests passed on CircleCI!
Details
ci/circleci: sytestpy2postgresmerged Your tests passed on CircleCI!
Details
ci/circleci: sytestpy3merged Your tests passed on CircleCI!
Details
ci/circleci: sytestpy3postgresmerged Your tests passed on CircleCI!
Details

erikjohnston added a commit that referenced this pull request Apr 10, 2019

Fix bug in presence handling deleted current state
Entries in current state can be "deleted", so we need to handle the case
where the new event_id is None.

Introduced in #4942.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.