New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

don't store more remote device lists if they have more than 1K devices #4397

Merged
merged 5 commits into from Jan 16, 2019

Conversation

Projects
None yet
4 participants
@richvdh
Copy link
Member

richvdh commented Jan 15, 2019

Backport of #4396 to develop.

If the remote server has more than ~1000 devices for this user we assume that something is going horribly wrong (e.g. a bot that logs in and creates a new device every time it tries to send a message). Maintaining lots of devices per user in the cache can cause serious performance issues as if this request takes more than 60s to complete, internal replication from the inbound federation worker to the synapse master may time out causing the inbound federation to fail and causing the remote server to retry, causing a DoS. So in this scenario we give up on storing the total list of devices and only handle the delta instead.

@richvdh richvdh requested a review from matrix-org/synapse-core Jan 15, 2019

@codecov-io

This comment has been minimized.

Copy link

codecov-io commented Jan 15, 2019

Codecov Report

Merging #4397 into develop will decrease coverage by <.01%.
The diff coverage is 0%.

Impacted file tree graph

@@             Coverage Diff             @@
##           develop    #4397      +/-   ##
===========================================
- Coverage    73.65%   73.65%   -0.01%     
===========================================
  Files          300      300              
  Lines        29815    29818       +3     
  Branches      4897     4898       +1     
===========================================
+ Hits         21960    21961       +1     
  Misses        6414     6414              
- Partials      1441     1443       +2
Impacted Files Coverage Δ
synapse/handlers/device.py 79.68% <0%> (-0.97%) ⬇️
synapse/handlers/search.py 80.24% <0%> (ø) ⬆️
synapse/handlers/federation.py 61.72% <0%> (ø) ⬆️
synapse/handlers/user_directory.py 71.38% <0%> (+0.3%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update aa955f2...3b31303. Read the comment docs.

@erikjohnston
Copy link
Member

erikjohnston left a comment

LGTM, though we should also probably do something so that synapse won't get to the stage of having so many devices for a user, somehow.

@richvdh richvdh merged commit 05e1296 into develop Jan 16, 2019

5 checks passed

ci/circleci: sytestpy2merged Your tests passed on CircleCI!
Details
ci/circleci: sytestpy2postgresmerged Your tests passed on CircleCI!
Details
ci/circleci: sytestpy3merged Your tests passed on CircleCI!
Details
ci/circleci: sytestpy3postgresmerged Your tests passed on CircleCI!
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details

@richvdh richvdh deleted the rav/bodge_device_update_dos branch Jan 16, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment