Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update the device list cache when keys/query is called #5693

Merged
merged 7 commits into from Jul 29, 2019

Conversation

@JorikSchellekens
Copy link
Contributor

commented Jul 16, 2019

Fixes #5441

The device_lists_remote_cache is not updated when making calls to client/keys/query. This caused devices to be removed when a remote homeserver was not responding. Now the cache is updated on this call. However, doing so for every call could needlessly put devices into the cache, this is a rare case but it has been addressed by only updating the cache if the request doesn't specify a particular device and the requesting user shares a room with the queried users.

@JorikSchellekens JorikSchellekens requested a review from matrix-org/synapse-core Jul 16, 2019

@JorikSchellekens

This comment has been minimized.

Copy link
Contributor Author

commented Jul 16, 2019

This relies on some utils in #5687 which haven't landed yet but they should do shortly. I've removed that dependence so that this can move along.

@JorikSchellekens JorikSchellekens changed the base branch from develop to joriks/opentracing_e2e Jul 17, 2019

@codecov

This comment has been minimized.

Copy link

commented Jul 17, 2019

Codecov Report

Merging #5693 into joriks/opentracing_e2e will decrease coverage by 39.23%.
The diff coverage is 6.18%.

@@                     Coverage Diff                     @@
##           joriks/opentracing_e2e    #5693       +/-   ##
===========================================================
- Coverage                   62.93%   23.69%   -39.24%     
===========================================================
  Files                         331      290       -41     
  Lines                       36375    32007     -4368     
  Branches                     5962     5221      -741     
===========================================================
- Hits                        22892     7585    -15307     
- Misses                      11827    24322    +12495     
+ Partials                     1656      100     -1556
@codecov

This comment has been minimized.

Copy link

commented Jul 17, 2019

Codecov Report

Merging #5693 into develop will decrease coverage by 0.03%.
The diff coverage is 5.66%.

@@             Coverage Diff             @@
##           develop    #5693      +/-   ##
===========================================
- Coverage    63.34%   63.31%   -0.04%     
===========================================
  Files          331      331              
  Lines        36393    36419      +26     
  Branches      6006     6013       +7     
===========================================
+ Hits         23054    23058       +4     
- Misses       11700    11723      +23     
+ Partials      1639     1638       -1

@JorikSchellekens JorikSchellekens changed the base branch from joriks/opentracing_e2e to develop Jul 17, 2019

@JorikSchellekens JorikSchellekens force-pushed the joriks/uisi_fix branch 2 times, most recently from a9289cc to 5de6656 Jul 17, 2019

@richvdh richvdh added this to In progress in Homeserver Task Board via automation Jul 18, 2019

@erikjohnston
Copy link
Member

left a comment

This looks good, just needs some comments to explain why we're doing what we're doing

synapse/handlers/e2e_keys.py Outdated Show resolved Hide resolved
synapse/handlers/e2e_keys.py Outdated Show resolved Hide resolved
synapse/handlers/e2e_keys.py Outdated Show resolved Hide resolved
synapse/handlers/e2e_keys.py Show resolved Hide resolved
synapse/handlers/e2e_keys.py Outdated Show resolved Hide resolved

@JorikSchellekens JorikSchellekens force-pushed the joriks/uisi_fix branch from fb384cb to 85a625d Jul 18, 2019

@JorikSchellekens

This comment has been minimized.

Copy link
Contributor Author

commented Jul 18, 2019

I wouldn't merge this until there's a sytest for it.

@JorikSchellekens JorikSchellekens force-pushed the joriks/uisi_fix branch from 85a625d to 1047e7f Jul 18, 2019

@JorikSchellekens JorikSchellekens requested a review from erikjohnston Jul 18, 2019

@erikjohnston
Copy link
Member

left a comment

Woop, but like you said lets wait for a sytest before merging.

synapse/handlers/device.py Outdated Show resolved Hide resolved
synapse/handlers/device.py Outdated Show resolved Hide resolved
synapse/handlers/e2e_keys.py Outdated Show resolved Hide resolved

@JorikSchellekens JorikSchellekens force-pushed the joriks/uisi_fix branch from e7cf779 to 52d1393 Jul 24, 2019

JorikSchellekens and others added 5 commits Jul 17, 2019
typo
Co-Authored-By: Erik Johnston <erik@matrix.org>
Docstrings
Co-Authored-By: Erik Johnston <erik@matrix.org>

@JorikSchellekens JorikSchellekens force-pushed the joriks/uisi_fix branch from 52d1393 to 1e3882b Jul 24, 2019

@JorikSchellekens

This comment has been minimized.

Copy link
Contributor Author

commented Jul 24, 2019

The test has been implemented but it still needs a review.
matrix-org/sytest#659

@erikjohnston erikjohnston merged commit 85b0bd8 into develop Jul 29, 2019

19 checks passed

buildkite/synapse Build #3057 passed (20 minutes, 8 seconds)
Details
buildkite/synapse/check-sample-config Passed (1 minute, 14 seconds)
Details
buildkite/synapse/check-style Passed (2 minutes, 29 seconds)
Details
buildkite/synapse/isort Passed (20 seconds)
Details
buildkite/synapse/newspaper-newsfile Passed (32 seconds)
Details
buildkite/synapse/packaging Passed (21 seconds)
Details
buildkite/synapse/pipeline Passed (16 seconds)
Details
buildkite/synapse/python-3-dot-5-slash-postgres-9-dot-5 Passed (17 minutes, 16 seconds)
Details
buildkite/synapse/python-3-dot-5-slash-sqlite Passed (4 minutes, 38 seconds)
Details
buildkite/synapse/python-3-dot-5-slash-sqlite-slash-old-deps Passed (10 minutes, 52 seconds)
Details
buildkite/synapse/python-3-dot-6-slash-sqlite Passed (10 minutes, 6 seconds)
Details
buildkite/synapse/python-3-dot-7-slash-postgres-11 Passed (11 minutes, 54 seconds)
Details
buildkite/synapse/python-3-dot-7-slash-postgres-9-dot-5 Passed (11 minutes, 51 seconds)
Details
buildkite/synapse/python-3-dot-7-slash-sqlite Passed (4 minutes, 32 seconds)
Details
buildkite/synapse/sytest-python-3-dot-5-slash-postgres-9-dot-6-slash-monolith Passed (6 minutes, 4 seconds)
Details
buildkite/synapse/sytest-python-3-dot-5-slash-postgres-9-dot-6-slash-workers Passed (6 minutes, 24 seconds)
Details
buildkite/synapse/sytest-python-3-dot-5-slash-sqlite-slash-monolith Passed (4 minutes, 31 seconds)
Details
codecov/patch 5.66% of diff hit (target 0%)
Details
codecov/project 63.31% (target 0%)
Details

Homeserver Task Board automation moved this from In progress to Done Jul 29, 2019

@JorikSchellekens JorikSchellekens deleted the joriks/uisi_fix branch Jul 29, 2019

babolivier added a commit that referenced this pull request Aug 15, 2019
Merge tag 'v1.3.0'
Synapse 1.3.0 (2019-08-15)
==========================

Bugfixes
--------

- Fix 500 Internal Server Error on `publicRooms` when the public room list was
  cached. ([\#5851](#5851))

Synapse 1.3.0rc1 (2019-08-13)
==========================

Features
--------

- Use `M_USER_DEACTIVATED` instead of `M_UNKNOWN` for errcode when a deactivated user attempts to login. ([\#5686](#5686))
- Add sd_notify hooks to ease systemd integration and allows usage of Type=Notify. ([\#5732](#5732))
- Synapse will no longer serve any media repo admin endpoints when `enable_media_repo` is set to False in the configuration. If a media repo worker is used, the admin APIs relating to the media repo will be served from it instead. ([\#5754](#5754), [\#5848](#5848))
- Synapse can now be configured to not join remote rooms of a given "complexity" (currently, state events) over federation. This option can be used to prevent adverse performance on resource-constrained homeservers. ([\#5783](#5783))
- Allow defining HTML templates to serve the user on account renewal attempt when using the account validity feature. ([\#5807](#5807))

Bugfixes
--------

- Fix UISIs during homeserver outage. ([\#5693](#5693), [\#5789](#5789))
- Fix stack overflow in server key lookup code. ([\#5724](#5724))
- start.sh no longer uses deprecated cli option. ([\#5725](#5725))
- Log when we receive an event receipt from an unexpected origin. ([\#5743](#5743))
- Fix debian packaging scripts to correctly build sid packages. ([\#5775](#5775))
- Correctly handle redactions of redactions. ([\#5788](#5788))
- Return 404 instead of 403 when accessing /rooms/{roomId}/event/{eventId} for an event without the appropriate permissions. ([\#5798](#5798))
- Fix check that tombstone is a state event in push rules. ([\#5804](#5804))
- Fix error when trying to login as a deactivated user when using a worker to handle login. ([\#5806](#5806))
- Fix bug where user `/sync` stream could get wedged in rare circumstances. ([\#5825](#5825))
- The purge_remote_media.sh script was fixed. ([\#5839](#5839))

Deprecations and Removals
-------------------------

- Synapse now no longer accepts the `-v`/`--verbose`, `-f`/`--log-file`, or `--log-config` command line flags, and removes the deprecated `verbose` and `log_file` configuration file options. Users of these options should migrate their options into the dedicated log configuration. ([\#5678](#5678), [\#5729](#5729))
- Remove non-functional 'expire_access_token' setting. ([\#5782](#5782))

Internal Changes
----------------

- Make Jaeger fully configurable. ([\#5694](#5694))
- Add precautionary measures to prevent future abuse of `window.opener` in default welcome page. ([\#5695](#5695))
- Reduce database IO usage by optimising queries for current membership. ([\#5706](#5706), [\#5738](#5738), [\#5746](#5746), [\#5752](#5752), [\#5770](#5770), [\#5774](#5774), [\#5792](#5792), [\#5793](#5793))
- Improve caching when fetching `get_filtered_current_state_ids`. ([\#5713](#5713))
- Don't accept opentracing data from clients. ([\#5715](#5715))
- Speed up PostgreSQL unit tests in CI. ([\#5717](#5717))
- Update the coding style document. ([\#5719](#5719))
- Improve database query performance when recording retry intervals for remote hosts. ([\#5720](#5720))
- Add a set of opentracing utils. ([\#5722](#5722))
- Cache result of get_version_string to reduce overhead of `/version` federation requests. ([\#5730](#5730))
- Return 'user_type' in admin API user endpoints results. ([\#5731](#5731))
- Don't package the sytest test blacklist file. ([\#5733](#5733))
- Replace uses of returnValue with plain return, as returnValue is not needed on Python 3. ([\#5736](#5736))
- Blacklist some flakey tests in worker mode. ([\#5740](#5740))
- Fix some error cases in the caching layer. ([\#5749](#5749))
- Add a prometheus metric for pending cache lookups. ([\#5750](#5750))
- Stop trying to fetch events with event_id=None. ([\#5753](#5753))
- Convert RedactionTestCase to modern test style. ([\#5768](#5768))
- Allow looping calls to be given arguments. ([\#5780](#5780))
- Set the logs emitted when checking typing and presence timeouts to DEBUG level, not INFO. ([\#5785](#5785))
- Remove DelayedCall debugging from the test suite, as it is no longer required in the vast majority of Synapse's tests. ([\#5787](#5787))
- Remove some spurious exceptions from the logs where we failed to talk to a remote server. ([\#5790](#5790))
- Improve performance when making `.well-known` requests by sharing the SSL options between requests. ([\#5794](#5794))
- Disable codecov GitHub comments on PRs. ([\#5796](#5796))
- Don't allow clients to send tombstone events that reference the room it's sent in. ([\#5801](#5801))
- Deny redactions of events sent in a different room. ([\#5802](#5802))
- Deny sending well known state types as non-state events. ([\#5805](#5805))
- Handle incorrectly encoded query params correctly by returning a 400. ([\#5808](#5808))
- Handle pusher being deleted during processing rather than logging an exception. ([\#5809](#5809))
- Return 502 not 500 when failing to reach any remote server. ([\#5810](#5810))
- Reduce global pauses in the events stream caused by expensive state resolution during persistence. ([\#5826](#5826))
- Add a lower bound to well-known lookup cache time to avoid repeated lookups. ([\#5836](#5836))
- Whitelist history visbility sytests in worker mode tests. ([\#5843](#5843))
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
2 participants
You can’t perform that action at this time.