Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a lower bound to well-known TTL. #5836

merged 4 commits into from Aug 13, 2019


Copy link

commented Aug 8, 2019

Since we do the well-known lookup for every single request (even when we already have an available connection in the HTTP pool) we probably do always want to cache the lookup results for at least a short time, even if the remote has set don't cache headers on it.

This also includes moving the well-known lookup code to a separate class (in a separate commit), in preparation for some more changes.

erikjohnston added 3 commits Aug 6, 2019
It costs both us and the remote server for us to fetch the well known
for every single request we send, so we add a minimum cache period. This
is set to 5m so that we still honour the basic premise of "refetch
@erikjohnston erikjohnston force-pushed the erikj/lower_bound_ttl_well_known branch from 88d5b5e to 41546f9 Aug 12, 2019
@erikjohnston erikjohnston requested a review from matrix-org/synapse-core Aug 12, 2019
@erikjohnston erikjohnston marked this pull request as ready for review Aug 12, 2019
@erikjohnston erikjohnston added this to In progress in Homeserver Task Board via automation Aug 12, 2019
@erikjohnston erikjohnston changed the title Fix up Matrix server name resolution Add a lower bound to well-known cache time. Aug 12, 2019
@erikjohnston erikjohnston changed the title Add a lower bound to well-known cache time. Add a lower bound to well-known TTL. Aug 12, 2019
Copy link

left a comment

the only trouble with this is that I can't really see the functional changes amid the refactoring. I think it's sane modulo the below, though.

synapse/http/federation/ Outdated Show resolved Hide resolved

This comment has been minimized.

Copy link
Member Author

commented Aug 13, 2019

Sorry, thought having the refactor in a separate commit would be fine.

@erikjohnston erikjohnston merged commit e9906b0 into develop Aug 13, 2019
19 checks passed
19 checks passed
buildkite/synapse Build #3349 passed (12 minutes, 33 seconds)
buildkite/synapse/check-sample-config Passed (1 minute, 13 seconds)
buildkite/synapse/check-style Passed (1 minute, 14 seconds)
buildkite/synapse/isort Passed (21 seconds)
buildkite/synapse/newspaper-newsfile Passed (37 seconds)
buildkite/synapse/packaging Passed (20 seconds)
buildkite/synapse/pipeline Passed (9 seconds)
buildkite/synapse/python-3-dot-5-slash-postgres-9-dot-5 Passed (10 minutes, 50 seconds)
buildkite/synapse/python-3-dot-5-slash-sqlite Passed (3 minutes, 40 seconds)
buildkite/synapse/python-3-dot-5-slash-sqlite-slash-old-deps Passed (5 minutes, 4 seconds)
buildkite/synapse/python-3-dot-6-slash-sqlite Passed (3 minutes, 36 seconds)
buildkite/synapse/python-3-dot-7-slash-postgres-11 Passed (11 minutes, 4 seconds)
buildkite/synapse/python-3-dot-7-slash-postgres-9-dot-5 Passed (10 minutes, 35 seconds)
buildkite/synapse/python-3-dot-7-slash-sqlite Passed (3 minutes, 29 seconds)
buildkite/synapse/sytest-python-3-dot-5-slash-postgres-9-dot-6-slash-monolith Passed (5 minutes, 36 seconds)
buildkite/synapse/sytest-python-3-dot-5-slash-postgres-9-dot-6-slash-workers Passed (6 minutes, 16 seconds)
buildkite/synapse/sytest-python-3-dot-5-slash-sqlite-slash-monolith Passed (4 minutes, 32 seconds)
codecov/patch 90.9% of diff hit (target 0%)
codecov/project 63.41% (target 0%)
Homeserver Task Board automation moved this from In progress to Done Aug 13, 2019

This comment has been minimized.

Copy link

commented Aug 13, 2019

Sorry, thought having the refactor in a separate commit would be fine.

It might have been if I'd spotted that...

babolivier added a commit that referenced this pull request Aug 15, 2019
Synapse 1.3.0 (2019-08-15)


- Fix 500 Internal Server Error on `publicRooms` when the public room list was
  cached. ([\#5851](#5851))

Synapse 1.3.0rc1 (2019-08-13)


- Use `M_USER_DEACTIVATED` instead of `M_UNKNOWN` for errcode when a deactivated user attempts to login. ([\#5686](#5686))
- Add sd_notify hooks to ease systemd integration and allows usage of Type=Notify. ([\#5732](#5732))
- Synapse will no longer serve any media repo admin endpoints when `enable_media_repo` is set to False in the configuration. If a media repo worker is used, the admin APIs relating to the media repo will be served from it instead. ([\#5754](#5754), [\#5848](#5848))
- Synapse can now be configured to not join remote rooms of a given "complexity" (currently, state events) over federation. This option can be used to prevent adverse performance on resource-constrained homeservers. ([\#5783](#5783))
- Allow defining HTML templates to serve the user on account renewal attempt when using the account validity feature. ([\#5807](#5807))


- Fix UISIs during homeserver outage. ([\#5693](#5693), [\#5789](#5789))
- Fix stack overflow in server key lookup code. ([\#5724](#5724))
- no longer uses deprecated cli option. ([\#5725](#5725))
- Log when we receive an event receipt from an unexpected origin. ([\#5743](#5743))
- Fix debian packaging scripts to correctly build sid packages. ([\#5775](#5775))
- Correctly handle redactions of redactions. ([\#5788](#5788))
- Return 404 instead of 403 when accessing /rooms/{roomId}/event/{eventId} for an event without the appropriate permissions. ([\#5798](#5798))
- Fix check that tombstone is a state event in push rules. ([\#5804](#5804))
- Fix error when trying to login as a deactivated user when using a worker to handle login. ([\#5806](#5806))
- Fix bug where user `/sync` stream could get wedged in rare circumstances. ([\#5825](#5825))
- The script was fixed. ([\#5839](#5839))

Deprecations and Removals

- Synapse now no longer accepts the `-v`/`--verbose`, `-f`/`--log-file`, or `--log-config` command line flags, and removes the deprecated `verbose` and `log_file` configuration file options. Users of these options should migrate their options into the dedicated log configuration. ([\#5678](#5678), [\#5729](#5729))
- Remove non-functional 'expire_access_token' setting. ([\#5782](#5782))

Internal Changes

- Make Jaeger fully configurable. ([\#5694](#5694))
- Add precautionary measures to prevent future abuse of `window.opener` in default welcome page. ([\#5695](#5695))
- Reduce database IO usage by optimising queries for current membership. ([\#5706](#5706), [\#5738](#5738), [\#5746](#5746), [\#5752](#5752), [\#5770](#5770), [\#5774](#5774), [\#5792](#5792), [\#5793](#5793))
- Improve caching when fetching `get_filtered_current_state_ids`. ([\#5713](#5713))
- Don't accept opentracing data from clients. ([\#5715](#5715))
- Speed up PostgreSQL unit tests in CI. ([\#5717](#5717))
- Update the coding style document. ([\#5719](#5719))
- Improve database query performance when recording retry intervals for remote hosts. ([\#5720](#5720))
- Add a set of opentracing utils. ([\#5722](#5722))
- Cache result of get_version_string to reduce overhead of `/version` federation requests. ([\#5730](#5730))
- Return 'user_type' in admin API user endpoints results. ([\#5731](#5731))
- Don't package the sytest test blacklist file. ([\#5733](#5733))
- Replace uses of returnValue with plain return, as returnValue is not needed on Python 3. ([\#5736](#5736))
- Blacklist some flakey tests in worker mode. ([\#5740](#5740))
- Fix some error cases in the caching layer. ([\#5749](#5749))
- Add a prometheus metric for pending cache lookups. ([\#5750](#5750))
- Stop trying to fetch events with event_id=None. ([\#5753](#5753))
- Convert RedactionTestCase to modern test style. ([\#5768](#5768))
- Allow looping calls to be given arguments. ([\#5780](#5780))
- Set the logs emitted when checking typing and presence timeouts to DEBUG level, not INFO. ([\#5785](#5785))
- Remove DelayedCall debugging from the test suite, as it is no longer required in the vast majority of Synapse's tests. ([\#5787](#5787))
- Remove some spurious exceptions from the logs where we failed to talk to a remote server. ([\#5790](#5790))
- Improve performance when making `.well-known` requests by sharing the SSL options between requests. ([\#5794](#5794))
- Disable codecov GitHub comments on PRs. ([\#5796](#5796))
- Don't allow clients to send tombstone events that reference the room it's sent in. ([\#5801](#5801))
- Deny redactions of events sent in a different room. ([\#5802](#5802))
- Deny sending well known state types as non-state events. ([\#5805](#5805))
- Handle incorrectly encoded query params correctly by returning a 400. ([\#5808](#5808))
- Handle pusher being deleted during processing rather than logging an exception. ([\#5809](#5809))
- Return 502 not 500 when failing to reach any remote server. ([\#5810](#5810))
- Reduce global pauses in the events stream caused by expensive state resolution during persistence. ([\#5826](#5826))
- Add a lower bound to well-known lookup cache time to avoid repeated lookups. ([\#5836](#5836))
- Whitelist history visbility sytests in worker mode tests. ([\#5843](#5843))
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
2 participants
You can’t perform that action at this time.