MSC3983: Sending One-Time Key (OTK) claims to appservices #3983

turt2live · 2023-03-23T18:41:37Z

Implementations:

bot-sdk: Support MSC3983: Keys Claim request turt2live/matrix-bot-sdk#307
Synapse: Implement MSC3983 to proxy /keys/claim queries to appservices. synapse#15314
Actual appservice which does something with this: ??? (possible effort would be NodeJS bindings: Support for MSC3983 matrix-rust-sdk-crypto-nodejs#29 )

proposals/3983-sending-otk-claims-to-appservices.md

See matrix-org/matrix-spec-proposals#3983

* Support MSC3983: Keys Claim request See matrix-org/matrix-spec-proposals#3983 * Include test code * Appease the linter

proposals/3983-sending-otk-claims-to-appservices.md

dkasak · 2023-03-27T14:59:37Z

proposals/3983-sending-otk-claims-to-appservices.md

+In this case, the appservice is responsible for ensuring it doesn't use a key twice. The
+`device_one_time_keys_count` field for the appservice (over MSC3202, for example) would be zero. In
+many implementations, when this field falls below a threshold it is common for upload requests to
+happen: appservices intending on using the new API should not perform those uploads as it means,
+quite simply, not using the new API.


I'm having a hard time understanding what this paragraph is attempting to say.

Is it simply trying to say that implementations implementing MSC3983 should ignore device_one_time_keys_count?

It's more of a warning to existing implementations of crypto: nearly all of them do a if (count < 50) { generateAndUploadKeys() } call, which would be a problem here.

Hmm sure, but what are implementations supposed to do instead? That's the part I'm unclear about from the above.

"not that" :p

Normally implementation details like this wouldn't be called out, however given the chance for everyone to have the exact same bug (intending to use new API, uploads keys by accident, new API isn't used), this is called out here.

dkasak · 2023-03-27T16:11:32Z

proposals/3983-sending-otk-claims-to-appservices.md

+will be bad), they can simply upload keys in advance, just like before this proposal. Similarly, if
+the appservice is trying to use the API but is offline, they *should* have a fallback key to continue
+using as, well, a fallback.


Who is offline in this sentence? And which API are we talking about? Also, who is they?

This whole MSC is in context of an appservice.

I gathered as much 😄 But I'm still having trouble parsing that sentence in exact terms. e.g. If I interpret "but is offline" as "but the appservice is offline", how is the appservice able to do anything given it's offline?

I'd suggest rewriting the sentence with less implicit words / pronouns.

The appservice is opting into using the API by not uploading keys, effectively. This is it "using" the API, which might be the confusing part?

dkasak · 2023-03-27T16:14:37Z

proposals/3983-sending-otk-claims-to-appservices.md

+
+An appservice aiming to bridge two different encryption systems might use this endpoint to save on data,
+though currently the encryption used on both sides of the bridge would need to be compatible (ie: signatures
+from device IDs and user IDs need to exist). In future, other MSCs might make encryption bridges easier to


Suggested change

from device IDs and user IDs need to exist). In future, other MSCs might make encryption bridges easier to

from device and user identities need to exist). In the future, other MSCs might make encryption bridges easier to

Device IDs and user IDs can't produce signatures, so the proposed change would make it a bit clearer to me.

clokep · 2023-04-13T18:27:31Z

proposals/3983-sending-otk-claims-to-appservices.md

+If the appservice responds with an error of any kind (including timeout), the homeserver uses the
+fallback key, if known. The homeserver additionally uses the fallback key (if known) to fill in
+missing keys from the appservice. For example, if the homeserver requested 2 keys for Alice but
+the appservice only provided 1, the homeserver would use the fallback key to fulfill the second.


I'm left wondering why the appservice would bother uploaded fallback keys when (to my reading) it could return them directly as part of the response.

I suppose uploading them would still be useful for error conditions, however.

clokep · 2023-04-19T19:43:03Z

proposals/3983-sending-otk-claims-to-appservices.md

+Independent of the appservice having `/keys/claim` proxied to it, it may be desireable for both the
+fallback and one-time key to be returned. Servers should *always* include the fallback key alongside
+the requested OTKs. When using this proposal's new endpoint, the server should use the fallback key
+from the appservice's response rather than a previously stored fallback key, if present (if the
+appservice doesn't respond with a fallback key then the server uses the stored fallback key instead,
+if known). 


Is this enough of a change that it need to be a v4 of the endpoint, perhaps? (Or a query parameter which defaults to the current behavior?)

I think it'd be fine, though open to thoughts. Currently clients appear to pick the "first" key returned, not validating if the extra data is useful to them. In theory this means we can include extra keys with no issues, though if there's a significant need to have it be a dedicated endpoint then let's do that.

We should probably only do this fallback stuff on an /unstable/org.matrix.msc3983 endpoint though, just in case.

matrix-org/synapse#15462 implements this as a separate endpoint.

Another piece of this I realized during implementation is if we need to also make a change on the federation side.

clokep · 2023-04-20T14:39:45Z

proposals/3983-sending-otk-claims-to-appservices.md

+the requested OTKs. When using this proposal's new endpoint, the server should use the fallback key
+from the appservice's response rather than a previously stored fallback key, if present (if the
+appservice doesn't respond with a fallback key then the server uses the stored fallback key instead,
+if known). 


This doesn't define if the appservice should be queried for fallback keys if a OTK is in the database.

In lieu of MSC text: the appservice is supposed to be queried, similar to MSC3984 where the server uses what it knows if the appservice didn't provide it.

Actually I suppose the text in the MSC is enough as is:

For users under an appservice's explicit namespace, if that user has no unused OTKs (excluding fallback keys) on the homeserver, the homeserver proxies the following APIs to the appservice using the new API described below:

This would not match what you just wrote though and would mean it is not possible for an appservice to only provide fallback keys. I'm not sure if that's a good thing or a bad thing though. 🤷

If we really do want that then I think we want to use different endpoints or something. Otherwise the appservice wouldn't know if we truly want to query for OTKs or only for fallback keys (it could end up returning OTKs which go unused?)

clokep · 2023-04-24T16:40:34Z

proposals/3983-sending-otk-claims-to-appservices.md

+Independent of the appservice having `/keys/claim` proxied to it, it may be desirable for both the
+fallback and one-time key to be returned. Servers should *always* include the fallback key alongside


In addition to both fallback and OTKs, @dkasak suggested that it could be helpful to request multiple keys at once over the C-S API. In particular having 0 or 1 keys of multiple algorithms could be useful.

I implemented this using the same unstable endpoint by accepting a map of {"<algo>": <count>} instead of a single algorithm. I'm now seeing that the above has an array of algorithms to pass to the appservice instead though. I guess I should consolidate those to be the same though.

I implemented this using the same unstable endpoint by accepting a map of {"<algo>": <count>} instead of a single algorithm. I'm now seeing that the above has an array of algorithms to pass to the appservice instead though. I guess I should consolidate those to be the same though.

The Synapse PR was updated to use the same form sent to appservices (i.e. a list of algorithms).

We'll probably want to add something saying that clients may need to deal with getting back fewer than the requested numbers of keys and that servers should protect against users request too many keys at once to avoid exhaustion.

@Fizzadar

Synapse 1.83.0 (2023-05-09) =========================== No significant changes since 1.83.0rc1. Synapse 1.83.0rc1 (2023-05-02) ============================== Features -------- - Experimental support to recursively provide relations per [MSC3981](matrix-org/matrix-spec-proposals#3981). ([\matrix-org#15315](matrix-org#15315)) - Experimental support for [MSC3970](matrix-org/matrix-spec-proposals#3970): Scope transaction IDs to devices. ([\matrix-org#15318](matrix-org#15318)) - Add an [admin API endpoint](https://matrix-org.github.io/synapse/v1.83/admin_api/experimental_features.html) to support per-user feature flags. ([\matrix-org#15344](matrix-org#15344)) - Add a module API to send an HTTP push notification. ([\matrix-org#15387](matrix-org#15387)) - Add an [admin API endpoint](https://matrix-org.github.io/synapse/v1.83/admin_api/statistics.html#get-largest-rooms-by-size-in-database) to query the largest rooms by disk space used in the database. ([\matrix-org#15482](matrix-org#15482)) Bugfixes -------- - Disable push rule evaluation for rooms excluded from sync. ([\matrix-org#15361](matrix-org#15361)) - Fix a long-standing bug where cached server key results which were directly fetched would not be properly re-used. ([\matrix-org#15417](matrix-org#15417)) - Fix a bug introduced in Synapse 1.73.0 where some experimental push rules were returned by default. ([\matrix-org#15494](matrix-org#15494)) Improved Documentation ---------------------- - Add Nginx loadbalancing example with sticky mxid for workers. ([\matrix-org#15411](matrix-org#15411)) - Update outdated development docs that mention restrictions in versions of SQLite that we no longer support. ([\matrix-org#15498](matrix-org#15498)) Internal Changes ---------------- - Speedup tests by caching HomeServerConfig instances. ([\matrix-org#15284](matrix-org#15284)) - Add denormalised event stream ordering column to membership state tables for future use. Contributed by Nick @ Beeper (@Fizzadar). ([\matrix-org#15356](matrix-org#15356)) - Always use multi-user device resync replication endpoints. ([\matrix-org#15418](matrix-org#15418)) - Add column `full_user_id` to tables `profiles` and `user_filters`. ([\matrix-org#15458](matrix-org#15458)) - Update support for [MSC3983](matrix-org/matrix-spec-proposals#3983) to allow always returning fallback-keys in a `/keys/claim` request. ([\matrix-org#15462](matrix-org#15462)) - Improve type hints. ([\matrix-org#15465](matrix-org#15465), [\matrix-org#15496](matrix-org#15496), [\matrix-org#15497](matrix-org#15497)) - Support claiming more than one OTK at a time. ([\matrix-org#15468](matrix-org#15468)) - Bump types-pyyaml from 6.0.12.8 to 6.0.12.9. ([\matrix-org#15471](matrix-org#15471)) - Bump pyasn1-modules from 0.2.8 to 0.3.0. ([\matrix-org#15473](matrix-org#15473)) - Bump cryptography from 40.0.1 to 40.0.2. ([\matrix-org#15474](matrix-org#15474)) - Bump types-netaddr from 0.8.0.7 to 0.8.0.8. ([\matrix-org#15475](matrix-org#15475)) - Bump types-jsonschema from 4.17.0.6 to 4.17.0.7. ([\matrix-org#15476](matrix-org#15476)) - Ask bug reporters to provide logs as text. ([\matrix-org#15479](matrix-org#15479)) - Add a Nix flake for use as a development environment. ([\matrix-org#15495](matrix-org#15495)) - Bump anyhow from 1.0.70 to 1.0.71. ([\matrix-org#15507](matrix-org#15507)) - Bump types-pillow from 9.4.0.19 to 9.5.0.2. ([\matrix-org#15508](matrix-org#15508)) - Bump packaging from 23.0 to 23.1. ([\matrix-org#15510](matrix-org#15510)) - Bump types-requests from 2.28.11.16 to 2.29.0.0. ([\matrix-org#15511](matrix-org#15511)) - Bump setuptools-rust from 1.5.2 to 1.6.0. ([\matrix-org#15512](matrix-org#15512)) - Update the check_schema_delta script to account for when the schema version has been bumped locally. ([\matrix-org#15466](matrix-org#15466)) # -----BEGIN PGP SIGNATURE----- # # iHUEABYIAB0WIQSTI7xPaHQ1yo0PA8uSL1esuTqr+QUCZFp2zQAKCRCSL1esuTqr # +YVzAP94u5kth80qhAB5x/vpCMrpPZC8Hk4kPTNwTuqCia7qGQEAlcEgsNXRmLnw # esBnwhQE+ntwodK4uvHKjVZafkDG2gs= # =N1Pk # -----END PGP SIGNATURE----- # gpg: Signature made Tue May 9 17:37:33 2023 BST # gpg: using EDDSA key 9323BC4F687435CA8D0F03CB922F57ACB93AABF9 # gpg: Can't check signature: No public key # Conflicts: # poetry.lock # synapse/events/utils.py # synapse/push/bulk_push_rule_evaluator.py # synapse/push/httppusher.py # synapse/server.py

@Fizzadar

Synapse 1.86.0 (2023-06-20) =========================== No significant changes since 1.86.0rc2. Synapse 1.86.0rc2 (2023-06-14) ============================== Bugfixes -------- - Fix an error when having workers of different versions running. ([\#15774](matrix-org/synapse#15774)) Synapse 1.86.0rc1 (2023-06-13) ============================== This version was tagged but never released. Features -------- - Stable support for [MSC3882](matrix-org/matrix-spec-proposals#3882) to allow an existing device/session to generate a login token for use on a new device/session. ([\#15388](matrix-org/synapse#15388)) - Support resolving a room's [canonical alias](https://spec.matrix.org/v1.7/client-server-api/#mroomcanonical_alias) via the module API. ([\#15450](matrix-org/synapse#15450)) - Enable support for [MSC3952](matrix-org/matrix-spec-proposals#3952): intentional mentions. ([\#15520](matrix-org/synapse#15520)) - Experimental [MSC3861](matrix-org/matrix-spec-proposals#3861) support: delegate auth to an OIDC provider. ([\#15582](matrix-org/synapse#15582)) - Add Synapse version deploy annotations to Grafana dashboard which enables easy correlation between behavior changes witnessed in a graph to a certain Synapse version and nail down regressions. ([\#15674](matrix-org/synapse#15674)) - Add a catch-all * to the supported relation types when redacting an event and its related events. This is an update to [MSC3912](matrix-org/matrix-spec-proposals#3861) implementation. ([\#15705](matrix-org/synapse#15705)) - Speed up `/messages` by backfilling in the background when there are no backward extremities where we are directly paginating. ([\#15710](matrix-org/synapse#15710)) - Expose a metric reporting the database background update status. ([\#15740](matrix-org/synapse#15740)) Bugfixes -------- - Correctly clear caches when we delete a room. ([\#15609](matrix-org/synapse#15609)) - Check permissions for enabling encryption earlier during room creation to avoid creating broken rooms. ([\#15695](matrix-org/synapse#15695)) Improved Documentation ---------------------- - Simplify query to find participating servers in a room. ([\#15732](matrix-org/synapse#15732)) Internal Changes ---------------- - Log when events are (maybe unexpectedly) filtered out of responses in tests. ([\#14213](matrix-org/synapse#14213)) - Read from column `full_user_id` rather than `user_id` of tables `profiles` and `user_filters`. ([\#15649](matrix-org/synapse#15649)) - Add support for tracing functions which return `Awaitable`s. ([\#15650](matrix-org/synapse#15650)) - Cache requests for user's devices over federation. ([\#15675](matrix-org/synapse#15675)) - Add fully qualified docker image names to Dockerfiles. ([\#15689](matrix-org/synapse#15689)) - Remove some unused code. ([\#15690](matrix-org/synapse#15690)) - Improve type hints. ([\#15694](matrix-org/synapse#15694), [\#15697](matrix-org/synapse#15697)) - Update docstring and traces on `maybe_backfill()` functions. ([\#15709](matrix-org/synapse#15709)) - Add context for when/why to use the `long_retries` option when sending Federation requests. ([\#15721](matrix-org/synapse#15721)) - Removed some unused fields. ([\#15723](matrix-org/synapse#15723)) - Update federation error to more plainly explain we can only authorize our own membership events. ([\#15725](matrix-org/synapse#15725)) - Prevent the `latest_deps` and `twisted_trunk` daily GitHub Actions workflows from running on forks of the codebase. ([\#15726](matrix-org/synapse#15726)) - Improve performance of user directory search. ([\#15729](matrix-org/synapse#15729)) - Remove redundant table join with `room_memberships` when doing a `is_host_joined()`/`is_host_invited()` call (`membership` is already part of the `current_state_events`). ([\#15731](matrix-org/synapse#15731)) - Remove superfluous `room_memberships` join from background update. ([\#15733](matrix-org/synapse#15733)) - Speed up typechecking CI. ([\#15752](matrix-org/synapse#15752)) - Bump minimum supported Rust version to 1.60.0. ([\#15768](matrix-org/synapse#15768)) ### Updates to locked dependencies * Bump importlib-metadata from 6.1.0 to 6.6.0. ([\#15711](matrix-org/synapse#15711)) * Bump library/redis from 6-bullseye to 7-bullseye in /docker. ([\#15712](matrix-org/synapse#15712)) * Bump log from 0.4.18 to 0.4.19. ([\#15761](matrix-org/synapse#15761)) * Bump phonenumbers from 8.13.11 to 8.13.13. ([\#15763](matrix-org/synapse#15763)) * Bump pyasn1 from 0.4.8 to 0.5.0. ([\#15713](matrix-org/synapse#15713)) * Bump pydantic from 1.10.8 to 1.10.9. ([\#15762](matrix-org/synapse#15762)) * Bump pyo3-log from 0.8.1 to 0.8.2. ([\#15759](matrix-org/synapse#15759)) * Bump pyopenssl from 23.1.1 to 23.2.0. ([\#15765](matrix-org/synapse#15765)) * Bump regex from 1.7.3 to 1.8.4. ([\#15769](matrix-org/synapse#15769)) * Bump sentry-sdk from 1.22.1 to 1.25.0. ([\#15714](matrix-org/synapse#15714)) * Bump sentry-sdk from 1.25.0 to 1.25.1. ([\#15764](matrix-org/synapse#15764)) * Bump serde from 1.0.163 to 1.0.164. ([\#15760](matrix-org/synapse#15760)) * Bump types-jsonschema from 4.17.0.7 to 4.17.0.8. ([\#15716](matrix-org/synapse#15716)) * Bump types-pyopenssl from 23.1.0.2 to 23.2.0.0. ([\#15766](matrix-org/synapse#15766)) * Bump types-requests from 2.31.0.0 to 2.31.0.1. ([\#15715](matrix-org/synapse#15715)) Synapse 1.85.2 (2023-06-08) =========================== Bugfixes -------- - Fix regression where using TLS for HTTP replication between workers did not work. Introduced in v1.85.0. ([\#15746](matrix-org/synapse#15746)) Synapse 1.85.1 (2023-06-07) =========================== Note: this release only fixes a bug that stopped some deployments from upgrading to v1.85.0. There is no need to upgrade to v1.85.1 if successfully running v1.85.0. Bugfixes -------- - Fix bug in schema delta that broke upgrades for some deployments. Introduced in v1.85.0. ([\#15738](matrix-org/synapse#15738), [\#15739](matrix-org/synapse#15739)) Synapse 1.85.0 (2023-06-06) =========================== No significant changes since 1.85.0rc2. ## Security advisory The following issues are fixed in 1.85.0 (and RCs). - [GHSA-26c5-ppr8-f33p](GHSA-26c5-ppr8-f33p) / [CVE-2023-32682](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-32682) — Low Severity It may be possible for a deactivated user to login when using uncommon configurations. - [GHSA-98px-6486-j7qc](GHSA-98px-6486-j7qc) / [CVE-2023-32683](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-32683) — Low Severity A discovered oEmbed or image URL can bypass the `url_preview_url_blacklist` setting potentially allowing server side request forgery or bypassing network policies. Impact is limited to IP addresses allowed by the `url_preview_ip_range_blacklist` setting (by default this only allows public IPs). See the advisories for more details. If you have any questions, email security@matrix.org. Synapse 1.85.0rc2 (2023-06-01) ============================== Bugfixes -------- - Fix a performance issue introduced in Synapse v1.83.0 which meant that purging rooms was very slow and database-intensive. ([\#15693](matrix-org/synapse#15693)) Deprecations and Removals ------------------------- - Deprecate calling the `/register` endpoint with an unspecced `user` property for application services. ([\#15703](matrix-org/synapse#15703)) Internal Changes ---------------- - Speed up background jobs `populate_full_user_id_user_filters` and `populate_full_user_id_profiles`. ([\#15700](matrix-org/synapse#15700)) Synapse 1.85.0rc1 (2023-05-30) ============================== Features -------- - Improve performance of backfill requests by performing backfill of previously failed requests in the background. ([\#15585](matrix-org/synapse#15585)) - Add a new [admin API](https://matrix-org.github.io/synapse/v1.85/usage/administration/admin_api/index.html) to [create a new device for a user](https://matrix-org.github.io/synapse/v1.85/admin_api/user_admin_api.html#create-a-device). ([\#15611](matrix-org/synapse#15611)) - Add Unix socket support for Redis connections. Contributed by Jason Little. ([\#15644](matrix-org/synapse#15644)) Bugfixes -------- - Fix a long-standing bug where setting the read marker could fail when using message retention. Contributed by Nick @ Beeper (@Fizzadar). ([\#15464](matrix-org/synapse#15464)) - Fix a long-standing bug where the `url_preview_url_blacklist` configuration setting was not applied to oEmbed or image URLs found while previewing a URL. ([\#15601](matrix-org/synapse#15601)) - Fix a long-standing bug where filters with multiple backslashes were rejected. ([\#15607](matrix-org/synapse#15607)) - Fix a bug introduced in Synapse 1.82.0 where the error message displayed when validation of the `app_service_config_files` config option fails would be incorrectly formatted. ([\#15614](matrix-org/synapse#15614)) - Fix a long-standing bug where deactivated users were still able to login using the custom `org.matrix.login.jwt` login type (if enabled). ([\#15624](matrix-org/synapse#15624)) - Fix a long-standing bug where deactivated users were able to login in uncommon situations. ([\#15634](matrix-org/synapse#15634)) Improved Documentation ---------------------- - Warn users that at least 3.75GB of space is needed for the nix Synapse development environment. ([\#15613](matrix-org/synapse#15613)) - Remove outdated comment from the generated and sample homeserver log configs. ([\#15648](matrix-org/synapse#15648)) - Improve contributor docs to make it more clear that Rust is a necessary prerequisite. Contributed by @grantm. ([\#15668](matrix-org/synapse#15668)) Deprecations and Removals ------------------------- - Remove the old version of the R30 (30-day retained users) phone-home metric. ([\#10428](matrix-org/synapse#10428)) Internal Changes ---------------- - Create dependabot changelogs at release time. ([\#15481](matrix-org/synapse#15481)) - Add not null constraint to column `full_user_id` of tables `profiles` and `user_filters`. ([\#15537](matrix-org/synapse#15537)) - Allow connecting to HTTP Replication Endpoints by using `worker_name` when constructing the request. ([\#15578](matrix-org/synapse#15578)) - Make the `thread_id` column on `event_push_actions`, `event_push_actions_staging`, and `event_push_summary` non-null. ([\#15597](matrix-org/synapse#15597)) - Run mypy type checking with the minimum supported Python version to catch new usage that isn't backwards-compatible. ([\#15602](matrix-org/synapse#15602)) - Fix subscriptable type usage in Python <3.9. ([\#15604](matrix-org/synapse#15604)) - Update internal terminology. ([\#15606](matrix-org/synapse#15606), [\#15620](matrix-org/synapse#15620)) - Instrument `state` and `state_group` storage-related operations to better picture what's happening when tracing. ([\#15610](matrix-org/synapse#15610), [\#15647](matrix-org/synapse#15647)) - Trace how many new events from the backfill response we need to process. ([\#15633](matrix-org/synapse#15633)) - Re-type config paths in `ConfigError`s to be `StrSequence`s instead of `Iterable[str]`s. ([\#15615](matrix-org/synapse#15615)) - Update Mutual Rooms ([MSC2666](matrix-org/matrix-spec-proposals#2666)) implementation to match new proposal text. ([\#15621](matrix-org/synapse#15621)) - Remove the unstable identifiers from faster joins ([MSC3706](matrix-org/matrix-spec-proposals#3706)). ([\#15625](matrix-org/synapse#15625)) - Fix the olddeps CI. ([\#15626](matrix-org/synapse#15626)) - Remove duplicate timestamp from test logs (`_trial_temp/test.log`). ([\#15636](matrix-org/synapse#15636)) - Fix two memory leaks in `trial` test runs. ([\#15630](matrix-org/synapse#15630)) - Limit the size of the `HomeServerConfig` cache in trial test runs. ([\#15646](matrix-org/synapse#15646)) - Improve type hints. ([\#15658](matrix-org/synapse#15658), [\#15659](matrix-org/synapse#15659)) - Add requesting user id parameter to key claim methods in `TransportLayerClient`. ([\#15663](matrix-org/synapse#15663)) - Speed up rebuilding of the user directory for local users. ([\#15665](matrix-org/synapse#15665)) - Implement "option 2" for [MSC3820](matrix-org/matrix-spec-proposals#3820): Room version 11. ([\#15666](matrix-org/synapse#15666), [\#15678](matrix-org/synapse#15678)) ### Updates to locked dependencies * Bump furo from 2023.3.27 to 2023.5.20. ([\#15642](matrix-org/synapse#15642)) * Bump log from 0.4.17 to 0.4.18. ([\#15681](matrix-org/synapse#15681)) * Bump prometheus-client from 0.16.0 to 0.17.0. ([\#15682](matrix-org/synapse#15682)) * Bump pydantic from 1.10.7 to 1.10.8. ([\#15685](matrix-org/synapse#15685)) * Bump pygithub from 1.58.1 to 1.58.2. ([\#15643](matrix-org/synapse#15643)) * Bump requests from 2.28.2 to 2.31.0. ([\#15651](matrix-org/synapse#15651)) * Bump sphinx from 6.1.3 to 6.2.1. ([\#15641](matrix-org/synapse#15641)) * Bump types-bleach from 6.0.0.1 to 6.0.0.3. ([\#15686](matrix-org/synapse#15686)) * Bump types-pillow from 9.5.0.2 to 9.5.0.4. ([\#15640](matrix-org/synapse#15640)) * Bump types-pyyaml from 6.0.12.9 to 6.0.12.10. ([\#15683](matrix-org/synapse#15683)) * Bump types-requests from 2.30.0.0 to 2.31.0.0. ([\#15684](matrix-org/synapse#15684)) * Bump types-setuptools from 67.7.0.2 to 67.8.0.0. ([\#15639](matrix-org/synapse#15639)) Synapse 1.84.1 (2023-05-26) =========================== This patch release fixes a major issue with homeservers that do not have an `instance_map` defined but which do use workers. If you have already upgraded to Synapse 1.84.0 and your homeserver is working normally, then there is no need to update to this patch release. Bugfixes -------- - Fix a bug introduced in Synapse v1.84.0 where workers do not start up when no `instance_map` was provided. ([\#15672](matrix-org/synapse#15672)) Internal Changes ---------------- - Add `dch` and `notify-send` to the development Nix flake so that the release script can be used. ([\#15673](matrix-org/synapse#15673)) Synapse 1.84.0 (2023-05-23) =========================== The `worker_replication_*` configuration settings have been deprecated in favour of configuring the main process consistently with other instances in the `instance_map`. The deprecated settings will be removed in Synapse v1.88.0, but changing your configuration in advance is recommended. See the [upgrade notes](https://github.com/matrix-org/synapse/blob/release-v1.84/docs/upgrade.md#upgrading-to-v1840) for more information. Bugfixes -------- - Fix a bug introduced in Synapse 1.84.0rc1 where errors during startup were not reported correctly on Python < 3.10. ([\#15599](matrix-org/synapse#15599)) Synapse 1.84.0rc1 (2023-05-16) ============================== Features -------- - Add an option to prevent media downloads from configured domains. ([\#15197](matrix-org/synapse#15197)) - Add `forget_rooms_on_leave` config option to automatically forget rooms when users leave them or are removed from them. ([\#15224](matrix-org/synapse#15224)) - Add redis TLS configuration options. ([\#15312](matrix-org/synapse#15312)) - Add a config option to delay push notifications by a random amount, to discourage time-based profiling. ([\#15516](matrix-org/synapse#15516)) - Stabilize support for [MSC2659](matrix-org/matrix-spec-proposals#2659): application service ping endpoint. Contributed by Tulir @ Beeper. ([\#15528](matrix-org/synapse#15528)) - Implement [MSC4009](matrix-org/matrix-spec-proposals#4009) to expand the supported characters in Matrix IDs. ([\#15536](matrix-org/synapse#15536)) - Advertise support for Matrix 1.6 on `/_matrix/client/versions`. ([\#15559](matrix-org/synapse#15559)) - Print full error and stack-trace of any exception that occurs during startup/initialization. ([\#15569](matrix-org/synapse#15569)) Bugfixes -------- - Don't fail on federation over TOR where SRV queries are not supported. Contributed by Zdzichu. ([\#15523](matrix-org/synapse#15523)) - Experimental support for [MSC4010](matrix-org/matrix-spec-proposals#4010) which rejects setting the `"m.push_rules"` via account data. ([\#15554](matrix-org/synapse#15554), [\#15555](matrix-org/synapse#15555)) - Fix a long-standing bug where an invalid membership event could cause an internal server error. ([\#15564](matrix-org/synapse#15564)) - Require at least poetry-core v1.1.0. ([\#15566](matrix-org/synapse#15566), [\#15571](matrix-org/synapse#15571)) Deprecations and Removals ------------------------- - Remove need for `worker_replication_*` based settings in worker configuration yaml by placing this data directly on the `instance_map` instead. ([\#15491](matrix-org/synapse#15491)) Updates to the Docker image --------------------------- - Add pkg-config package to Stage 0 to be able to build Dockerfile on ppc64le architecture. ([\#15567](matrix-org/synapse#15567)) Improved Documentation ---------------------- - Clarify documentation of the "Create or modify account" Admin API. ([\#15544](matrix-org/synapse#15544)) - Fix path to the `statistics/database/rooms` admin API in documentation. ([\#15560](matrix-org/synapse#15560)) - Update and improve Mastodon Single Sign-On documentation. ([\#15587](matrix-org/synapse#15587)) Internal Changes ---------------- - Use oEmbed to generate URL previews for YouTube Shorts. ([\#15025](matrix-org/synapse#15025)) - Create new `Client` for use with HTTP Replication between workers. Contributed by Jason Little. ([\#15470](matrix-org/synapse#15470)) - Bump pyicu from 2.10.2 to 2.11. ([\#15509](matrix-org/synapse#15509)) - Remove references to supporting per-user flag for [MSC2654](matrix-org/matrix-spec-proposals#2654). ([\#15522](matrix-org/synapse#15522)) - Don't use a trusted key server when running the demo scripts. ([\#15527](matrix-org/synapse#15527)) - Speed up rebuilding of the user directory for local users. ([\#15529](matrix-org/synapse#15529)) - Speed up deleting of old rows in `event_push_actions`. ([\#15531](matrix-org/synapse#15531)) - Install the `xmlsec` and `mdbook` packages and switch back to the upstream [cachix/devenv](https://github.com/cachix/devenv) repo in the nix development environment. ([\#15532](matrix-org/synapse#15532), [\#15533](matrix-org/synapse#15533), [\#15545](matrix-org/synapse#15545)) - Implement [MSC3987](matrix-org/matrix-spec-proposals#3987) by removing `"dont_notify"` from the list of actions in default push rules. ([\#15534](matrix-org/synapse#15534)) - Move various module API callback registration methods to a dedicated class. ([\#15535](matrix-org/synapse#15535)) - Proxy `/user/devices` federation queries to application services for [MSC3984](matrix-org/matrix-spec-proposals#3984). ([\#15539](matrix-org/synapse#15539)) - Factor out an `is_mine_server_name` method. ([\#15542](matrix-org/synapse#15542)) - Allow running Complement tests using [podman](https://podman.io/) by adding a `PODMAN` environment variable to `scripts-dev/complement.sh`. ([\#15543](matrix-org/synapse#15543)) - Bump serde from 1.0.160 to 1.0.162. ([\#15548](matrix-org/synapse#15548)) - Bump types-setuptools from 67.6.0.5 to 67.7.0.1. ([\#15549](matrix-org/synapse#15549)) - Bump sentry-sdk from 1.19.1 to 1.22.1. ([\#15550](matrix-org/synapse#15550)) - Bump ruff from 0.0.259 to 0.0.265. ([\#15551](matrix-org/synapse#15551)) - Bump hiredis from 2.2.2 to 2.2.3. ([\#15552](matrix-org/synapse#15552)) - Bump types-requests from 2.29.0.0 to 2.30.0.0. ([\#15553](matrix-org/synapse#15553)) - Add `org.matrix.msc3981` info to `/_matrix/client/versions`. ([\#15558](matrix-org/synapse#15558)) - Declare unstable support for [MSC3391](matrix-org/matrix-spec-proposals#3391) under `/_matrix/client/versions` if the experimental implementation is enabled. ([\#15562](matrix-org/synapse#15562)) - Implement [MSC3821](matrix-org/matrix-spec-proposals#3821) to update the redaction rules. ([\#15563](matrix-org/synapse#15563)) - Implement updated redaction rules from [MSC3389](matrix-org/matrix-spec-proposals#3389). ([\#15565](matrix-org/synapse#15565)) - Allow `pip install` to use setuptools_rust 1.6.0 when building Synapse. ([\#15570](matrix-org/synapse#15570)) - Deal with upcoming Github Actions deprecations. ([\#15576](matrix-org/synapse#15576)) - Export `run_as_background_process` from the module API. ([\#15577](matrix-org/synapse#15577)) - Update build system requirements to allow building with poetry-core==1.6.0. ([\#15588](matrix-org/synapse#15588)) - Bump serde from 1.0.162 to 1.0.163. ([\#15589](matrix-org/synapse#15589)) - Bump phonenumbers from 8.13.7 to 8.13.11. ([\#15590](matrix-org/synapse#15590)) - Bump types-psycopg2 from 2.9.21.9 to 2.9.21.10. ([\#15591](matrix-org/synapse#15591)) - Bump types-commonmark from 0.9.2.2 to 0.9.2.3. ([\#15592](matrix-org/synapse#15592)) - Bump types-setuptools from 67.7.0.1 to 67.7.0.2. ([\#15594](matrix-org/synapse#15594)) Synapse 1.83.0 (2023-05-09) =========================== No significant changes since 1.83.0rc1. Synapse 1.83.0rc1 (2023-05-02) ============================== Features -------- - Experimental support to recursively provide relations per [MSC3981](matrix-org/matrix-spec-proposals#3981). ([\#15315](matrix-org/synapse#15315)) - Experimental support for [MSC3970](matrix-org/matrix-spec-proposals#3970): Scope transaction IDs to devices. ([\#15318](matrix-org/synapse#15318)) - Add an [admin API endpoint](https://matrix-org.github.io/synapse/v1.83/admin_api/experimental_features.html) to support per-user feature flags. ([\#15344](matrix-org/synapse#15344)) - Add a module API to send an HTTP push notification. ([\#15387](matrix-org/synapse#15387)) - Add an [admin API endpoint](https://matrix-org.github.io/synapse/v1.83/admin_api/statistics.html#get-largest-rooms-by-size-in-database) to query the largest rooms by disk space used in the database. ([\#15482](matrix-org/synapse#15482)) Bugfixes -------- - Disable push rule evaluation for rooms excluded from sync. ([\#15361](matrix-org/synapse#15361)) - Fix a long-standing bug where cached server key results which were directly fetched would not be properly re-used. ([\#15417](matrix-org/synapse#15417)) - Fix a bug introduced in Synapse 1.73.0 where some experimental push rules were returned by default. ([\#15494](matrix-org/synapse#15494)) Improved Documentation ---------------------- - Add Nginx loadbalancing example with sticky mxid for workers. ([\#15411](matrix-org/synapse#15411)) - Update outdated development docs that mention restrictions in versions of SQLite that we no longer support. ([\#15498](matrix-org/synapse#15498)) Internal Changes ---------------- - Speedup tests by caching HomeServerConfig instances. ([\#15284](matrix-org/synapse#15284)) - Add denormalised event stream ordering column to membership state tables for future use. Contributed by Nick @ Beeper (@Fizzadar). ([\#15356](matrix-org/synapse#15356)) - Always use multi-user device resync replication endpoints. ([\#15418](matrix-org/synapse#15418)) - Add column `full_user_id` to tables `profiles` and `user_filters`. ([\#15458](matrix-org/synapse#15458)) - Update support for [MSC3983](matrix-org/matrix-spec-proposals#3983) to allow always returning fallback-keys in a `/keys/claim` request. ([\#15462](matrix-org/synapse#15462)) - Improve type hints. ([\#15465](matrix-org/synapse#15465), [\#15496](matrix-org/synapse#15496), [\#15497](matrix-org/synapse#15497)) - Support claiming more than one OTK at a time. ([\#15468](matrix-org/synapse#15468)) - Bump types-pyyaml from 6.0.12.8 to 6.0.12.9. ([\#15471](matrix-org/synapse#15471)) - Bump pyasn1-modules from 0.2.8 to 0.3.0. ([\#15473](matrix-org/synapse#15473)) - Bump cryptography from 40.0.1 to 40.0.2. ([\#15474](matrix-org/synapse#15474)) - Bump types-netaddr from 0.8.0.7 to 0.8.0.8. ([\#15475](matrix-org/synapse#15475)) - Bump types-jsonschema from 4.17.0.6 to 4.17.0.7. ([\#15476](matrix-org/synapse#15476)) - Ask bug reporters to provide logs as text. ([\#15479](matrix-org/synapse#15479)) - Add a Nix flake for use as a development environment. ([\#15495](matrix-org/synapse#15495)) - Bump anyhow from 1.0.70 to 1.0.71. ([\#15507](matrix-org/synapse#15507)) - Bump types-pillow from 9.4.0.19 to 9.5.0.2. ([\#15508](matrix-org/synapse#15508)) - Bump packaging from 23.0 to 23.1. ([\#15510](matrix-org/synapse#15510)) - Bump types-requests from 2.28.11.16 to 2.29.0.0. ([\#15511](matrix-org/synapse#15511)) - Bump setuptools-rust from 1.5.2 to 1.6.0. ([\#15512](matrix-org/synapse#15512)) - Update the check_schema_delta script to account for when the schema version has been bumped locally. ([\#15466](matrix-org/synapse#15466)) Synapse 1.82.0 (2023-04-25) =========================== No significant changes since 1.82.0rc1. Synapse 1.82.0rc1 (2023-04-18) ============================== Features -------- - Allow loading the `/directory/room/{roomAlias}` endpoint on workers. ([\#15333](matrix-org/synapse#15333)) - Add some validation to `instance_map` configuration loading. ([\#15431](matrix-org/synapse#15431)) - Allow loading the `/capabilities` endpoint on workers. ([\#15436](matrix-org/synapse#15436)) Bugfixes -------- - Delete server-side backup keys when deactivating an account. ([\#15181](matrix-org/synapse#15181)) - Fix and document untold assumption that `on_logged_out` module hooks will be called before the deletion of pushers. ([\#15410](matrix-org/synapse#15410)) - Improve robustness when handling a perspective key response by deduplicating received server keys. ([\#15423](matrix-org/synapse#15423)) - Synapse now correctly fails to start if the config option `app_service_config_files` is not a list. ([\#15425](matrix-org/synapse#15425)) - Disable loading `RefreshTokenServlet` (`/_matrix/client/(r0|v3|unstable)/refresh`) on workers. ([\#15428](matrix-org/synapse#15428)) Improved Documentation ---------------------- - Note that the `delete_stale_devices_after` background job always runs on the main process. ([\#15452](matrix-org/synapse#15452)) Deprecations and Removals ------------------------- - Remove the broken, unspecced registration fallback. Note that the *login* fallback is unaffected by this change. ([\#15405](matrix-org/synapse#15405)) Internal Changes ---------------- - Bump black from 23.1.0 to 23.3.0. ([\#15372](matrix-org/synapse#15372)) - Bump pyopenssl from 23.1.0 to 23.1.1. ([\#15373](matrix-org/synapse#15373)) - Bump types-psycopg2 from 2.9.21.8 to 2.9.21.9. ([\#15374](matrix-org/synapse#15374)) - Bump types-netaddr from 0.8.0.6 to 0.8.0.7. ([\#15375](matrix-org/synapse#15375)) - Bump types-opentracing from 2.4.10.3 to 2.4.10.4. ([\#15376](matrix-org/synapse#15376)) - Bump dawidd6/action-download-artifact from 2.26.0 to 2.26.1. ([\#15404](matrix-org/synapse#15404)) - Bump parameterized from 0.8.1 to 0.9.0. ([\#15412](matrix-org/synapse#15412)) - Bump types-pillow from 9.4.0.17 to 9.4.0.19. ([\#15413](matrix-org/synapse#15413)) - Bump sentry-sdk from 1.17.0 to 1.19.1. ([\#15414](matrix-org/synapse#15414)) - Bump immutabledict from 2.2.3 to 2.2.4. ([\#15415](matrix-org/synapse#15415)) - Bump dawidd6/action-download-artifact from 2.26.1 to 2.27.0. ([\#15441](matrix-org/synapse#15441)) - Bump serde_json from 1.0.95 to 1.0.96. ([\#15442](matrix-org/synapse#15442)) - Bump serde from 1.0.159 to 1.0.160. ([\#15443](matrix-org/synapse#15443)) - Bump pillow from 9.4.0 to 9.5.0. ([\#15444](matrix-org/synapse#15444)) - Bump furo from 2023.3.23 to 2023.3.27. ([\#15445](matrix-org/synapse#15445)) - Bump types-pyopenssl from 23.1.0.0 to 23.1.0.2. ([\#15446](matrix-org/synapse#15446)) - Bump mypy from 1.0.0 to 1.0.1. ([\#15447](matrix-org/synapse#15447)) - Bump psycopg2 from 2.9.5 to 2.9.6. ([\#15448](matrix-org/synapse#15448)) - Improve DB performance of clearing out old data from `stream_ordering_to_exterm`. ([\#15382](matrix-org/synapse#15382), [\#15429](matrix-org/synapse#15429)) - Implement [MSC3989](matrix-org/matrix-spec-proposals#3989) redaction algorithm. ([\#15393](matrix-org/synapse#15393)) - Implement [MSC2175](matrix-org/matrix-spec-proposals#2175) to stop adding `creator` to create events. ([\#15394](matrix-org/synapse#15394)) - Implement [MSC2174](matrix-org/matrix-spec-proposals#2174) to move the `redacts` key to a `content` property. ([\#15395](matrix-org/synapse#15395)) - Trust dtonlay/rust-toolchain in CI. ([\#15406](matrix-org/synapse#15406)) - Explicitly install Synapse during typechecking in CI. ([\#15409](matrix-org/synapse#15409)) - Only load the SSO redirect servlet if SSO is enabled. ([\#15421](matrix-org/synapse#15421)) - Refactor `SimpleHttpClient` to pull out a base class. ([\#15427](matrix-org/synapse#15427)) - Improve type hints. ([\#15432](matrix-org/synapse#15432)) - Convert async to normal tests in `TestSSOHandler`. ([\#15433](matrix-org/synapse#15433)) - Speed up the user directory background update. ([\#15435](matrix-org/synapse#15435)) - Disable directory listing for static resources in `/_matrix/static/`. ([\#15438](matrix-org/synapse#15438)) - Move various module API callback registration methods to a dedicated class. ([\#15453](matrix-org/synapse#15453)) Synapse 1.81.0 (2023-04-11) =========================== Synapse now attempts the versioned appservice paths before falling back to the [legacy paths](https://spec.matrix.org/v1.6/application-service-api/#legacy-routes). Usage of the legacy routes should be considered deprecated. Additionally, Synapse has supported sending the application service access token via [the `Authorization` header](https://spec.matrix.org/v1.6/application-service-api/#authorization) since v1.70.0. For backwards compatibility it is *also* sent as the `access_token` query parameter. This is insecure and should be considered deprecated. A future version of Synapse (v1.88.0 or later) will remove support for legacy application service routes and query parameter authorization. No significant changes since 1.81.0rc2. Synapse 1.81.0rc2 (2023-04-06) ============================== Bugfixes -------- - Fix the `set_device_id_for_pushers_txn` background update crash. ([\#15391](matrix-org/synapse#15391)) Internal Changes ---------------- - Update CI to run complement under the latest stable go version. ([\#15403](matrix-org/synapse#15403)) Synapse 1.81.0rc1 (2023-04-04) ============================== Features -------- - Add the ability to enable/disable registrations when in the OIDC flow. ([\#14978](matrix-org/synapse#14978)) - Add a primitive helper script for listing worker endpoints. ([\#15243](matrix-org/synapse#15243)) - Experimental support for passing One Time Key and device key requests to application services ([MSC3983](matrix-org/matrix-spec-proposals#3983) and [MSC3984](matrix-org/matrix-spec-proposals#3984)). ([\#15314](matrix-org/synapse#15314), [\#15321](matrix-org/synapse#15321)) - Allow loading `/password_policy` endpoint on workers. ([\#15331](matrix-org/synapse#15331)) - Add experimental support for Unix sockets. Contributed by Jason Little. ([\#15353](matrix-org/synapse#15353)) - Build Debian packages for Ubuntu 23.04 (Lunar Lobster). ([\#15381](matrix-org/synapse#15381)) Bugfixes -------- - Fix a long-standing bug where edits of non-`m.room.message` events would not be correctly bundled. ([\#15295](matrix-org/synapse#15295)) - Fix a bug introduced in Synapse v1.55.0 which could delay remote homeservers being able to decrypt encrypted messages sent by local users. ([\#15297](matrix-org/synapse#15297)) - Add a check to [SQLite port_db script](https://matrix-org.github.io/synapse/latest/postgres.html#porting-from-sqlite) to ensure that the sqlite database passed to the script exists before trying to port from it. ([\#15306](matrix-org/synapse#15306)) - Fix a bug introduced in Synapse 1.76.0 where responses from worker deployments could include an internal `_INT_STREAM_POS` key. ([\#15309](matrix-org/synapse#15309)) - Fix a long-standing bug that Synpase only used the [legacy appservice routes](https://spec.matrix.org/v1.6/application-service-api/#legacy-routes). ([\#15317](matrix-org/synapse#15317)) - Fix a long-standing bug preventing users from rejoining rooms after being banned and unbanned over federation. Contributed by Nico. ([\#15323](matrix-org/synapse#15323)) - Fix bug in worker mode where on a rolling restart of workers the "typing" worker would consume 100% CPU until it got restarted. ([\#15332](matrix-org/synapse#15332)) - Fix a long-standing bug where some to_device messages could be dropped when using workers. ([\#15349](matrix-org/synapse#15349)) - Fix a bug introduced in Synapse 1.70.0 where the background sync from a faster join could spin for hours when one of the events involved had been marked for backoff. ([\#15351](matrix-org/synapse#15351)) - Fix missing app variable in mail subject for password resets. Contributed by Cyberes. ([\#15352](matrix-org/synapse#15352)) - Fix a rare bug introduced in Synapse 1.66.0 where initial syncs would fail when the user had been kicked from a faster joined room that had not finished syncing. ([\#15383](matrix-org/synapse#15383)) Improved Documentation ---------------------- - Fix a typo in login requests ratelimit defaults. ([\#15341](matrix-org/synapse#15341)) - Add some clarification to the doc/comments regarding TCP replication. ([\#15354](matrix-org/synapse#15354)) - Note that Synapse 1.74 queued a rebuild of the user directory tables. ([\#15386](matrix-org/synapse#15386)) Internal Changes ---------------- - Use `immutabledict` instead of `frozendict`. ([\#15113](matrix-org/synapse#15113)) - Add developer documentation for the Federation Sender and add a documentation mechanism using Sphinx. ([\#15265](matrix-org/synapse#15265), [\#15336](matrix-org/synapse#15336)) - Make the pushers rely on the `device_id` instead of the `access_token_id` for various operations. ([\#15280](matrix-org/synapse#15280)) - Bump sentry-sdk from 1.15.0 to 1.17.0. ([\#15285](matrix-org/synapse#15285)) - Allow running the Twisted trunk job against other branches. ([\#15302](matrix-org/synapse#15302)) - Remind the releaser to ask for changelog feedback in [#synapse-dev](https://matrix.to/#/#synapse-dev:matrix.org). ([\#15303](matrix-org/synapse#15303)) - Bump dtolnay/rust-toolchain from e12eda571dc9a5ee5d58eecf4738ec291c66f295 to fc3253060d0c959bea12a59f10f8391454a0b02d. ([\#15304](matrix-org/synapse#15304)) - Reject events with an invalid "mentions" property per [MSC3952](matrix-org/matrix-spec-proposals#3952). ([\#15311](matrix-org/synapse#15311)) - As an optimisation, use `TRUNCATE` on Postgres when clearing the user directory tables. ([\#15316](matrix-org/synapse#15316)) - Fix `.gitignore` rule for the Complement source tarball downloaded automatically by `complement.sh`. ([\#15319](matrix-org/synapse#15319)) - Bump serde from 1.0.157 to 1.0.158. ([\#15324](matrix-org/synapse#15324)) - Bump regex from 1.7.1 to 1.7.3. ([\#15325](matrix-org/synapse#15325)) - Bump types-pyopenssl from 23.0.0.4 to 23.1.0.0. ([\#15326](matrix-org/synapse#15326)) - Bump furo from 2022.12.7 to 2023.3.23. ([\#15327](matrix-org/synapse#15327)) - Bump ruff from 0.0.252 to 0.0.259. ([\#15328](matrix-org/synapse#15328)) - Bump cryptography from 40.0.0 to 40.0.1. ([\#15329](matrix-org/synapse#15329)) - Bump mypy-zope from 0.9.0 to 0.9.1. ([\#15330](matrix-org/synapse#15330)) - Speed up unit tests when using SQLite3. ([\#15334](matrix-org/synapse#15334)) - Speed up pydantic CI job. ([\#15339](matrix-org/synapse#15339)) - Speed up sample config CI job. ([\#15340](matrix-org/synapse#15340)) - Fix copyright year in SSO footer template. ([\#15358](matrix-org/synapse#15358)) - Bump peaceiris/actions-gh-pages from 3.9.2 to 3.9.3. ([\#15369](matrix-org/synapse#15369)) - Bump serde from 1.0.158 to 1.0.159. ([\#15370](matrix-org/synapse#15370)) - Bump serde_json from 1.0.94 to 1.0.95. ([\#15371](matrix-org/synapse#15371)) - Speed up membership queries for users with forgotten rooms. ([\#15385](matrix-org/synapse#15385))

@PaarthShah

Please note that this will be the last release of Synapse that is compatible with Python 3.7 and earlier. This is due to Python 3.7 now having reached End of Life; see our [deprecation policy](https://matrix-org.github.io/synapse/v1.87/deprecation_policy.html) for more details. - Pin `pydantic` to `^1.7.4` to avoid backwards-incompatible API changes from the 2.0.0 release. Resolves matrix-org#15858. Contributed by @PaarthShah. ([\matrix-org#15862](matrix-org#15862)) - Split out 2022 changes from the changelog so the rendered version in GitHub doesn't timeout as much. ([\matrix-org#15846](matrix-org#15846)) - Improve `/messages` response time by avoiding backfill when we already have messages to return. ([\matrix-org#15737](matrix-org#15737)) - Add spam checker module API for logins. ([\matrix-org#15838](matrix-org#15838)) - Fix a long-standing bug where media files were served in an unsafe manner. Contributed by @joshqou. ([\matrix-org#15680](matrix-org#15680)) - Avoid invalidating a cache that was just prefilled. ([\matrix-org#15758](matrix-org#15758)) - Fix requesting multiple keys at once over federation, related to [MSC3983](matrix-org/matrix-spec-proposals#3983). ([\matrix-org#15770](matrix-org#15770)) - Fix joining rooms through aliases where the alias server isn't a real homeserver. Contributed by @tulir @ Beeper. ([\matrix-org#15776](matrix-org#15776)) - Fix a bug in push rules handling leading to an invalid (per spec) `is_user_mention` rule sent to clients. Also fix wrong rule names for `is_user_mention` and `is_room_mention`. ([\matrix-org#15781](matrix-org#15781)) - Fix a bug introduced in 1.57.0 where the wrong table would be locked on updating database rows when using SQLite as the database backend. ([\matrix-org#15788](matrix-org#15788)) - Fix Sytest environmental variable evaluation in CI. ([\matrix-org#15804](matrix-org#15804)) - Fix forgotten rooms missing from initial sync after rejoining them. Contributed by Nico from Famedly. ([\matrix-org#15815](matrix-org#15815)) - Fix sqlite `user_filters` upgrade introduced in v1.86.0. ([\matrix-org#15817](matrix-org#15817)) - Document `looping_call()` functionality that will wait for the given function to finish before scheduling another. ([\matrix-org#15772](matrix-org#15772)) - Fix a typo in the [Admin API](https://matrix-org.github.io/synapse/latest/usage/administration/admin_api/index.html). ([\matrix-org#15805](matrix-org#15805)) - Fix typo in MSC number in faster remote room join architecture doc. ([\matrix-org#15812](matrix-org#15812)) - Remove experimental [MSC2716](matrix-org/matrix-spec-proposals#2716) implementation to incrementally import history into existing rooms. ([\matrix-org#15748](matrix-org#15748)) - Replace `EventContext` fields `prev_group` and `delta_ids` with field `state_group_deltas`. ([\matrix-org#15233](matrix-org#15233)) - Regularly try to send transactions to other servers after they failed instead of waiting for a new event to be available before trying. ([\matrix-org#15743](matrix-org#15743)) - Fix requesting multiple keys at once over federation, related to [MSC3983](matrix-org/matrix-spec-proposals#3983). ([\matrix-org#15755](matrix-org#15755)) - Allow for the configuration of max request retries and min/max retry delays in the matrix federation client. ([\matrix-org#15783](matrix-org#15783)) - Switch from `matrix://` to `matrix-federation://` scheme for internal Synapse routing of outbound federation traffic. ([\matrix-org#15806](matrix-org#15806)) - Fix harmless exceptions being printed when running the port DB script. ([\matrix-org#15814](matrix-org#15814)) * Bump attrs from 22.2.0 to 23.1.0. ([\matrix-org#15801](matrix-org#15801)) * Bump cryptography from 40.0.2 to 41.0.1. ([\matrix-org#15800](matrix-org#15800)) * Bump ijson from 3.2.0.post0 to 3.2.1. ([\matrix-org#15802](matrix-org#15802)) * Bump phonenumbers from 8.13.13 to 8.13.14. ([\matrix-org#15798](matrix-org#15798)) * Bump ruff from 0.0.265 to 0.0.272. ([\matrix-org#15799](matrix-org#15799)) * Bump ruff from 0.0.272 to 0.0.275. ([\matrix-org#15833](matrix-org#15833)) * Bump serde_json from 1.0.96 to 1.0.97. ([\matrix-org#15797](matrix-org#15797)) * Bump serde_json from 1.0.97 to 1.0.99. ([\matrix-org#15832](matrix-org#15832)) * Bump towncrier from 22.12.0 to 23.6.0. ([\matrix-org#15831](matrix-org#15831)) * Bump types-opentracing from 2.4.10.4 to 2.4.10.5. ([\matrix-org#15830](matrix-org#15830)) * Bump types-setuptools from 67.8.0.0 to 68.0.0.0. ([\matrix-org#15835](matrix-org#15835))

Sending One-Time Key (OTK) claims to appservices

f82e463

turt2live marked this pull request as ready for review March 23, 2023 18:41

turt2live added 2 commits March 23, 2023 12:44

Fix typos

90006ad

Actually fix typos?

b128030

uhoreg reviewed Mar 23, 2023

View reviewed changes

proposals/3983-sending-otk-claims-to-appservices.md Outdated Show resolved Hide resolved

turt2live added 2 commits March 23, 2023 15:12

Oops, auth is important

0630814

Clarify that OTKs should only be used once still

bbc79ab

clokep mentioned this pull request Mar 23, 2023

Implement MSC3983 to proxy /keys/claim queries to appservices. matrix-org/synapse#15314

Merged

turt2live added a commit to turt2live/matrix-bot-sdk that referenced this pull request Mar 23, 2023

Support MSC3983: Keys Claim request

f4a15ac

See matrix-org/matrix-spec-proposals#3983

turt2live mentioned this pull request Mar 23, 2023

Support MSC3983: Keys Claim request turt2live/matrix-bot-sdk#307

Merged

turt2live added a commit to turt2live/matrix-bot-sdk that referenced this pull request Mar 23, 2023

Support MSC3983: Keys Claim request (#307)

fb121c2

* Support MSC3983: Keys Claim request See matrix-org/matrix-spec-proposals#3983 * Include test code * Appease the linter

turt2live mentioned this pull request Sep 20, 2023

NodeJS bindings: Support for MSC3983 matrix-org/matrix-rust-sdk-crypto-nodejs#29

Open

turt2live commented Mar 24, 2023

View reviewed changes

proposals/3983-sending-otk-claims-to-appservices.md Outdated Show resolved Hide resolved

turt2live commented Mar 24, 2023

View reviewed changes

proposals/3983-sending-otk-claims-to-appservices.md Outdated Show resolved Hide resolved

clokep reviewed Mar 24, 2023

View reviewed changes

proposals/3983-sending-otk-claims-to-appservices.md Show resolved Hide resolved

tulir reviewed Mar 24, 2023

View reviewed changes

proposals/3983-sending-otk-claims-to-appservices.md Outdated Show resolved Hide resolved

turt2live added 6 commits March 24, 2023 10:52

Acknowledge performance hit

91e6d02

Mention e2be approach currently in use

0b9f6d9

Mention encryption bridge usecase

8a1dd86

Use the fallback key when missing *some* keys too

5a4a6c6

Mention MSC3984 for key queries

c770168

Clarify that 405/M_UNRECOGNISED is a good idea

a95bf4f

dkasak reviewed Mar 27, 2023

View reviewed changes

clokep reviewed Apr 13, 2023

View reviewed changes

turt2live added 2 commits April 19, 2023 11:17

Add a section about overriding fallback keys

2177b93

Update 3983-sending-otk-claims-to-appservices.md

052d480

clokep reviewed Apr 19, 2023

View reviewed changes

clokep reviewed Apr 20, 2023

View reviewed changes

clokep mentioned this pull request Apr 20, 2023

Add unstable /keys/claim endpoint which always returns fallback keys matrix-org/synapse#15462

Merged

clokep reviewed Apr 24, 2023

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

MSC3983: Sending One-Time Key (OTK) claims to appservices #3983

MSC3983: Sending One-Time Key (OTK) claims to appservices #3983

turt2live commented Mar 23, 2023 •

edited

dkasak Mar 27, 2023

dkasak Mar 27, 2023

turt2live Mar 27, 2023

dkasak Mar 27, 2023 •

edited

turt2live Mar 27, 2023

dkasak Mar 27, 2023

turt2live Mar 27, 2023

dkasak Mar 27, 2023

turt2live Mar 27, 2023

dkasak Mar 27, 2023

clokep Apr 13, 2023

clokep Apr 19, 2023

turt2live Apr 19, 2023

clokep Apr 20, 2023

clokep Apr 24, 2023

clokep Apr 20, 2023

turt2live Apr 20, 2023

clokep Apr 20, 2023

clokep Apr 24, 2023

clokep Apr 24, 2023

clokep Apr 27, 2023

	from device IDs and user IDs need to exist). In future, other MSCs might make encryption bridges easier to
	from device and user identities need to exist). In the future, other MSCs might make encryption bridges easier to

		Independent of the appservice having `/keys/claim` proxied to it, it may be desirable for both the
		fallback and one-time key to be returned. Servers should always include the fallback key alongside

MSC3983: Sending One-Time Key (OTK) claims to appservices #3983

Are you sure you want to change the base?

MSC3983: Sending One-Time Key (OTK) claims to appservices #3983

Conversation

turt2live commented Mar 23, 2023 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dkasak Mar 27, 2023 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

turt2live commented Mar 23, 2023 •

edited

dkasak Mar 27, 2023 •

edited