Merge branch 'main' into openapi-patch

changelogs/client_server/newsfragments/1366.feature

@@ -0,0 +1 @@
+Add `/rooms/<roomID>/timestamp_to_event` endpoint, as per [MSC3030](https://github.com/matrix-org/matrix-spec-proposals/pull/3030).

changelogs/client_server/newsfragments/1381.clarification

@@ -0,0 +1 @@
+Clarify parts of the end-to-end encryption sections.

changelogs/client_server/newsfragments/1382.clarification

@@ -0,0 +1 @@
+Move login API definitions to the right heading. Contributed by @HarHarLinks.

changelogs/internal/newsfragments/1368.clarification

@@ -0,0 +1 @@
+Improve safety of the proposals retrieval script in the event of failure.

changelogs/internal/newsfragments/1384.clarification

@@ -0,0 +1 @@
+Stop autogenerating examples where we already have an example.

changelogs/room_versions/newsfragments/1397.feature

@@ -0,0 +1 @@
+Update the default room version to 10, as per [MSC3904](https://github.com/matrix-org/matrix-spec-proposals/pull/3904).

changelogs/server_server/newsfragments/1350.breaking

@@ -0,0 +1 @@
+Remove `keyId` from the server `/keys` endpoints, as per [MSC3938](https://github.com/matrix-org/matrix-spec-proposals/pull/3938).

changelogs/server_server/newsfragments/1366.feature

@@ -0,0 +1 @@
+Add `/timestamp_to_event/<roomID>` endpoint, as per [MSC3030](https://github.com/matrix-org/matrix-spec-proposals/pull/3030).

changelogs/server_server/newsfragments/1371.clarification

@@ -0,0 +1 @@
+Correct the default invite level definition in the "Checks performed on receipt of a PDU" section.

changelogs/server_server/newsfragments/1376.clarification

@@ -0,0 +1 @@
+Clarify that CNAMEs are permissible for server names.

changelogs/server_server/newsfragments/1383.clarification

@@ -0,0 +1 @@
+Fix `edu_type` in EDU examples.

changelogs/server_server/newsfragments/1393.feature

@@ -0,0 +1 @@
+Extend `/_matrix/federation/v2/send_join` to allow omitting membership events, per [MSC3706](https://github.com/matrix-org/matrix-doc/pull/3706).

changelogs/server_server/newsfragments/1398.feature

@@ -0,0 +1 @@
+Extend `/_matrix/federation/v2/send_join` to allow omitting membership events, per [MSC3706](https://github.com/matrix-org/matrix-doc/pull/3706).

content/client-server-api/_index.md

@@ -1145,6 +1145,12 @@ client supports it, the client should redirect the user to the
 is complete, the client will need to submit a `/login` request matching
 `m.login.token`.
 
+{{% http-api spec="client-server" api="login" %}}
+
+{{% http-api spec="client-server" api="refresh" %}}
+
+{{% http-api spec="client-server" api="logout" %}}
+
 #### Appservice Login
 
 {{% added-in v="1.2" %}}
@@ -1182,12 +1188,6 @@ If the access token does correspond to an appservice, but the user id does
 not lie within its namespace then the homeserver will respond with an
 errcode of `M_EXCLUSIVE`.
 
-{{% http-api spec="client-server" api="login" %}}
-
-{{% http-api spec="client-server" api="refresh" %}}
-
-{{% http-api spec="client-server" api="logout" %}}
-
 #### Login Fallback
 
 If a client does not recognize any or all login flows it can use the
@@ -1818,6 +1818,8 @@ There are several APIs provided to `GET` events for a room:
 
 {{% http-api spec="client-server" api="message_pagination" %}}
 
+{{% http-api spec="client-server" api="room_event_by_timestamp" %}}
+
 {{% http-api spec="client-server" api="room_initial_sync" %}}
 
 ### Sending events to a room
@@ -2646,4 +2648,4 @@ systems.
 {{< cs-module name="spaces" >}}
 {{< cs-module name="event_replacements" >}}
 {{< cs-module name="threading" >}}
-{{< cs-module name="reference_relations" >}}
+{{< cs-module name="reference_relations" >}}

content/client-server-api/modules/end_to_end_encryption.md

@@ -51,6 +51,9 @@ keys.
 
 ##### Key algorithms
 
+Different key algorithms are used for different purposes.  Each key algorithm
+is identified by a name and is represented in a specific way.
+
 The name `ed25519` corresponds to the
 [Ed25519](http://ed25519.cr.yp.to/) signature algorithm. The key is a
 32-byte Ed25519 public key, encoded using [unpadded Base64](/appendices/#unpadded-base64). Example:
@@ -64,9 +67,9 @@ Example:
 
     "JGLn/yafz74HB2AbPLYJWIVGnKAtqECOBf11yyXac2Y"
 
-The name `signed_curve25519` also corresponds to the Curve25519
-algorithm, but a key using this algorithm is represented by an object
-with the following properties:
+The name `signed_curve25519` also corresponds to the Curve25519 ECDH algorithm,
+but the key is signed so that it can be authenticated. A key using this
+algorithm is represented by an object with the following properties:
 
 `KeyObject`
 
@@ -88,23 +91,55 @@ Example:
 }
 ```
 
+`ed25519` and `curve25519` keys are used for [device keys](#device-keys).
+Additionally, `ed25519` keys are used for [cross-signing keys](#cross-signing).
+
+`signed_curve25519` keys are used for [one-time and fallback keys](#one-time-and-fallback-keys).
+
 ##### Device keys
 
 Each device should have one Ed25519 signing key. This key should be
 generated on the device from a cryptographically secure source, and the
 private part of the key should never be exported from the device. This
-key is used as the fingerprint for a device by other clients.
+key is used as the fingerprint for a device by other clients, and signs the
+device's other keys.
 
 A device will generally need to generate a number of additional keys.
 Details of these will vary depending on the messaging algorithm in use.
 
-Algorithms generally require device identity keys as well as signing
-keys. Some algorithms also require one-time keys to improve their
-secrecy and deniability. These keys are used once during session
-establishment, and are then thrown away.
+For Olm version 1, each device also requires a single Curve25519 identity
+key.
+
+##### One-time and fallback keys
+
+In addition to the device keys, which are long-lived, some encryption
+algorithms require that devices may also have a number of one-time keys, which
+are only used once and discarded after use. For Olm version 1, devices use
+`signed_curve25519` one-time keys, signed by the device's Ed25519 key.
+
+Devices will generate one-time keys and upload them to the server; these will
+later be [claimed](#post_matrixclientv3keysclaim) by other users. Servers must
+ensure that each one-time key is only claimed once: a homeserver should discard
+the one time key once it has been given to another user.
+
+{{% added-in v="1.2" %}} Fallback keys are similar to one-time keys, but are
+not consumed once used. If a fallback key has been uploaded, it will be
+returned by the server when the device has run out of one-time keys and a user
+tries to claim a key. Fallback keys should be replaced with new fallback keys
+as soon as possible after they have been used.
+
+{{% boxes/warning %}}
+Fallback keys are used to prevent one-time key exhaustion when devices
+are offline/unable to upload additional keys, though sessions started using
+fallback keys could be vulnerable to replay attacks.
+{{% /boxes/warning %}}
 
-For Olm version 1, each device requires a single Curve25519 identity
-key, and a number of signed Curve25519 one-time keys.
+Devices will be informed, [via
+`/sync`](#a-namee2e-extensions-to-sync-extensions-to-sync), about the number of
+one-time keys remaining that can be claimed, as well as whether the fallback
+keys have been used. The device can thus ensure that, while it is online, there
+is a sufficient supply of one-time keys available, and that the fallback keys
+get replaced if they have been used.
 
 ##### Uploading keys
 
@@ -115,11 +150,8 @@ signed by that key, as described in [Signing
 JSON](/appendices/#signing-json).
 
 One-time and fallback keys are also uploaded to the homeserver using the
-[`/keys/upload`](/client-server-api/#post_matrixclientv3keysupload) API.
-
-{{% added-in v="1.2" %}} Fallback keys are similar to one-time keys, but
-are not consumed once used. They are only used when the device has run out
-of one-time keys, and can be replaced by a new fallback key.
+[`/keys/upload`](/client-server-api/#post_matrixclientv3keysupload) API. New
+one-time and fallback keys are uploaded as needed.
 
 Devices must store the private part of each key they upload. They can
 discard the private part of a one-time key when they receive a message
@@ -129,12 +161,6 @@ never know that it can discard the key. Therefore a device could end up
 trying to store too many private keys. A device that is trying to store
 too many private keys may discard keys starting with the oldest.
 
-{{% boxes/warning %}}
-Fallback keys are used to prevent one-time key exhaustion when devices
-are offline/unable to upload additional keys, though sessions started using
-fallback keys could be vulnerable to replay attacks.
-{{% /boxes/warning %}}
-
 {{% boxes/warning %}}
 Clients should not store the private half of fallback keys indefinitely
 to avoid situations where attackers can decrypt past messages sent using
@@ -353,21 +379,6 @@ Example:
 }
 ```
 
-##### Claiming one-time keys
-
-A client wanting to set up a session with another device can claim a
-one-time key for that device. This is done by making a request to the
-[`/keys/claim`](/client-server-api/#post_matrixclientv3keysclaim) API.
-
-A homeserver should rate-limit the number of one-time keys that a given
-user or remote server can claim. A homeserver should discard the public
-part of a one time key once it has given that key to another user.
-
-{{% added-in v="1.2" %}} If the device has run out of one-time keys which
-can be claimed, the homeserver will return the fallback key (if one was
-uploaded). Fallback keys are not deleted once used and should be replaced
-by the device when they are able to upload more one-time keys.
-
 #### Device verification
 
 Before Alice sends Bob encrypted data, or trusts data received from him,

content/rooms/_index.md

@@ -52,7 +52,7 @@ stable and unstable periodically for a variety of reasons, including
 discovered security vulnerabilities and age.
 
 Clients should not ask room administrators to upgrade their rooms if the
-room is running a stable version. Servers SHOULD use **room version 9** as
+room is running a stable version. Servers SHOULD use **room version 10** as
 the default room version when creating new rooms.
 
 The available room versions are:

content/rooms/fragments/v5-signing-requirements.md

@@ -3,7 +3,7 @@ When validating event signatures, servers MUST enforce the
 `valid_until_ts` property from a key request is at least as large as the
 `origin_server_ts` for the event being validated. Servers missing a copy
 of the signing key MUST try to obtain one via the [GET
-/\_matrix/key/v2/server](/server-server-api#get_matrixkeyv2serverkeyid)
+/\_matrix/key/v2/server](/server-server-api#get_matrixkeyv2server)
 or [POST
 /\_matrix/key/v2/query](/server-server-api#post_matrixkeyv2query)
 APIs. When using the `/query` endpoint, servers MUST set the

content/server-server-api.md

@@ -110,11 +110,14 @@ to send. The process overall is as follows:
     given. The target server must present a valid certificate for the IP
     address. The `Host` header in the request should be set to the
     server name, including the port if the server name included one.
-2.  If the hostname is not an IP literal, and the server name includes
-    an explicit port, resolve the IP address using AAAA or A records.
+
+2.  If the hostname is not an IP literal, and the server name includes an
+    explicit port, resolve the hostname to an IP address using CNAME, AAAA or A
+    records.
     Requests are made to the resolved IP address and given port with a
     `Host` header of the original server name (with port). The target
     server must present a valid certificate for the hostname.
+
 3.  If the hostname is not an IP literal, a regular HTTPS request is
     made to `https://<hostname>/.well-known/matrix/server`, expecting
     the schema defined later in this section. 30x redirects should be
@@ -140,7 +143,7 @@ to send. The process overall is as follows:
         one was provided.
     -   If `<delegated_hostname>` is not an IP literal, and
         `<delegated_port>` is present, an IP address is discovered by
-        looking up an AAAA or A record for `<delegated_hostname>`. The
+        looking up CNAME, AAAA or A records for `<delegated_hostname>`.  The
         resulting IP address is used, alongside the `<delegated_port>`.
         Requests must be made with a `Host` header of
         `<delegated_hostname>:<delegated_port>`. The target server must
@@ -153,20 +156,22 @@ to send. The process overall is as follows:
         a `Host` header containing the `<delegated_hostname>`. The
         target server must present a valid certificate for
         `<delegated_hostname>`.
-    -   If no SRV record is found, an IP address is resolved using AAAA
+    -   If no SRV record is found, an IP address is resolved using CNAME, AAAA
         or A records. Requests are then made to the resolve IP address
         and a port of 8448, using a `Host` header of
         `<delegated_hostname>`. The target server must present a valid
         certificate for `<delegated_hostname>`.
+
 4.  If the `/.well-known` request resulted in an error response, a
     server is found by resolving an SRV record for
     `_matrix._tcp.<hostname>`. This may result in a hostname (to be
     resolved using AAAA or A records) and port. Requests are made to the
     resolved IP address and port, using 8448 as a default port, with a
     `Host` header of `<hostname>`. The target server must present a
     valid certificate for `<hostname>`.
+
 5.  If the `/.well-known` request returned an error response, and the
-    SRV record was not found, an IP address is resolved using AAAA and A
+    SRV record was not found, an IP address is resolved using CNAME, AAAA and A
     records. Requests are made to the resolved IP address using port
     8448 and a `Host` header containing the `<hostname>`. The target
     server must present a valid certificate for `<hostname>`.
@@ -180,6 +185,13 @@ delegation are:
      and other applications using SRV records such [XMPP](https://datatracker.ietf.org/doc/html/rfc6120#section-13.7.2.1).
 {{% /boxes/note %}}
 
+{{% boxes/note %}}
+Note that the target of a SRV record may *not* be a CNAME, as
+mandated by [RFC2782](https://www.rfc-editor.org/rfc/rfc2782.html):
+
+> the name MUST NOT be an alias (in the sense of RFC 1034 or RFC 2181)
+{{% /boxes/note %}}
+
 {{% http-api spec="server-server" api="wellknown" %}}
 
 ### Server implementation
@@ -196,11 +208,11 @@ draft](https://github.com/matrix-org/matrix-doc/blob/51faf8ed2e4a63d4cfd6d231836
 {{% /boxes/note %}}
 
 Each homeserver publishes its public keys under
-`/_matrix/key/v2/server/{keyId}`. Homeservers query for keys by either
-getting `/_matrix/key/v2/server/{keyId}` directly or by querying an
+`/_matrix/key/v2/server`. Homeservers query for keys by either
+getting `/_matrix/key/v2/server` directly or by querying an
 intermediate notary server using a
-`/_matrix/key/v2/query/{serverName}/{keyId}` API. Intermediate notary
-servers query the `/_matrix/key/v2/server/{keyId}` API on behalf of
+`/_matrix/key/v2/query/{serverName}` API. Intermediate notary
+servers query the `/_matrix/key/v2/server` API on behalf of
 another server and sign the response with their own key. A server may
 query multiple notary servers to ensure that they all report the same
 public keys.
@@ -227,7 +239,7 @@ homeserver and for signing events. It contains a list of
 Servers may query another server's keys through a notary server. The
 notary server may be another homeserver. The notary server will retrieve
 keys from the queried servers through use of the
-`/_matrix/key/v2/server/{keyId}` API. The notary server will
+`/_matrix/key/v2/server` API. The notary server will
 additionally sign the response from the queried server before returning
 the results.
 
@@ -437,21 +449,25 @@ them.
 
 #### Definitions
 
-**Required Power Level** \
-A given event type has an associated *required power level*. This is
-given by the current `m.room.power_levels` event. The event type is
-either listed explicitly in the `events` section or given by either
-`state_default` or `events_default` depending on if the event is a state
-event or not.
+Required Power Level
+
+: A given event type has an associated *required power level*. This is given by
+  the current [`m.room.power_levels`](/client-server-api/#mroompower_levels)
+  event. The event type is either listed explicitly in the `events` section or
+  given by either `state_default` or `events_default` depending on if the event
+  is a state event or not.
+
+Invite Level, Kick Level, Ban Level, Redact Level
+
+: The levels given by the `invite`, `kick`, `ban`, and `redact` properties in
+  the current [`m.room.power_levels`](/client-server-api/#mroompower_levels)
+  state. The invite level defaults to 0 if unspecified. The kick level, ban level
+  and redact level each default to 50 if unspecified.
 
-**Invite Level, Kick Level, Ban Level, Redact Level** \
-The levels given by the `invite`, `kick`, `ban`, and `redact` properties
-in the current `m.room.power_levels` state. Each defaults to 50 if
-unspecified.
+Target User
 
-**Target User** \
-For an `m.room.member` state event, the user given by the `state_key` of
-the event.
+: For an [`m.room.member`](/client-server-api/#mroommember) state event, the
+  user given by the `state_key` of the event.
 
 {{% boxes/warning %}}
 Some [room versions](/rooms) accept power level values to be represented as
@@ -653,7 +669,7 @@ EDUs, by comparison to PDUs, do not have an ID, a room ID, or a list of
 "previous" IDs. They are intended to be non-persistent data such as user
 presence, typing notifications, etc.
 
-{{% definition path="api/server-server/definitions/edu" %}}
+{{% definition path="api/server-server/definitions/edu_with_example" %}}
 
 ## Room State Resolution
 

data/api/client-server/administrative_contact.yaml

@@ -210,14 +210,12 @@ paths:
               client_secret:
                 type: string
                 description: The client secret used in the session with the homeserver.
+                example: "d0nt-T3ll"
               sid:
                 type: string
                 description: The session identifier given by the homeserver.
+                example: "abc123987"
             required: ["client_secret",  "sid"]
-            example: {
-              "sid": "abc123987",
-              "client_secret": "d0nt-T3ll"
-            }
       responses:
         200:
           description: The addition was successful.

data/api/client-server/cross_signing.yaml

@@ -75,6 +75,11 @@ paths:
                 allOf:
                 - $ref: "definitions/auth_data.yaml"
             example: {
+              "auth": {
+                "type": "example.type.foo",
+                "session": "xxxxx",
+                "example_credential": "verypoorsharedsecret"
+              },
               "master_key": {
                 "user_id": "@alice:example.com",
                 "usage": ["master"],