Define hkdf-hmac-sha256.v2 MAC method for SAS verification

changelogs/client_server/newsfragments/1412.feature

@@ -0,0 +1 @@
+Define `hkdf-hmac-sha256.v2` MAC method for SAS verification, as per [MSC 3783](Define hkdf-hmac-sha256.v2 MAC method for SAS verification).

content/client-server-api/modules/end_to_end_encryption.md

@@ -671,10 +671,7 @@ The process between Alice and Bob verifying each other would be:
 15. Assuming they match, Alice and Bob's devices calculate the HMAC of
     their own device keys and a comma-separated sorted list of the key
     IDs that they wish the other user to verify, using SHA-256 as the
-    hash function. HMAC is defined in [RFC
-    2104](https://tools.ietf.org/html/rfc2104). The key for the HMAC is
-    different for each item and is calculated by generating 32 bytes
-    (256 bits) using [the key verification HKDF](#hkdf-calculation).
+    hash function. The HMAC calculation is defined [below](#mac-calculation).
 16. Alice's device sends Bob's device an `m.key.verification.mac`
     message containing the MAC of Alice's device keys and the MAC of her
     key IDs to be verified. Bob's device does the same for Bob's device
@@ -767,7 +764,42 @@ following error codes are used in addition to those already specified:
 
 {{% event event="m.key.verification.mac" %}}
 
-###### HKDF calculation
+###### MAC calculation
+
+For verification of each party's device keys, a MAC is calculated individually
+for each the keys that are to be verified. As well, a MAC is calculated for a
+list of the keys IDs. The MAC used is HMAC as defined in [RFC
+2104](https://tools.ietf.org/html/rfc2104), using SHA-256 as the hash function.
+The HMAC key is calculated using HKDF as defined in [RFC
+5869](https://tools.ietf.org/html/rfc5869), using SHA-256 as the hash
+function. The shared secret is supplied as the input keying material. No salt
+is used, and in the info parameter is the concatenation of:
+
+-   The string `MATRIX_KEY_VERIFICATION_MAC`.
+-   The Matrix ID of the user whose key is being MAC-ed.
+-   The Device ID of the device sending the MAC.
+-   The Matrix ID of the other user.
+-   The Device ID of the device receiving the MAC.
+-   The `transaction_id` being used.
+-   The Key ID of the key being MAC-ed, or the string `KEY_IDS` if the
+    item being MAC-ed is the list of key IDs.
+
+If the key list is being MACed, the list is sorted lexicographically and
+comma-separated with no extra whitespace added. In this way, the recipient can
+reconstruct the list from the names in the `mac` property of the
+`m.key.verification.mac` message and ensure that no keys were added or removed.
+
+{{% boxes/note %}}
+The MAC method `hkdf-hmac-sha256` used an incorrect base64 encoding, due to a
+bug in the original implementation in libolm.  To remedy this,
+`hkdf-hmac-sha256.v2` was introduced, which calculates the MAC in the same way,
+but uses a correct base64 encoding. `hkdf-hmac-sha256` is deprecated and will
+be removed in a future version of the spec. Use of `hkdf-hmac-sha256` should
+be avoided whenever possible: if both parties support `hkdf-hmac-sha256.v2`,
+then `hkdf-hmac-sha256` MUST not be used.
+{{% /boxes/note %}}
+
+###### SAS HKDF calculation
 
 In all of the SAS methods, HKDF is as defined in [RFC
 5869](https://tools.ietf.org/html/rfc5869) and uses the previously
@@ -815,23 +847,9 @@ HKDF is used over the plain shared secret as it results in a harder
 attack as well as more uniform data to work with.
 {{% /boxes/rationale %}}
 
-For verification of each party's device keys, HKDF is as defined in RFC
-5869 and uses SHA-256 as the hash function. The shared secret is
-supplied as the input keying material. No salt is used, and in the info
-parameter is the concatenation of:
-
--   The string `MATRIX_KEY_VERIFICATION_MAC`.
--   The Matrix ID of the user whose key is being MAC-ed.
--   The Device ID of the device sending the MAC.
--   The Matrix ID of the other user.
--   The Device ID of the device receiving the MAC.
--   The `transaction_id` being used.
--   The Key ID of the key being MAC-ed, or the string `KEY_IDS` if the
-    item being MAC-ed is the list of key IDs.
-
 ###### SAS method: `decimal`
 
-Generate 5 bytes using [HKDF](#hkdf-calculation) then take sequences of 13 bits
+Generate 5 bytes using [HKDF](#sas-hkdf-calculation) then take sequences of 13 bits
 to convert to decimal numbers (resulting in 3 numbers between 0 and 8191
 inclusive each). Add 1000 to each calculated number.
 
@@ -849,7 +867,7 @@ separator, such as dashes, or with the numbers on individual lines.
 
 ###### SAS method: `emoji`
 
-Generate 6 bytes using [HKDF](#hkdf-calculation) then split the first 42 bits
+Generate 6 bytes using [HKDF](#sas-hkdf-calculation) then split the first 42 bits
 into 7 groups of 6 bits, similar to how one would base64 encode
 something. Convert each group of 6 bits to a number and use the
 following table to get the corresponding emoji:

data/event-schemas/examples/m.key.verification.accept.yaml

@@ -5,7 +5,7 @@
         "method": "m.sas.v1",
         "key_agreement_protocol": "curve25519",
         "hash": "sha256",
-        "message_authentication_code": "hkdf-hmac-sha256",
+        "message_authentication_code": "hkdf-hmac-sha256.v2",
         "short_authentication_string": ["decimal", "emoji"],
         "commitment": "fQpGIW1Snz+pwLZu6sTy2aHy/DYWWTspTJRPyNp0PKkymfIsNffysMl6ObMMFdIJhk6g6pwlIqZ54rxo8SLmAg"
     }

data/event-schemas/examples/m.key.verification.start$m.sas.v1.yaml

@@ -6,7 +6,7 @@
         "method": "m.sas.v1",
         "key_agreement_protocols": ["curve25519"],
         "hashes": ["sha256"],
-        "message_authentication_codes": ["hkdf-hmac-sha256"],
+        "message_authentication_codes": ["hkdf-hmac-sha256.v2", "hkdf-hmac-sha256"],
         "short_authentication_string": ["decimal", "emoji"]
     }
 }

data/event-schemas/schema/m.key.verification.mac.yaml

@@ -3,7 +3,9 @@ allOf:
   - $ref: core-event-schema/event.yaml
 
 description: |-
-  Sends the MAC of a device's key to the partner device.
+  Sends the MAC of a device's key to the partner device.  The MAC is calculated
+  using the method given in `message_authentication_code` property of the
+  `m.key.verification.accept` message.
 properties:
   content:
     properties:

data/event-schemas/schema/m.key.verification.start$m.sas.v1.yaml

@@ -42,7 +42,10 @@ properties:
         type: array
         description: |-
           The message authentication codes that the sending device understands.
-          Must include at least `hkdf-hmac-sha256`.
+          Must include at least `hkdf-hmac-sha256.v2`.  Should also include
+          `hkdf-hmac-sha256` for compatibility with older clients, though this
+          identifier is deprecated and will be removed in a future version of
+          the spec.
         items:
           type: string
       short_authentication_string: