Merge tag 'v23.1.0' into sc

* Remove extensible events v1 field population on legacy events ([\matrix-org#3040](matrix-org#3040)).
* Improve hasUserReadEvent and getUserReadUpTo realibility with threads ([\matrix-org#3031](matrix-org#3031)). Fixes element-hq/element-web#24164.
* Remove video track when muting video ([\matrix-org#3028](matrix-org#3028)). Fixes element-hq/element-call#209.
* Make poll start event type available (PSG-962) ([\matrix-org#3034](matrix-org#3034)).
* Add alt event type matching in Relations model ([\matrix-org#3018](matrix-org#3018)).
* Remove usage of v1 Identity Server API ([\matrix-org#3003](matrix-org#3003)).
* Add `device_id` to `/account/whoami` types ([\matrix-org#3005](matrix-org#3005)).
* Implement MSC3912: Relation-based redactions ([\matrix-org#2954](matrix-org#2954)).
* Introduce a mechanism for using the rust-crypto-sdk ([\matrix-org#2969](matrix-org#2969)).
* Support MSC3391: Account data deletion ([\matrix-org#2967](matrix-org#2967)).
* Fix threaded cache receipt when event holds multiple receipts ([\matrix-org#3026](matrix-org#3026)).
* Fix false key requests after verifying new device ([\matrix-org#3029](matrix-org#3029)). Fixes element-hq/element-web#24167 and element-hq/element-web#23333.
* Avoid triggering decryption errors when decrypting redacted events ([\matrix-org#3004](matrix-org#3004)). Fixes element-hq/element-web#24084.
* bugfix: upload OTKs in sliding sync mode ([\matrix-org#3008](matrix-org#3008)).
* Apply edits discovered from sync after thread is initialised ([\matrix-org#3002](matrix-org#3002)). Fixes element-hq/element-web#23921.
* Sliding sync: Fix issue where no unsubs are sent when switching rooms ([\matrix-org#2991](matrix-org#2991)).
* Threads are missing from the timeline ([\matrix-org#2996](matrix-org#2996)). Fixes element-hq/element-web#24036.
* Close all streams when a call ends ([\matrix-org#2992](matrix-org#2992)). Fixes element-hq/element-call#742.
* Resume to-device message queue after resumed sync ([\matrix-org#2920](matrix-org#2920)). Fixes matrix-org/element-web-rageshakes#17170.
* Fix browser entrypoint ([\matrix-org#3051](matrix-org#3051)). Fixes matrix-org#3013.
* Fix failure to start in firefox private browser ([\matrix-org#3058](matrix-org#3058)). Fixes element-hq/element-web#24216.
* Correctly handle limited sync responses by resetting the thread timeline ([\matrix-org#3056](matrix-org#3056)). Fixes element-hq/element-web#23952.

.git-blame-ignore-revs

@@ -38,4 +38,5 @@ cee7f7a280a8c20bafc21c0a2911f60851f7a7ca
 7ed65407e6cdf292ce3cf659310c68d19dcd52b2
 # Switch to ESLint from JSHint (Google eslint rules as a base)
 e057956ede9ad1a931ff8050c411aca7907e0394
-
+# prettier
+349c2c2587c2885bb69eda4aa078b5383724cf5e

CHANGELOG.md

@@ -1,3 +1,34 @@
+Changes in [23.1.0](https://github.com/matrix-org/matrix-js-sdk/releases/tag/v23.1.0) (2023-01-18)
+==================================================================================================
+
+## 🦖 Deprecations
+ * Remove extensible events v1 field population on legacy events ([\#3040](https://github.com/matrix-org/matrix-js-sdk/pull/3040)).
+
+## ✨ Features
+ * Improve hasUserReadEvent and getUserReadUpTo realibility with threads ([\#3031](https://github.com/matrix-org/matrix-js-sdk/pull/3031)). Fixes vector-im/element-web#24164.
+ * Remove video track when muting video ([\#3028](https://github.com/matrix-org/matrix-js-sdk/pull/3028)). Fixes vector-im/element-call#209.
+ * Make poll start event type available (PSG-962) ([\#3034](https://github.com/matrix-org/matrix-js-sdk/pull/3034)).
+ * Add alt event type matching in Relations model ([\#3018](https://github.com/matrix-org/matrix-js-sdk/pull/3018)).
+ * Remove usage of v1 Identity Server API ([\#3003](https://github.com/matrix-org/matrix-js-sdk/pull/3003)).
+ * Add `device_id` to `/account/whoami` types ([\#3005](https://github.com/matrix-org/matrix-js-sdk/pull/3005)).
+ * Implement MSC3912: Relation-based redactions ([\#2954](https://github.com/matrix-org/matrix-js-sdk/pull/2954)).
+ * Introduce a mechanism for using the rust-crypto-sdk ([\#2969](https://github.com/matrix-org/matrix-js-sdk/pull/2969)).
+ * Support MSC3391: Account data deletion ([\#2967](https://github.com/matrix-org/matrix-js-sdk/pull/2967)).
+
+## 🐛 Bug Fixes
+ * Fix threaded cache receipt when event holds multiple receipts ([\#3026](https://github.com/matrix-org/matrix-js-sdk/pull/3026)).
+ * Fix false key requests after verifying new device ([\#3029](https://github.com/matrix-org/matrix-js-sdk/pull/3029)). Fixes vector-im/element-web#24167 and vector-im/element-web#23333.
+ * Avoid triggering decryption errors when decrypting redacted events ([\#3004](https://github.com/matrix-org/matrix-js-sdk/pull/3004)). Fixes vector-im/element-web#24084.
+ * bugfix: upload OTKs in sliding sync mode ([\#3008](https://github.com/matrix-org/matrix-js-sdk/pull/3008)).
+ * Apply edits discovered from sync after thread is initialised ([\#3002](https://github.com/matrix-org/matrix-js-sdk/pull/3002)). Fixes vector-im/element-web#23921.
+ * Sliding sync: Fix issue where no unsubs are sent when switching rooms ([\#2991](https://github.com/matrix-org/matrix-js-sdk/pull/2991)).
+ * Threads are missing from the timeline ([\#2996](https://github.com/matrix-org/matrix-js-sdk/pull/2996)). Fixes vector-im/element-web#24036.
+ * Close all streams when a call ends ([\#2992](https://github.com/matrix-org/matrix-js-sdk/pull/2992)). Fixes vector-im/element-call#742.
+ * Resume to-device message queue after resumed sync ([\#2920](https://github.com/matrix-org/matrix-js-sdk/pull/2920)). Fixes matrix-org/element-web-rageshakes#17170.
+ * Fix browser entrypoint ([\#3051](https://github.com/matrix-org/matrix-js-sdk/pull/3051)). Fixes #3013.
+ * Fix failure to start in firefox private browser ([\#3058](https://github.com/matrix-org/matrix-js-sdk/pull/3058)). Fixes vector-im/element-web#24216.
+ * Correctly handle limited sync responses by resetting the thread timeline ([\#3056](https://github.com/matrix-org/matrix-js-sdk/pull/3056)). Fixes vector-im/element-web#23952.
+
 Changes in [23.0.0](https://github.com/matrix-org/matrix-js-sdk/releases/tag/v23.0.0) (2022-12-21)
 ==================================================================================================
 

README.md

@@ -47,7 +47,7 @@ if you do not have it already.
 
 ```javascript
 import * as sdk from "matrix-js-sdk";
-const client = sdk.createClient("https://matrix.org");
+const client = sdk.createClient({ baseUrl: "https://matrix.org" });
 client.publicRooms(function (err, data) {
     console.log("Public Rooms: %s", JSON.stringify(data));
 });

examples/browser/README.md

@@ -1,6 +1,7 @@
 To try it out, **you must build the SDK first** and then host this folder:
 
 ```
+ $ yarn install
  $ yarn build
  $ cd examples/browser
  $ python -m http.server 8003

package.json

@@ -1,6 +1,6 @@
 {
     "name": "matrix-js-sdk",
-    "version": "23.0.0",
+    "version": "23.1.0",
     "description": "Matrix Client-Server SDK for Javascript",
     "engines": {
         "node": ">=16.0.0"
@@ -14,7 +14,7 @@
         "build:dev": "yarn clean && git rev-parse HEAD > git-revision.txt && yarn build:compile && yarn build:types",
         "build:types": "tsc -p tsconfig-build.json --emitDeclarationOnly",
         "build:compile": "babel -d lib --verbose --extensions \".ts,.js\" src",
-        "build:compile-browser": "mkdirp dist && browserify -d src/browser-index.ts -p [ tsify -p ./tsconfig-build.json ] -t [ babelify --sourceMaps=inline --presets [ @babel/preset-env @babel/preset-typescript ] ] | exorcist dist/browser-matrix.js.map > dist/browser-matrix.js",
+        "build:compile-browser": "mkdir dist && browserify -d src/browser-index.ts -p [ tsify -p ./tsconfig-build.json ] -t [ babelify --sourceMaps=inline --presets [ @babel/preset-env @babel/preset-typescript ] ] | exorcist dist/browser-matrix.js.map > dist/browser-matrix.js",
         "build:minify-browser": "terser dist/browser-matrix.js --compress --mangle --source-map --output dist/browser-matrix.min.js",
         "gendoc": "typedoc",
         "lint": "yarn lint:types && yarn lint:js",
@@ -33,10 +33,11 @@
         "matrix-org"
     ],
     "main": "./lib/index.js",
-    "browser": "./lib/browser-index.ts",
+    "browser": "./lib/browser-index.js",
     "matrix_src_main": "./src/index.ts",
     "matrix_src_browser": "./src/browser-index.ts",
     "matrix_lib_main": "./lib/index.js",
+    "matrix_lib_browser": "./lib/browser-index.js",
     "matrix_lib_typings": "./lib/index.d.ts",
     "author": "matrix.org",
     "license": "Apache-2.0",
@@ -54,17 +55,17 @@
     ],
     "dependencies": {
         "@babel/runtime": "^7.12.5",
+        "@matrix-org/matrix-sdk-crypto-js": "^0.1.0-alpha.2",
         "another-json": "^0.2.0",
         "bs58": "^5.0.0",
         "content-type": "^1.0.4",
         "loglevel": "^1.7.1",
         "matrix-events-sdk": "0.0.1",
         "matrix-widget-api": "^1.0.0",
         "p-retry": "4",
-        "qs": "^6.9.6",
         "sdp-transform": "^2.14.1",
         "unhomoglyph": "^1.0.6",
-        "uuid": "7"
+        "uuid": "9"
     },
     "devDependencies": {
         "@babel/cli": "^7.12.10",
@@ -97,7 +98,7 @@
         "browserify": "^17.0.0",
         "docdash": "^2.0.0",
         "domexception": "^4.0.0",
-        "eslint": "8.28.0",
+        "eslint": "8.29.0",
         "eslint-config-google": "^0.14.0",
         "eslint-config-prettier": "^8.5.0",
         "eslint-import-resolver-typescript": "^3.5.1",
@@ -113,7 +114,7 @@
         "jest-localstorage-mock": "^2.4.6",
         "jest-mock": "^29.0.0",
         "matrix-mock-request": "^2.5.0",
-        "prettier": "2.8.0",
+        "prettier": "2.8.1",
         "rimraf": "^3.0.2",
         "terser": "^5.5.1",
         "tsify": "^5.0.2",

post-release.sh

@@ -21,9 +21,9 @@ if [ "$(git branch -lr | grep origin/develop -c)" -ge 1 ]; then
             # to the TypeScript source.
             src_value=$(jq -r ".matrix_src_$i" package.json)
             if [ "$src_value" != "null" ]; then
-                jq ".$i = .matrix_src_$i" package.json > package.json.new && mv package.json.new package.json
+                jq ".$i = .matrix_src_$i" package.json > package.json.new && mv package.json.new package.json && yarn prettier --write package.json
             else
-                jq "del(.$i)" package.json > package.json.new && mv package.json.new package.json
+                jq "del(.$i)" package.json > package.json.new && mv package.json.new package.json && yarn prettier --write package.json
             fi
         fi
     done

spec/TestClient.ts

@@ -24,7 +24,7 @@ import MockHttpBackend from "matrix-mock-request";
 import { LocalStorageCryptoStore } from "../src/crypto/store/localStorage-crypto-store";
 import { logger } from "../src/logger";
 import { syncPromise } from "./test-utils/test-utils";
-import { createClient } from "../src/matrix";
+import { createClient, IStartClientOpts } from "../src/matrix";
 import { ICreateClientOpts, IDownloadKeyResult, MatrixClient, PendingEventOrdering } from "../src/client";
 import { MockStorageApi } from "./MockStorageApi";
 import { encodeUri } from "../src/utils";
@@ -79,9 +79,12 @@ export class TestClient {
     /**
      * start the client, and wait for it to initialise.
      */
-    public start(): Promise<void> {
+    public start(opts: IStartClientOpts = {}): Promise<void> {
         logger.log(this + ": starting");
-        this.httpBackend.when("GET", "/versions").respond(200, {});
+        this.httpBackend.when("GET", "/versions").respond(200, {
+            // we have tests that rely on support for lazy-loading members
+            versions: ["r0.5.0"],
+        });
         this.httpBackend.when("GET", "/pushrules").respond(200, {});
         this.httpBackend.when("POST", "/filter").respond(200, { filter_id: "fid" });
         this.expectDeviceKeyUpload();
@@ -93,6 +96,8 @@ export class TestClient {
         this.client.startClient({
             // set this so that we can get hold of failed events
             pendingEventOrdering: PendingEventOrdering.Detached,
+
+            ...opts,
         });
 
         return Promise.all([this.httpBackend.flushAllExpected(), syncPromise(this.client)]).then(() => {

spec/integ/matrix-client-event-timeline.spec.ts

@@ -1073,6 +1073,97 @@ describe("MatrixClient event timelines", function () {
         });
     });
 
+    it("should ensure thread events are ordered correctly", async () => {
+        // Test data for a second reply to the first thread
+        const THREAD_REPLY2 = utils.mkEvent({
+            room: roomId,
+            user: userId,
+            type: "m.room.message",
+            content: {
+                "body": "thread reply 2",
+                "msgtype": "m.text",
+                "m.relates_to": {
+                    // We can't use the const here because we change server support mode for test
+                    rel_type: "io.element.thread",
+                    event_id: THREAD_ROOT.event_id,
+                },
+            },
+            event: true,
+        });
+        THREAD_REPLY2.localTimestamp += 1000;
+
+        // Test data for a second reply to the first thread
+        const THREAD_REPLY3 = utils.mkEvent({
+            room: roomId,
+            user: userId,
+            type: "m.room.message",
+            content: {
+                "body": "thread reply 3",
+                "msgtype": "m.text",
+                "m.relates_to": {
+                    // We can't use the const here because we change server support mode for test
+                    rel_type: "io.element.thread",
+                    event_id: THREAD_ROOT.event_id,
+                },
+            },
+            event: true,
+        });
+        THREAD_REPLY3.localTimestamp += 2000;
+
+        // Test data for the first thread, with the second reply
+        const THREAD_ROOT_UPDATED = {
+            ...THREAD_ROOT,
+            unsigned: {
+                ...THREAD_ROOT.unsigned,
+                "m.relations": {
+                    ...THREAD_ROOT.unsigned!["m.relations"],
+                    "io.element.thread": {
+                        ...THREAD_ROOT.unsigned!["m.relations"]!["io.element.thread"],
+                        count: 3,
+                        latest_event: THREAD_REPLY3.event,
+                    },
+                },
+            },
+        };
+
+        // @ts-ignore
+        client.clientOpts.experimentalThreadSupport = true;
+        Thread.setServerSideSupport(FeatureSupport.Stable);
+        Thread.setServerSideListSupport(FeatureSupport.Stable);
+        Thread.setServerSideFwdPaginationSupport(FeatureSupport.Stable);
+
+        client.fetchRoomEvent = () => Promise.resolve(THREAD_ROOT_UPDATED);
+
+        await client.stopClient(); // we don't need the client to be syncing at this time
+        const room = client.getRoom(roomId)!;
+
+        const prom = emitPromise(room, ThreadEvent.Update);
+        // Assume we're seeing the reply while loading backlog
+        room.addLiveEvents([THREAD_REPLY2]);
+        httpBackend
+            .when(
+                "GET",
+                "/_matrix/client/v1/rooms/!foo%3Abar/relations/" +
+                    encodeURIComponent(THREAD_ROOT_UPDATED.event_id!) +
+                    "/" +
+                    encodeURIComponent(THREAD_RELATION_TYPE.name),
+            )
+            .respond(200, {
+                chunk: [THREAD_REPLY3.event, THREAD_REPLY2.event, THREAD_REPLY],
+            });
+        await flushHttp(prom);
+        // but while loading the metadata, a new reply has arrived
+        room.addLiveEvents([THREAD_REPLY3]);
+        const thread = room.getThread(THREAD_ROOT_UPDATED.event_id!)!;
+        // then the events should still be all in the right order
+        expect(thread.events.map((it) => it.getId())).toEqual([
+            THREAD_ROOT.event_id,
+            THREAD_REPLY.event_id,
+            THREAD_REPLY2.getId(),
+            THREAD_REPLY3.getId(),
+        ]);
+    });
+
     describe("paginateEventTimeline for thread list timeline", function () {
         const RANDOM_TOKEN = "7280349c7bee430f91defe2a38a0a08c";
 

spec/integ/matrix-client-methods.spec.ts

@@ -35,9 +35,7 @@ describe("MatrixClient", function () {
     let store: MemoryStore | undefined;
 
     const defaultClientOpts: IStoredClientOpts = {
-        canResetEntireTimeline: (roomId) => false,
         experimentalThreadSupport: false,
-        crypto: {} as unknown as IStoredClientOpts["crypto"],
     };
     const setupTests = (): [MatrixClient, HttpBackend, MemoryStore] => {
         const store = new MemoryStore();
@@ -1179,11 +1177,10 @@ describe("MatrixClient", function () {
                 .when("PUT", "/send")
                 .check((req) => {
                     expect(req.data).toStrictEqual({
-                        "msgtype": "m.emote",
-                        "body": "Body",
-                        "formatted_body": "<h1>Body</h1>",
-                        "format": "org.matrix.custom.html",
-                        "org.matrix.msc1767.message": expect.anything(),
+                        msgtype: "m.emote",
+                        body: "Body",
+                        formatted_body: "<h1>Body</h1>",
+                        format: "org.matrix.custom.html",
                     });
                 })
                 .respond(200, { event_id: "$foobar" });
@@ -1199,11 +1196,10 @@ describe("MatrixClient", function () {
                 .when("PUT", "/send")
                 .check((req) => {
                     expect(req.data).toStrictEqual({
-                        "msgtype": "m.text",
-                        "body": "Body",
-                        "formatted_body": "<h1>Body</h1>",
-                        "format": "org.matrix.custom.html",
-                        "org.matrix.msc1767.message": expect.anything(),
+                        msgtype: "m.text",
+                        body: "Body",
+                        formatted_body: "<h1>Body</h1>",
+                        format: "org.matrix.custom.html",
                     });
                 })
                 .respond(200, { event_id: "$foobar" });

spec/integ/matrix-client-syncing.spec.ts

@@ -1543,6 +1543,52 @@ describe("MatrixClient syncing", () => {
                 });
             });
         });
+
+        it("only replays receipts relevant to the current context", async () => {
+            const THREAD_ID = "$unknownthread:localhost";
+
+            const receipt = {
+                type: "m.receipt",
+                room_id: "!foo:bar",
+                content: {
+                    "$event1:localhost": {
+                        [ReceiptType.Read]: {
+                            "@alice:localhost": { ts: 666, thread_id: THREAD_ID },
+                        },
+                    },
+                    "$otherevent:localhost": {
+                        [ReceiptType.Read]: {
+                            "@alice:localhost": { ts: 999, thread_id: "$otherthread:localhost" },
+                        },
+                    },
+                },
+            };
+            syncData.rooms.join[roomOne].ephemeral.events = [receipt];
+
+            httpBackend!.when("GET", "/sync").respond(200, syncData);
+            client!.startClient();
+
+            return Promise.all([httpBackend!.flushAllExpected(), awaitSyncEvent()]).then(() => {
+                const room = client?.getRoom(roomOne);
+                expect(room).toBeInstanceOf(Room);
+
+                expect(room?.cachedThreadReadReceipts.has(THREAD_ID)).toBe(true);
+
+                const thread = room!.createThread(THREAD_ID, undefined, [], true);
+
+                expect(room?.cachedThreadReadReceipts.has(THREAD_ID)).toBe(false);
+
+                const receipt = thread.getReadReceiptForUserId("@alice:localhost");
+
+                expect(receipt).toStrictEqual({
+                    data: {
+                        thread_id: "$unknownthread:localhost",
+                        ts: 666,
+                    },
+                    eventId: "$event1:localhost",
+                });
+            });
+        });
     });
 
     describe("of a room", () => {