MSC2246: Asynchronous media uploads #2246

tulir · 2019-08-24T13:51:24Z

Implementations:

Server: Asynchronous uploads (MSC2246) t2bot/matrix-media-repo#364, MSC2246 async uploads synapse#12484, Asynchronous Uploads synapse#15503
Client upload (actually a bridge): mautrix/go@3b99110 + mautrix/whatsapp@f79ca42
Client download should work with all existing clients to some extent with the 20s default stalling

Signed-off-by: Tulir Asokan tulir@maunium.net

Signed-off-by: Tulir Asokan <tulir@maunium.net>

ara4n · 2019-08-24T14:11:31Z

sounds excellent to me; would love to unblock streaming transfers at last.
need to consider risk of DoS by creating lots of IDs, and define the json if any handed to the /create (for instance, could be useful in future for defining whether a give mxc url should be locked behind auth of some kind)

Signed-off-by: Tulir Asokan <tulir@maunium.net>

turt2live

also generally looking in the right direction - I've left some early comments which are hopefully helpful.

proposals/2246-asynchronous-uploads.md

Co-Authored-By: Travis Ralston <travpc@gmail.com>

bmarty · 2019-08-27T17:04:10Z

The idea is then to send a media event, after the new /create endpoint, and before sending the media itself; isn't it? We have to consider the failure of the upload: it can lead to media event with eternal not found media.
Also the client should have a way to know that the media is not yet available to the server for a better UI, by displaying a waiting message (ex: Bob is sending an image) for instance?

proposals/2246-asynchronous-uploads.md

Spec for matrix-org/matrix-spec-proposals#2246

turt2live · 2023-05-03T15:49:42Z

Merged 🎉

See matrix-org/matrix-spec-proposals#2246 for details Signed-off-by: Sumner Evans <sumner@beeper.com>

As of MSC2246 [1], the PUT method is allowed on the /_matrix/media/v3/upload endpoint. See the spec documentation here: https://spec.matrix.org/v1.7/client-server-api/#put_matrixmediav3uploadservernamemediaid [1]: matrix-org/matrix-spec-proposals#2246 Signed-off-by: Sumner Evans <sumner@beeper.com>

See matrix-org/matrix-spec-proposals#2246 for details Signed-off-by: Sumner Evans <sumner@beeper.com>

As of MSC2246 [1], the PUT method is allowed on the /_matrix/media/v3/upload endpoint. See the spec documentation here: https://spec.matrix.org/v1.7/client-server-api/#put_matrixmediav3uploadservernamemediaid [1]: matrix-org/matrix-spec-proposals#2246 Signed-off-by: Sumner Evans <sumner@beeper.com>

See matrix-org/matrix-spec-proposals#2246 for details Signed-off-by: Sumner Evans <sumner@beeper.com>

tezlm · 2023-10-17T03:53:34Z

I know I'm very late, but it's surprising that nobody mentioned tus yet, which allows for resumable file uploads. Since the core is headers + PATCH it can luckily be retrofitted into matrix in a future msc, if anyone wants it.

See matrix-org/matrix-spec-proposals#2246 for details Signed-off-by: Sumner Evans <sumner@beeper.com>

sumnerevans · 2023-11-21T17:12:16Z

For future code sleuths:

Async uploads PR: matrix-org/synapse#15503

Released in Synapse:

https://github.com/matrix-org/synapse/releases/tag/v1.97.0rc1

Proposal for asynchronous media uploads

b439277

Signed-off-by: Tulir Asokan <tulir@maunium.net>

Add security consideration and mention possible /create request body

a83c79c

Signed-off-by: Tulir Asokan <tulir@maunium.net>

turt2live added the proposal A matrix spec change proposal label Aug 24, 2019

Expand on security consideration

9a395ed

turt2live self-requested a review August 26, 2019 02:50

turt2live previously requested changes Aug 26, 2019

View reviewed changes

tulir and others added 2 commits August 26, 2019 22:51

Change error code for existing media PUT

29e3463

Co-Authored-By: Travis Ralston <travpc@gmail.com>

Add spam prevention error, example response and auth requirements

7cf22be

manuroe reviewed Aug 28, 2019

View reviewed changes

proposals/2246-asynchronous-uploads.md Outdated Show resolved Hide resolved

turt2live changed the title ~~MSC2246: Asynchronous media uploads~~ [WIP] MSC2246: Asynchronous media uploads Dec 7, 2019

turt2live added the kind:feature MSC for not-core and not-maintenance stuff label Apr 20, 2020

tulir mentioned this pull request Jun 18, 2020

When multiple messages are forwarded on telegram, they come to matrix on random order mautrix/telegram#486

Closed

tulir mentioned this pull request Aug 1, 2020

Preserve message order when bridging from Telegram to Matrix mautrix/telegram#505

Closed

tulir added 4 commits August 2, 2020 01:38

Add query parameters to control downloading

658aac8

Rename max_stall to max_stall_ms

bbd7d08

Merge branch 'master' into asynchronous_uploads

0bffcb7

Specify /create request body content

4d009a9

tulir changed the title ~~[WIP] MSC2246: Asynchronous media uploads~~ MSC2246: Asynchronous media uploads Aug 1, 2020

tulir marked this pull request as ready for review August 1, 2020 22:46

turt2live added the proposal-in-review label Aug 1, 2020

This comment was marked as off-topic.

Sign in to view

turt2live added the needs-implementation This MSC does not have a qualifying implementation for the SCT to review. The MSC cannot enter FCP. label Jun 8, 2021

turt2live force-pushed the old_master branch from e895827 to dca99ee Compare August 30, 2021 22:34

tulir added 3 commits March 14, 2022 12:30

Update links, versions and prefixes

1cbc04e

Add recommended maximum delay for starting upload

c65f2bf

Explicitly deny uploading to other users' media IDs

63cef50

ShadowJonathan reviewed Mar 14, 2022

View reviewed changes

proposals/2246-asynchronous-uploads.md Outdated Show resolved Hide resolved

proposals/2246-asynchronous-uploads.md Outdated Show resolved Hide resolved

proposals/2246-asynchronous-uploads.md Outdated Show resolved Hide resolved

Prevent spam by ratelimiting instead of limiting number of IDs

8ccf85f

richvdh added merged A proposal whose PR has merged into the spec! and removed spec-pr-in-review A proposal which has been PR'd against the spec and is in review labels May 3, 2023

richvdh added a commit to matrix-org/matrix-spec that referenced this pull request May 3, 2023

Spec asynchronous uploads (#1499)

283c587

Spec for matrix-org/matrix-spec-proposals#2246

This was referenced May 4, 2023

client-api: Fixes for Asynchronous media uploads ruma/ruma#1542

Merged

Implement Matrix v1.7 changes ruma/ruma#1520

Closed

clokep mentioned this pull request May 23, 2023

Declare support for Matrix v1.7 matrix-org/synapse#15661

Closed

12 tasks

sumnerevans added a commit to beeper/synapse-legacy-fork that referenced this pull request Jun 9, 2023

media/create: add /create endpoint

03578be

See matrix-org/matrix-spec-proposals#2246 for details Signed-off-by: Sumner Evans <sumner@beeper.com>

sumnerevans mentioned this pull request Jun 9, 2023

unknown endpoints: allow PUT /_matrix/media/v3/upload matrix-org/complement#638

Merged

1 task

sumnerevans added a commit to beeper/synapse-legacy-fork that referenced this pull request Jun 9, 2023

media/create: add /create endpoint

cf0971f

See matrix-org/matrix-spec-proposals#2246 for details Signed-off-by: Sumner Evans <sumner@beeper.com>

sumnerevans added a commit to beeper/synapse-legacy-fork that referenced this pull request Jun 9, 2023

media/create: add /create endpoint

2d6840c

See matrix-org/matrix-spec-proposals#2246 for details Signed-off-by: Sumner Evans <sumner@beeper.com>

sumnerevans added a commit to beeper/synapse-legacy-fork that referenced this pull request Jun 10, 2023

media/create: add /create endpoint

838511b

See matrix-org/matrix-spec-proposals#2246 for details Signed-off-by: Sumner Evans <sumner@beeper.com>

sumnerevans added a commit to beeper/synapse-legacy-fork that referenced this pull request Jun 10, 2023

media/create: add /create endpoint

a473232

See matrix-org/matrix-spec-proposals#2246 for details Signed-off-by: Sumner Evans <sumner@beeper.com>

sumnerevans added a commit to beeper/synapse-legacy-fork that referenced this pull request Aug 14, 2023

media/create: add /create endpoint

4da752a

See matrix-org/matrix-spec-proposals#2246 for details Signed-off-by: Sumner Evans <sumner@beeper.com>

sumnerevans added a commit to beeper/synapse-legacy-fork that referenced this pull request Aug 14, 2023

media/create: add /create endpoint

ba03a12

See matrix-org/matrix-spec-proposals#2246 for details Signed-off-by: Sumner Evans <sumner@beeper.com>

sumnerevans added a commit to beeper/synapse-legacy-fork that referenced this pull request Aug 23, 2023

media/create: add /create endpoint

0c94a2b

See matrix-org/matrix-spec-proposals#2246 for details Signed-off-by: Sumner Evans <sumner@beeper.com>

sumnerevans added a commit to beeper/synapse-legacy-fork that referenced this pull request Sep 5, 2023

media/create: add /create endpoint

c826981

See matrix-org/matrix-spec-proposals#2246 for details Signed-off-by: Sumner Evans <sumner@beeper.com>

sumnerevans added a commit to beeper/synapse-legacy-fork that referenced this pull request Sep 5, 2023

media/create: add /create endpoint

12c9353

See matrix-org/matrix-spec-proposals#2246 for details Signed-off-by: Sumner Evans <sumner@beeper.com>

S7evinK mentioned this pull request Oct 2, 2023

Declare support for Matrix 1.7 matrix-org/dendrite#3225

Open

16 tasks

sumnerevans added a commit to beeper/synapse-legacy-fork that referenced this pull request Oct 24, 2023

media/create: add /create endpoint

d4b2bb7

See matrix-org/matrix-spec-proposals#2246 for details Signed-off-by: Sumner Evans <sumner@beeper.com>

sumnerevans added a commit to beeper/synapse-legacy-fork that referenced this pull request Oct 24, 2023

media/create: add /create endpoint

e673172

See matrix-org/matrix-spec-proposals#2246 for details Signed-off-by: Sumner Evans <sumner@beeper.com>

sumnerevans added a commit to beeper/synapse-legacy-fork that referenced this pull request Oct 24, 2023

media/create: add /create endpoint

958900d

See matrix-org/matrix-spec-proposals#2246 for details Signed-off-by: Sumner Evans <sumner@beeper.com>

sumnerevans added a commit to beeper/synapse-legacy-fork that referenced this pull request Oct 24, 2023

media/create: add /create endpoint

e11f51c

See matrix-org/matrix-spec-proposals#2246 for details Signed-off-by: Sumner Evans <sumner@beeper.com>

sumnerevans added a commit to beeper/synapse-legacy-fork that referenced this pull request Oct 26, 2023

media/create: add /create endpoint

447f353

See matrix-org/matrix-spec-proposals#2246 for details Signed-off-by: Sumner Evans <sumner@beeper.com>

sumnerevans added a commit to beeper/synapse-legacy-fork that referenced this pull request Nov 9, 2023

media/create: add /create endpoint

f0d53e8

See matrix-org/matrix-spec-proposals#2246 for details Signed-off-by: Sumner Evans <sumner@beeper.com>

clokep mentioned this pull request Nov 15, 2023

async uploads: add tests matrix-org/complement#641

Merged

1 task

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

MSC2246: Asynchronous media uploads #2246

MSC2246: Asynchronous media uploads #2246

tulir commented Aug 24, 2019 •

edited by anoadragon453

ara4n commented Aug 24, 2019

turt2live left a comment

bmarty commented Aug 27, 2019

This comment was marked as off-topic.

turt2live commented May 3, 2023

tezlm commented Oct 17, 2023

sumnerevans commented Nov 21, 2023

MSC2246: Asynchronous media uploads #2246

MSC2246: Asynchronous media uploads #2246

Conversation

tulir commented Aug 24, 2019 • edited by anoadragon453

ara4n commented Aug 24, 2019

turt2live left a comment

Choose a reason for hiding this comment

bmarty commented Aug 27, 2019

This comment was marked as off-topic.

turt2live commented May 3, 2023

tezlm commented Oct 17, 2023

sumnerevans commented Nov 21, 2023

tulir commented Aug 24, 2019 •

edited by anoadragon453