Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MSC3859: Add well known media domain proposal #3859

Closed
wants to merge 9 commits into from

Conversation

Fizzadar
Copy link
Contributor

@Fizzadar Fizzadar commented Aug 4, 2022

Rendered

Signed off by Nick @ Beeper (@Fizzadar) nick@beeper.com.

UPDATE (2022-08-18): we are closing this MSC in favour of MSC3860 & MSC3870.

@Fizzadar Fizzadar changed the title Add well known media domain proposal MSC3859: Add well known media domain proposal Aug 4, 2022
proposals/3859-well-known-media-domain.md Outdated Show resolved Hide resolved
proposals/3859-well-known-media-domain.md Outdated Show resolved Hide resolved
proposals/3859-well-known-media-domain.md Outdated Show resolved Hide resolved
@turt2live turt2live added proposal A matrix spec change proposal client-server Client-Server API kind:feature MSC for not-core and not-maintenance stuff needs-implementation This MSC does not have a qualifying implementation for the SCT to review. The MSC cannot enter FCP. labels Aug 4, 2022
@turt2live
Copy link
Member

For sign off: We will need an email address, or the form completed with dco@matrix.org

}
```

As above, the non-media endpoint must also serve media requests.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please follow the proposal template. You're missing some sections that I would expect to see for this type of proposal, in particular "Alternatives", "Security considerations", and "Unstable prefix".

Some possible alternatives:

  • allow the homeserver to send an HTTP (3xx status code) on /media queries
  • include the media server in the /login response, rather than in .well-known

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missng sections added in 86123b0.

Re: login response I don't think this works because media is accessible without authentication with a known MXID which login would prevent.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re: login response I don't think this works because media is accessible without authentication with a known MXID which login would prevent.

That seems reasonable. I'd suggest still including that in the "Alternatives" section -- part of the purpose of that section is to indicate other possible ways to achieve the goal, and say why the proposed solution is better.

proposals/3859-well-known-media-domain.md Outdated Show resolved Hide resolved
proposals/3859-well-known-media-domain.md Outdated Show resolved Hide resolved
proposals/3859-well-known-media-domain.md Outdated Show resolved Hide resolved
Comment on lines +15 to +17
This cannot be achieved currently unless the entire homeserver is served behind a CDN, which may have other cost or
management implications (or simply be impossible, depending on CDN). A separate domain allows server admins to
decide the best setup for their users.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This needs expanding, or at least covering in the alternatives. As mentioned, for my own server I'm one flag away from having CDN-driven media without spec changes. Is there a particular CDN provider in mind for your use case which prevents working within the existing spec?

For reference, the setup I've been looking at is effectively deploying my own CDN on top of a cloud provider: there's a reverse proxy in several geographical areas using DNS to direct traffic there, with the media repo running in that datacenter to serve media from its CDN-backed backend (so all media is available everywhere). The traffic routing for client-server and server-server traffic is effectively unchanged in this setup, though might use a backbone back to the real server hosted in a single geographical area. The latency would be nearly the same as if a user was talking to the server over an ocean directly.

Granted, this is not a super cost effective solution, but it does work within how Matrix expects a server to operate. In practice, we'd also want to support proper geographically-segmented homeservers for traffic balancing in that respect, however that's much harder to do (and likely an implementation problem rather than spec).

A benefit of this proposal is it allows the media domain to be different from the homeserver domain, which is also something we want to do, though I do wonder if making the media repo be a proper first class citizen in the spec is a good idea.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will add this to alternatives! The reverse proxy / CDN over everything solution certainly works but forces all traffic to be routed via that solution. Our use case specifically is we'd like to run Synapse inside AWS (as we currently do) but have the media routes served elsewhere due to AWS' prohibitive bandwidth costs. Without a separate domain the only option for that is a solution similar to yours where a matrix-aware reverse proxy needs to sit on the CDN side routing requests accordingly.

While this is certainly possible (cloudflare workers, fly.io, custom build CDN) it's represents a significant amount of work vs. a separate domain which to me feels like a very simple and easy solution. I can't see any negatives to doing this (esp. as an optional field).

Does having a separate domain really make the media repo a first class citizen? Perhaps there should be some kind of reliance on a homeserver to avoid that, or perhaps it's actually a good thing? (Multiple homeserver owners could share a media repo install or something along those lines, way beyond this proposal).

proposals/3859-well-known-media-domain.md Outdated Show resolved Hide resolved
Fizzadar and others added 3 commits August 9, 2022 08:37
Co-authored-by: Travis Ralston <travpc@gmail.com>
Co-authored-by: Travis Ralston <travpc@gmail.com>
Co-authored-by: Travis Ralston <travpc@gmail.com>
### Extend the client .well-known

Add a new optional field, `m.media_server` that can specify a separate URL to be used for media requests. Clients can
then optionally use this field for all media endpoints, including both download and upload.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If it's for upload as well, I'm guessing the client is expected to use the same access token as it would use when communicating with the homeserver. Is that correct?

@Fizzadar
Copy link
Contributor Author

Fizzadar commented Aug 17, 2022

Based upon internal discussion we no longer think this is the right approach and thus we are closing this MSC in favour of MSC3860 & MSC3870.

I will leave the branch up for context and if anyone wishes to take this on under a new MSC.

@Fizzadar Fizzadar closed this Aug 17, 2022
@turt2live turt2live added the obsolete A proposal which has been overtaken by other proposals label Aug 17, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
client-server Client-Server API kind:feature MSC for not-core and not-maintenance stuff needs-implementation This MSC does not have a qualifying implementation for the SCT to review. The MSC cannot enter FCP. obsolete A proposal which has been overtaken by other proposals proposal A matrix spec change proposal
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants