New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Multi-tenancy with custom domains #2668

Open
Gargron opened this Issue Apr 30, 2017 · 27 comments

Comments

Projects
None yet
@Gargron
Copy link
Member

Gargron commented Apr 30, 2017

More technical look at #897

Idea: Let users point DNS records to an existing instance, configure specific user to be served as-if from that domain.

What parts must work correctly for this:

  • Separate field for storing whether an account is remote or local rather than relying on domain being null - store custom domain in that column?
  • All functions that check whether an account is local must check the custom domain as well as the "local domain" of the instance
  • Webfinger must answer queries for the faux-remote account
  • Other accounts must not be served from that custom domain. Accessing the website through the custom domain should be limited to only public content from the owner of the domain.

Concerns:

  • Switching instances while keeping the same domain will present the same problem as erasing/reinstalling an old instance on the same domain - other servers will remember data that you don't, any numeric IDs may reset and overlap (but generated string URIs include not just numeric ID, but also timestamp of creation, so that's not a big concern)
  • Might be the first credible case for exporting/importing data verbatim, e.g. including all small details. Just like with local users having a custom domain, statuses should be amended to be local and able to have verbatim URIs like remote ones
@ZiiX

This comment has been minimized.

Copy link
Contributor

ZiiX commented May 1, 2017

Great option to consider! You know I'm for it. With this feature, I'm thinking it would be also awesome to consider personal file/media settings as an option with this. WebDAV, S3, Dropbox, Amazon, Google, etc - this allows their media hierarchy to follow them too.

@ramsey

This comment has been minimized.

Copy link

ramsey commented May 1, 2017

I feel like there are a lot of security/identity concerns to consider, especially with importing data verbatim. I could forge an import and pretend to be someone else, replaying all their posts into the new server. There’s no mechanism for signing posts and asserting one’s identity.

@saper

This comment has been minimized.

Copy link
Contributor

saper commented May 1, 2017

@Gargron what about having a separate domain table and foreign key to that.

Also I think the primary domain should be treated just like all those tenants. Do you have a reason to have it "special"?

@Gargron

This comment has been minimized.

Copy link
Member

Gargron commented May 1, 2017

@saper Initial design decision was:

  • Your domain could be adjusted (even just only at the very start when you're setting up/troubleshooting)
  • Don't want to have 20.000 copies of the same string in the db if it can be avoided by making it null and generating local URIs on the fly
  • Simple distinction for local vs remote content - remote content has a non-null URI

Of course right now we're exploring a use case where, perhaps, that design should be replaced... At the same time, you're right, we could have a separate domain table, then there wouldn't be 20.000 copies of the same string. Though you'd need to join the accounts and domains table on basically every single query.

@saper

This comment has been minimized.

Copy link
Contributor

saper commented May 1, 2017

@jeremyBanks

This comment has been minimized.

Copy link

jeremyBanks commented May 1, 2017

Will it be possible to point to a different user name? If you need to use the same name across instances, that could limit your ability to migrate.

@ThibG

This comment has been minimized.

Copy link
Collaborator

ThibG commented May 1, 2017

Off the top of my head:

  • Some of this is already handled with the distinction between LOCAL_DOMAIN and WEB_DOMAIN, except that you'd need to use a custom user-specific domain if not null instead of LOCAL_DOMAIN, which is easy when generating something, but can get messy when looking up an user. Also, Mastodon still makes some assumptions that are broken when WEB_DOMAIN is different from the user's domain (see #2012 and #2032), although until now, this can be worked around, but…
  • What about username conflicts? Should user@redirected and user@local conflict or not? If not, the workaround for misdirected webfinger queries (y0t4@46555ce) can't work in this setting, as it would need to disambiguate.
  • Further, should the URIs of redirected accounts follow the same template as local ones? If so, and user@redirected and user@local are different users, how to disambiguate? The easiest way to handle that is probably to have https://web_domain/@user@redirected and https://web_domain/users/user@redirected etc.
  • With that in place, using a remote Mastodon instance for your domain seems easy: just redirect (or proxy, but #2375) /.well-known/host-meta (and possibly /.well-known/webfinger), or use a CNAME RR for your domain, if it's to be only used for Mastodon.
  • Migration probably will be a bit tricky: in OStatus, users are not identified by their acct: (user@redirected) URIs but by their ActivityStream actor URI (which is so far an URL to the user's web profile, and will thus reference the “backend” Mastodon instance). The easiest way to migrate seem to just point to another backend, however, your actor URI will most probably change as a result. I'm not sure how bad a thing that is. Also, it won't be immediate, as remote instances having your account details will need to refresh it from webfinger to switch to the new instance, and that's currently only done in a few situations, and at most once a day.
@semarie

This comment has been minimized.

Copy link

semarie commented May 1, 2017

having this kind of configuration at DNS level would be great. But isn't more simple to have it at webserver level ? it would mean having a minimal code on remote webserver side (could be done with simple PHP script and/or webserver redirection).

As example, considere user@mastodon.social who want to use user@my.domain

On the remote side (my.domain):

  • /.well-known/host_meta : reply with LRDD link to webfinger on my.domain
  • /.well-known/webfinger : get information (mostly magic-public-key) from mastodon.social, and build a reply with acct:user@my.domain. Use local URL for atom, salmon, ...
  • for URL entries points (atom, salmon, ...), use HTTP redirection (Location HTTP Header) to point to mastodon.social

On mastodon side (mastodon.social):

  • permit the user to config is account as user@my.domain
  • use acct:user@my.domain in replies
@russss

This comment has been minimized.

Copy link

russss commented May 1, 2017

I'm keen on @semarie's proposal, and in fact I started looking into the feasibility of it a while ago. The advantages as I see it are:

  • You don't need a separate hostname purely for this - which means (for example) my username could be the same as my email address.
  • Multiple users on the same domain could be housed on different Mastodon instances.

This would be similar to how OpenID works, and it feels like a more "federated" situation than pointing domain names around.

It also seems that in the original proposal, the host Mastodon instance would have to obtain an SSL certificate for each domain (or serve them unsecured which is annoying). Even with LetsEncrypt this is fiddly.

@saper

This comment has been minimized.

Copy link
Contributor

saper commented May 1, 2017

here is what I understand under domain multi-tenancy:

Ability to point multiple A and AAAA DNS entries, add a new domain to the certificate and make sure the domains do not get mixed up.

  1. Different domain names are treated like independent instances.

1a the simplest implementation would be just to reuse the code base and point to different databases on request. This is probably achievable now with a web server configuration. I think this should be tested prior to proceeding further (how to talk to redis, store media for multiple instances, mail support etc.).

Every instance is completely independent, has its own member namespace and they federate independently. They only get upgraded at the same time. You have to run db:migrate multiple times, with a new RUBY_ENV for ever domain in the loop.

I think there must be some folk trying this right now?

MediaWiki has an interesting hack that allows sharing only some database tables between such instances, not sure it would be helpful here.

A dedicated asset domain could be another thing, but it should be made in a way that does not break the possibility to export the domain to a fully standalone instance.

1b accept multiple WEB_DOMAINS and issues and hacks related to that. This is more tricky in the implementation as it might require a new domain table, new queries etc. This provides little more benefit to 1a but it carries more of the development effort. The web server/PostgreSQL configuration is simplified compared to (1a)and every domain lives independently in the webspace, with its own member namespace.

Interesting questions appear here, like should those instances automatically be federated or not?
(I think not, they should feel independent).

There is a possibility to shortcut the traffic between the domains internally for performance reasons (maybe not necessary).

1c Accept multiple domains under one certificate - basically it is the scheme used by Google multi-tenant Apps. Everything is under google.com/a/domainname/restoftheurl

The problem here is the initial redirect - how to securely redirect domain.fluffy to mastodon.social/a/domain.fluffy on the web? In case of SMTP it works because nobody validates SMTP domains in certs and we have MX records.

One solution could be to have some custom SRV record that is universally accepted (haven't checked the spec yet). StatusNet, bot frameworks etc should also use it.

A quick check with the WebFinger RFC tells me it only supports HTTP redirect to a different domain and not a well known URL but you still need to be able to accept initial request securely.

And here comes another variant:

2a. Some fluffy form of "user@domain" as an identity alias on mastodon.social without DNS entries. I don't know how would that possibly work, except for the user being mrserious@mastodn.socialvall the time on the protocol level and just bring presented as @pinky@fluffyanimal.farm on mastodon.social only.

2b. A variant of (2a) could be considered where mastodon.social tries to convince other instances that mrserious wants to be presented as @pinky@fluffyanimal.farm, which opens a whole can of worms (misrepresentation).

May be this use case should be further elaborated (maybe it is a separate issue).

I think there is a natural development progression to go from 1a via 1b maybe to 1c.

it would be good to refer to one of those variants as we speak (or define additional ones)

@s-leroux

This comment has been minimized.

Copy link

s-leroux commented May 1, 2017

Great idea. But let me rephrase to see if I understand the typical use case:

Someone connects to me as @sylvain@yesik.it

First we query the yesik.it DNS for the _mastodon TXT record.
If not found, "yesik.it" is assumed to run its own Mastodon instance.
Otherwise, the TXT record (CNAME?) holds my current instance. Say its "mastodon.social"

So, on the background, Mastodon will understand the physical account name is @sylvain@mastodon.social
But in the UI and for all user interactions, only my logical account name is used @sylvain@yesik.it

Basically we add here a new layer of indirection. Or did I missed the point?


Several remarks:

  1. It doesn't help if someone bookmarked my account URL (https://mastodon.social/@YesIKnowIT). Or does it?
  2. What if several members of the same domain have accounts on different instances?
  3. What if I actually made several domains point toward the same account (aka, @sylvain@chicoree.fr, @sylvain@yesik.it ... being different alias to the same account -- just like I can do it for my email).
@saper

This comment has been minimized.

Copy link
Contributor

saper commented May 2, 2017

@s-leroux so what you mean is something like federated alias.

Basically one more assumption had to be broken - not all users from @Domain are hosted on a single instance.

That is another piece of complexity, basically "domain" server would need to provide redirects/pointers,
but I understand that the trouble is for the target host to accept user@otherdomain stuff and somehow treat it's as own

Not sure it's multi-tenant domain handling, though

@alexgleason

This comment has been minimized.

Copy link
Contributor

alexgleason commented May 3, 2017

I like this idea a lot, but not sure the benefits outweigh the costs.

I think the goal is to give users more control. The easiest way to do that now is to run your own instance. That's hard though for non-tech users, while setting some DNS values is easy given a tutorial.

Would it be better to make it easy to start your own instance? I feel like a 1-click install on a web host would be ideal. I think IndieHosters is supporting Mastodon, maybe they'd be willing to set something up like this. @pierreozoux

Basically I'm wondering what the goals actually are and how to best reach those goals. I can definitely see the benefits of supporting remote domain names, but running your own instance is more beneficial: you own/control your data, there's no reason to migrate, and it produces no technical debt to the project itself. The paradigm currently in place makes sense: if you're part of a server, you're part of that community and you take their name. If you're remote, you have a remote name. This is intuitive.

On the other hand, email has some similar paradigms and any service can send emails from your domain name with the proper authentication (eg Postmark). But this is usually a generic service and to my knowledge is not built into any email providers themselves.

@pierreozoux

This comment has been minimized.

Copy link
Contributor

pierreozoux commented May 3, 2017

I think it is a tricky discussion.

In one hand, with one postfix, I can host various domains, and it is convenient.
As a user, I can migrate from one hoster to another, it is a bit painful, but it is doable.

On the other hand, on the web, one domain name means the control over your own piece of land, your rules, your design, so it also makes sense to have your own instance (from the indieweb principles).

It is not that hard to host a mastodon instance, or pay somebody to do it. And then, it is easy to just transfer from one server to another, or from one hoster to another. And the change that has to be made to add this feature are quiet heavy I guess. Then, they also have to be maintained.

I think that multitenancy, is always a headache for developers, and it is never really well done, so even if I have the option, as a hoster, I never choose that. It is always easier to migrate people with their own instances.
(All the options I hear, DNS, running a small app server will touch 0.01% of our user base IMHO)

I'm probably not the best judge to help here, but I think the discussion has to evolve around cost/benefit for the software in itself. I'm a fan of unix philosophy, and I think the effort can serve a lot better in other areas :)

And to conclude, I also think that the discussion should happen at the standard itsefl, at ostatus level.

@jeremyBanks

This comment has been minimized.

Copy link

jeremyBanks commented May 3, 2017

"It's not hard to host a Mastodon instance", said one engineer to the next.

Dozens of projects have told themselves this, but the reality of self-hosting applications like this is that it does not happen for a non-trivial number of users, despite great efforts like Sandstorm. One-click install for end-users is not a responsible solution, unless you also have automatic fast and secure updating on multiple levels. We might get there one day, but don't let that ideal get in the way of a realistic solution.

Configuring DNS entries as a one-time (or at least rare) action is something that a moderately tech-competent person can handle. It will at least get Mastodon on par with email in terms of identifier portability, and is probably the most viable mid-term solution (assuming the current identifier scheme is preserved instead of moving to something cryptographic).

@DanielHeath

This comment has been minimized.

Copy link

DanielHeath commented Jun 29, 2017

Postfix is a good example of something that is easy to operate.
Enable unattended-upgrades and security patches happen magically.
You can automatically transfer all your mail from one provider to another via standard APIs.
This is still not enough - the vast majority of users prefer free-with-ads (eg gmail) or pay-for-hosting (eg fastmail).

Hosting your own Mastodon instance means caring about:

  • Backups (you presumably care about your own history)
  • Patching security bugs quickly (Much as I love rails, it requires more frequent patches than other stuff I run)
  • SSL (letsencrypt has made this easier, but it's still a job to setup)
  • Diagnosing & fixing issues (if something goes wrong, can you pull the relevant bit of the logs to file a bug?)
  • Uptime (ok, it's a personal blog, but it still needs to be fixed when it crashes)

I build and operate production services for work - I don't need or want another thing to carry pager for.

@DanielHeath

This comment has been minimized.

Copy link

DanielHeath commented Jun 29, 2017

@s-leroux I think a mastodon-specific TXT record is unnecessary; a regular old CNAME should do fine.

I think that separate-but-related is the ability to redirect an account from an old name to a new name. It's nice to be able to change your identity (even on the same domain) without breaking all the old URLs and losing your federated followers - there's no reason this sort of redirect shouldn't work across domains too.

@s-leroux

This comment has been minimized.

Copy link

s-leroux commented Jun 30, 2017

@DanielHeath The problem with CNAME is cannot coexist with any other record for the same resource (except DNSSEC-related records).

In particular, you cannot have both a CNAME and a MX record for the same domain.
Practically, if you use a CNAME to point your bare domain to some "mastodon provider", you no longer are able to use a mail provider on your bare domain. The workaround being to create a sub-domain per service. But that feels ugly to me.

@DanielHeath

This comment has been minimized.

Copy link

DanielHeath commented Jun 30, 2017

@s-leroux

This comment has been minimized.

Copy link

s-leroux commented Jun 30, 2017

@DanielHeath I understand your point of view concerning someone having full access to his server.

But imagine a business or other kind of organization deciding to provide Mastodon access to their team.

However, they already have mail and web site on their primary naked domain (MX and A records). In most cases those services are externalized. The web server having chances of being a mutualized one.
The only standard record that might "bind" their domain to some mastodon provider would be the SRV record. Unless we force the user to create a specific sub-domain? Anyway, in the cases I have in mind, if I explain to them they will now have to host themselves a server just to install a reverse proxy or some other service -- surely they will answer, "never mind, we will continue using TweetDeck".

My point is if we want to target a wide audience, any viable solution should satisfy those two constraints:

  1. It should be possible to co-host Mastodon as well as classical email and web services on the primary naked domain
  2. For the purpose of adding the "Mastodon alias", we should assume the user only has access to his domain DNS configuration.
@DanielHeath

This comment has been minimized.

Copy link

DanielHeath commented Jun 30, 2017

@duck57

This comment has been minimized.

Copy link

duck57 commented Jul 6, 2017

One other use case for this (assuming I understand correctly) is for a current self-hosting user to add other domains to their existing instance. This probably will mostly be used for novelty accounts, but having the ability to use one instance of Mastodon on your server to serve several domains would be a nice-to-have feature and there are probably better uses than silly account domain names that I'm just not thinking of at the moment.

@azbulutlu

This comment has been minimized.

Copy link

azbulutlu commented Jul 6, 2017

some questions, as a user.

  1. I use shared hosting, I have my domain, that has my website and email address. Can I use this feature? (not everyone can afford to pay for additional vps hosting. not everyone has time/skill/energy to run a server. even if a person has a vps, running a public facing server with good uptime is a lot of overhead on top of that)

  2. What is my profile url? (note that when giving social media links on a website, I use the profile url).

  3. what is my mastodon handle? (if I give someone my profile link, ideally they should be able to add me back, like they do with the profile url of a regular account. So follow button, or typing @username@customdomain or pasting the link and finding the account that way)

  4. (potentially silly question) in terms of the idea for shared certificate, my domain already has a certificate, does that mean I cannot use this domain?

what are the answers in case I use a subdomain that is dedicated to mastodon redirecting instead (e.g. social.mycustomdomain.com)

@duck57 A scenario would be that: you can have 2 domains (e.g. both your personal page and a project/blog) You might want to have "official" accounts for both, without having 2 mastodon installs (which would be a big overhead). It is not uncommon to have separate social media accounts for a personal project vs personal page, even if the project is not that of a group.

@duck57

This comment has been minimized.

Copy link

duck57 commented Jul 10, 2017

@azbulutlu Thanks for coming up with a much more "professional" use case than I thought of. One install per server makes much more sense than one install per domain. Perhaps you could use some NGINX magic to proxy traffic from one domain to the other as an interim solution, but someone with more NGINX experience than I should comment if this is actually feasible as a solution or not.

@PubliqPhirm PubliqPhirm referenced this issue Nov 16, 2017

Open

Account aliases. #3796

1 of 2 tasks complete
@DanielHeath

This comment has been minimized.

Copy link

DanielHeath commented Jul 20, 2018

@duck57 trying to proxy is fraught - you have to correctly rewrite every instance of the domain name in the body of all resources, but you also have to consider whether you're breaking CSP (since you might have changed the body of a resource which has a hash provided).

It can be made to work but it's not something I'd want to rely on, especially if I'll be updating the backend over time (which might introduce new ways for the nginx config to be wrong).

@duck57

This comment has been minimized.

Copy link

duck57 commented Jul 25, 2018

@DanielHeath Something I have seen people do is use cnames, but I'm not sure if that solves this issue or simply allows a user to access multiple accounts on a single instance through an app that assumes one account per instance per user.

@DanielHeath

This comment has been minimized.

Copy link

DanielHeath commented Jul 25, 2018

If I cname toots.daniel.heath.cc to mastodon.social, and load up http://toots.daniel.heath.cc, I just get a 'welcome to nginx' page.

@Gargron Gargron added suggestion and removed enhancement labels Oct 20, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment