Distributed email based federated cloud id discovery

[icewind1991]

Federated cloud ids are often far from ideal (foo@bar@example.com, foo@example.com/cloud foo@cloud.example.com instead of just foo@example.com) because it needs to contain all information needed to contact the remote server.

It would be a lot nicer for users if they can just enter an email in the share field and have automatically resolve that to a remote (or local) user.

One option to resolve emails would be to have one (or more) centralized servers which contain a mapping but centralized components is something which should be avoided where possible.

For instances where people use their own domain for emails (which would cover most larger instances) we could use the control the admin has over the mail domain to setup a distributed resolver.
Some ways this could be done would be

• have a TXT dns entry which points to the resolver endpoint
• have something like example.com/.well-known/cloud-resolver point to the resolver endpoint

Where the resolver would be a simple api that matches federated cloud ids to emails for all users in the instance.

(note that we can't have the resolver itself in .well-known since it needs to be possible to have the resolver on a different server than the one that server the main site.)

The full process when sharing would be something like

1. Check if the provided email belongs to a local user
2. See if the domain provides a resolver
3. Check if the email belongs to a cloud id
4. setup the share with the discovered cloud id

Nothing of it should be hard to implement we just need to make sure we decide on some standard way an admin can setup a resolver.

Personally I would prefer the dns route but either of the discovery methods should work (or we could just support both)

cc @LukasReschke @karlitschek @schiessle @rullzer @jancborchardt

[Bugsbane]

Sounds sensible to me, especially given that people are already to accustomed to using email addresses as usernames, to the point that I know many people have already mistaken federated cloud id's email addresses. If email addresses and federated id's were the same and would work for either purpose, it would very likely eliminate some of the accidents, going back and forth with misunderstandings about which was which. This is especially true when cloud ID's are already being used to log in and check email.

[karlitschek]

Thats a neat idea. And I like the fact that this doesn't require a centralized instance of course, beside DNS. The problem is that not a lot of people have access to their DNS server or have the knowledge to configure this. So this is probably only something for the 2% while we still need something for the 98%

But why not as an additional option. What do you think @schiessle ?

[rullzer]

Can't we as a fallback just send an e-mail?

@schiessle already had an awesome PR to change the flow of mounting a link share. Where it is actually converted into a federated share on mount.

So if the e-mail adress I'm sending to can't be resolved to a federated cloud id. We send an e-mail with a special link. This could show some info about the share (or just the public link view?). Then the user has a big button and to your nextcloud.

Because of the shared addressbook once the share is accepted in most cases the addressbooks will sync. So the next time they can more easy find the sharee.

[MorrisJobke]

Yes. We need something for all the gmail and gmx and t-online email addresses as well. And there the "send an email with a special link" is the best option that also doesn't leak any private information (no central directory). Based on this the sending instance then also could get reported back what cloud id was used to receive this share and store this link.

Regarding this feedback to the sending instance: maybe allow the addressbook sync in a more privacy aware way - only sync the cloud ids with email addresses to those instances that send me a link for this person:

Server A:

• user1
• user2
  Server B:
• user 3
• user 4
  Server C:
• user 5
• user 6

Now "user 1" shares with "user 3" and "user 4" shares with "user 6". The sync will now include following after both of the shares are accepted:

• "user 1" is synced to Server B
• "user 3" is synced to Server A
• "user 4" is synced to Server C
• "user 6" is synced to Server B

Following user data is not shared (even if the server trust each other):

• "user 2" to Server B
• "user 4" to Server A
• "user 3" to Server C
• "user 5" to Server B

This would remove the gut feeling for me, because I don't want to sync all my users to a remote server, but only the ones that actually connect each other. Or is this too much and wouldn't help here?

[LukasReschke]

Seems like a new feature => Moving to 11

[patricktokeeffe]

I'm currently deciding between user@domain.com/cloud or user@cloud.domain.com - neither is very nice. I'd much rather have user@domain.com correctly redirect to somewhere else.

I agree with OP this is prob best done as a dns TXT record since it's simple and robust. But support for the .well-known approach would be good as well since it'd only require access to the domain's web server.

Could the resolver api be as simple as, for domain.com (with nextcloud at cloud.domain.com/):

@ 10800 IN TXT "nextcloud-resolver=cloud.domain.com"

where /.well-known/nextcloud-resolver contains

cloud.domain.com

I recognize this doesn't support third-party email domains (GMX, Gmail, etc) but that seems like a smaller, separate problem where email makes more sense.

[tbrasser]

I'm sorry if this is the wrong place, but I've been thinking about something similar to accomplish with my Nextcloud instance and whole Home Server (and domain name) set-up.

My current situation:
Nextcloud domain: https://cloud.mydomain.family
Email scheme: username@mydomain.family
Federated Cloud ID: username@cloud.mydomain.family
What I have in mind:
Nextcloud domain: https://mydomain.family
E-mail scheme: username@mydomain.family
Federated Cloud ID: username@mydomain.family
Would this work right now already? (Since nextcloud will run on my main domain root which would result in the correct Federated Cloud ID's) Or is this the right issue to keep track of, because it still needs to redirect?

(And although I've yet to dive in to the wonderful world of LDAP/AD, in an utopia I'd like to set-up my home network and Nextcloud and other stuff with LDAP and have the UID also be username@mydomain.family. One can dream...)

[schiessle]

@tbrasser this would already work today. Just install Nextcloud on your domain root and create a user with the same username as your email alias.

[alexgleason]

It would be great if Nextcloud IDs corresponded to email addresses because that would work seamlessly.

I would also like support for multiple domains under the same Nextcloud install. This corresponds with the ability to host multiple email domains on the same mail server.

[tbrasser]

@alexgleason the first thing you mention works as @schiessle mentioned. You can also add multiple domains to nextcloud's trusted domains, but I think federated cloud ID's will only work for the main domain? (not entirely sure about that)

[mirsal]

AFAIK, the best practice for dns-based service discovery is not using TXT but SRV records, which would look like:

_nextcloud._tcp.example.com. 3600 IN SRV 10 0 443 cloud.example.com.

for user@example.com federated cloud IDs on https://cloud.example.com/

See:

• https://tools.ietf.org/html/rfc2782
• https://en.wikipedia.org/wiki/SRV_record

[MorrisJobke]

Nothing for 14 it seems -> moved to 15

[mikecardwell]

This seems even more important to me now that Nextcloud 15 was released with federated social features. My server can be found at (obfuscated using example.com) - https://cloud.example.com - Because my main website is at https://example.com - My cloud username is firstname, but my email address starts firstname.surname. So my email address, which I want people to use to find me on the fediverse is firstname.surname@example.com, yet at the moment, they would have to use firstname@cloud.example.com

I'm not going to start building an identify on the fediverse using firstname@cloud.example.com so I'm not going to touch Nextclouds new social federation functionality. I will wait until I can use my commonly known identity instead, i.e firstname.surname@example.com

[mikecardwell]

In addition to my previous comment, you may want to look into PKA at https://keyserver.mattrude.com/guides/public-key-association/

It allows you to associate an email address with a URL for PGP, in the DNS. E.g, if you want to find the PGP key for matt@mattrude.com (from the above URL), you would do a TXT record lookup of matt._pka.mattrude.com:

$ dig +short txt matt._pka.mattrude.com
"v=pka1;fpr=71FD20E328158C322133FBBEC4909EE495B0761F;uri=https://mattrude.com/keys/0xc4909ee495b0761f.asc"

I feel like something similar for Nextcloud would be ideal. So I could just do e.g:

$ dig firstname.surname._nextcloud.example.com
"firstname@cloud.example.com"

[jhesketh]

The challenge with your suggestions is that they would require a DNS entry for each user in the cloud. Instead we should implement a well-known or SRV record to point to the nextcloud server. This, however, does not provide a solution for translating a username, so at the very least the nextcloud user needs to share the same email address as the one being queried.

[mikecardwell]

It doesn't require everybody to have a DNS entry. It only requires those who need this functionality, to have a DNS entry.

If you only wanted to translate the domain, and not the entire email address, you could just stick a domain only in the TXT record, and use a wildcard:

*._nextcloud.example.com IN TXT "example.com"

That way, a nexcloud server would simply follow the following logic when trying to locate firstname.surname@example.com:

1. dig txt firstname.surname@example.com
2. If the result of 1 is an email address like firstname@cloud.example.com, then we have the address
3. If the result of 1 is a domain like "cloud.example.com", then prefix with the local part and we have the address: firstname.lastname@cloud.example.com
4. If the result of 1 is an NXDOMAIN, or a NOERROR with no TXT RR, then there is no DNS alias, so use the originally entered address: firstname.lastname@example.com

Your well known SRV record is no easier to set up than the above, and is much less flexible

[mikecardwell]

It could also just be a CNAME. In my requested case, the alias would look like:

firstname.surname._nextcloud.cloud.example.com IN CNAME firstname._nextcloud.example.com

If you just wanted to map the domain:

*._nextcloud.cloud.example.com IN CNAME _nextcloud.example.com

[jhesketh]

I see your point, but I don't think it is the best approach. If I am understanding correctly, you are suggesting a way to translate an email address into a federated nextcloud username (eg: firstname.lastname@example.com -> firstname@cloud.example.com).

What I think would be neater is to query the nextcloud server: "Do you have a user with this $email_address?". This means that the client or DNS does not need to do any translation of an email address, but simply asks the nextcloud server if they know what to do with "firstname.lastname@example.com".

The question then becomes: which nextcloud server do I query about an email address? And the answer to that is through DNS (SRV, TXT, etc), .well-known, or a combination of the two.

This will avoid having to do a DNS entry for every user that requires some kind of translation outside of just the domain-name.

We could further improve the system by allowing multiple email addresses to be associated with a single nextcloud user. So long as the email address has a pointer to that nextcloud instance, the translation can occur.

What the nextcloud server returns to the requesting client is a different discussion. For example, if I entered an email address "firstname.lastname@example.com", would my client then show "firstname@cloud.example.com" after it has done the translation? I would prefer not to, so I think we need to extend the federation model to include the original reference as well as the federated username. The original reference is all that should be exposed to the user to avoid confusion, with perhaps the exception of showing a 'real name' if the permissions allow.

[mikecardwell]

I host email for multiple people on example.com - It sounds to me like what you're suggesting is that all of those people need to use the same Nextcloud server if they wish to use their example.com email addresses as their federated identity.

I would prefer to allow my users to use whatever Nextcloud server they want, not one ordained by me.

[jhesketh]

Right, okay, it sounds like our needs are different.

The situation that I was designing to was more for organisations and companies who have an email domain and are setting up a nextcloud to go with it. I imagine this is a more common scenario, especially among corporate users, but we should look to support both if possible.

The challenge with your scenario is that your users can "bring their own cloud" but still need you to enter the details into DNS (or whatever broker is designed). This doesn't scale very well so maybe we should look for something where an individual with an email address can say "this is my preferred nextcloud" regardless of who is operating their email.

This starts to look like a centralised service though (eg somewhere to query emails and get a nextcloud back) which is not what we want. We could have an email provider run such a service and then have DNS point to that service or to the authoritative nextcloud for that email.

I'm not sure what the best option here is. I do think the situation I've described is easier to cater for so maybe it's a good place to start, but I'm clearly biased. A broker would be a new and separate service that would need to be designed and would also require authenticating with the email server - it would be a lot of work. Maybe a middle ground is supporting DNS records to do both what you and I have described?

It's useful to note that when somebody shares something with an email address that user has the option to import the share into their logged in nextcloud, thus creating the link at that point. I'm not sure how something like the new social features would scale with this.

Anyway, these are just my opinions and shouldn't be taken as anything other than just my musing :-).

[meskio]

It will be great to be able to define the domain for the identifiers on the federation, so the website can be in a different (sub)domain than the one used for the federated identifiers (I don't think it needs to be connected to an email address).

To check out how other services solve this problem I'll list few here:

• XMPP does it by an SRV record:

_xmpp-server._tcp.example.com. 86400 IN SRV 5 0 5269 xmpp.example.com.

• matrix uses a json at https://example.com/.well-known/matrix/client:

{
  "m.homeserver": {
    "base_url": "https://matrix.example.com"
  },
  "m.identity_server": {
    "base_url": "https://identity.example.com"
  },
  "org.example.custom.property": {
    "app_url": "https://custom.app.example.org"
  }
}

• ActivityPub (like mastodon) uses webfinger to locate federated identifiers and webgfinger uses host-meta. An xml in https://example.com/.well-known/host-meta:

<?xml version="1.0" encoding="UTF-8"?>
<XRD xmlns="http://docs.oasis-open.org/ns/xri/xrd-1.0">
  <Link rel="lrdd" type="application/xrd+xml" template="https://social.example.com/.well-known/webfinger?resource={uri}"/>
</XRD>

I find easier for a web application to use a .well-known url as you have access to it from javascript if needed. The solution of matrix for me looks the easiest, in the case of nextcloud it could be just having something like https://example.com/.well-known/nextcloud/federation with a simple json.

[DanScharon]

Is there anything planned or on the roadmap? There are several great examples mentioned here that could be used as a template for an implementation by Nextcloud.