New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

proposal: Vanity URLs via DNS #21284

Closed
benbjohnson opened this Issue Aug 2, 2017 · 14 comments

Comments

Projects
None yet
6 participants
@benbjohnson

benbjohnson commented Aug 2, 2017

Problem

Using the hosted URL of a package (e.g. github.com/benbjohnson/foo) for the import URL causes portability issues since all dependent code requires changes if the hosting is changed. This occurs in a few notable situations:

  1. A hosting provider shuts down (e.g. Google Code).
  2. A project moves to a new GitHub organization or owner.
  3. A project moves to a new hosting provider (e.g. migrating from GitHub to GitLab).

This currently can be solved using vanity URLs but this approach requires that the project owner host her or his own vanity URL server which requires more effort and it is likely less reliable than GitHub.

Solution

Instead of using a vanity server, I propose that the Go toolchain supports vanity URLs via DNS TXT or SRV records. The toolchain can lookup the record at import time and fetch the project from its host (e.g. GitHub). Lookups can be cached based on the TTL in the DNS record.

This would provide the following benefits:

  1. DNS is highly reliable.
  2. Easy to change hosting provider.
  3. Could support backup mirrors by using SRV priority.

In a discussion with @rakyll on Twitter, I provided the example that rakyll.pizza could point to her github.com/rakyll account. Another option would be to point to a specific project such as using hey.ninja to point at github.com/rakyll/hey. (Ridiculous TLD names are optional, but fun)

I don't have specifics on the implementation right now but I am proposing this initially to see if it is of interest to the Go community.

@gopherbot gopherbot added this to the Proposal milestone Aug 2, 2017

@gopherbot gopherbot added the Proposal label Aug 2, 2017

@rakyll

This comment has been minimized.

Show comment
Hide comment
@rakyll

rakyll Aug 2, 2017

Member

We can easily differentiate whether the record is an org or a repo and give the user to flexibility to use either of their URLs.

Member

rakyll commented Aug 2, 2017

We can easily differentiate whether the record is an org or a repo and give the user to flexibility to use either of their URLs.

@titanous

This comment has been minimized.

Show comment
Hide comment
@titanous

titanous Aug 2, 2017

Member

The major downside to using DNS for this is that there is no authenticity guarantee, whereas the current requirement to serve a meta tag over HTTPS ensures that an attacker cannot maliciously redirect a package download just by spoofing a DNS response. For this reason, I'd strongly suggest sticking to serving vanity import URLs over HTTPS.

Member

titanous commented Aug 2, 2017

The major downside to using DNS for this is that there is no authenticity guarantee, whereas the current requirement to serve a meta tag over HTTPS ensures that an attacker cannot maliciously redirect a package download just by spoofing a DNS response. For this reason, I'd strongly suggest sticking to serving vanity import URLs over HTTPS.

@stp-ip

This comment has been minimized.

Show comment
Hide comment
@stp-ip

stp-ip Aug 2, 2017

There is a small project on my list to make it easy to do redirects with a minimal go server (or use a hosted one) and a standardized DNS TXT record to store the actual redirect. We might just provide one of the redirect servers for convenience.

stp-ip commented Aug 2, 2017

There is a small project on my list to make it easy to do redirects with a minimal go server (or use a hosted one) and a standardized DNS TXT record to store the actual redirect. We might just provide one of the redirect servers for convenience.

@rakyll

This comment has been minimized.

Show comment
Hide comment
@rakyll

rakyll Aug 2, 2017

Member

I have a minimal server at https://github.com/googlecloudplatform/govanityurls, and I can move it to x/tools -- a more suitable place now without any dependencies to App Engine -- and we can collaborate. In my personal experience, downtime of this server has never been an issue (at least not as much as downtime of GitHub), but I host the server on a very stable provider with very stable networking.

As far as I understand from the linked Twitter thread, the remote idea of owning/maintaining servers scare people and hence we were considering options that doesn't require a server. Though we should never sacrifice security.

/cc @zombiezen

Member

rakyll commented Aug 2, 2017

I have a minimal server at https://github.com/googlecloudplatform/govanityurls, and I can move it to x/tools -- a more suitable place now without any dependencies to App Engine -- and we can collaborate. In my personal experience, downtime of this server has never been an issue (at least not as much as downtime of GitHub), but I host the server on a very stable provider with very stable networking.

As far as I understand from the linked Twitter thread, the remote idea of owning/maintaining servers scare people and hence we were considering options that doesn't require a server. Though we should never sacrifice security.

/cc @zombiezen

@benbjohnson

This comment has been minimized.

Show comment
Hide comment
@benbjohnson

benbjohnson Aug 2, 2017

@titanous That's a good point. It's certainly not worth giving up security. Is that something that could be fixed using DNSSEC?

benbjohnson commented Aug 2, 2017

@titanous That's a good point. It's certainly not worth giving up security. Is that something that could be fixed using DNSSEC?

@titanous

This comment has been minimized.

Show comment
Hide comment
@titanous

titanous Aug 2, 2017

Member

@benbjohnson No, DNSSEC is not widely deployed and there's no evidence that it will ever be deployed to the ubiquitous level that would be necessary for this. See these articles for details:

Member

titanous commented Aug 2, 2017

@benbjohnson No, DNSSEC is not widely deployed and there's no evidence that it will ever be deployed to the ubiquitous level that would be necessary for this. See these articles for details:

@benbjohnson

This comment has been minimized.

Show comment
Hide comment
@benbjohnson

benbjohnson Aug 2, 2017

Yeah, looks like sockpuppet.org really doesn't like DNSSEC. :)

I'll close this proposal for now. If anyone has improvements around it then I can reopen.

benbjohnson commented Aug 2, 2017

Yeah, looks like sockpuppet.org really doesn't like DNSSEC. :)

I'll close this proposal for now. If anyone has improvements around it then I can reopen.

@benbjohnson benbjohnson closed this Aug 2, 2017

@daenney

This comment has been minimized.

Show comment
Hide comment
@daenney

daenney Aug 4, 2017

Why not allow for both? If your domain does support DNSSEC, and a lot of g/cTLDs do, it's a question of deploying it yourself or having a DNS provider that does. Just b/c the owner of sockpuppet.org doesn't like DNSSEC and happens to have a few blog posts about it doesn't make this shouldn't be pursued. About 89% of TLDs are signed and cTLDs current sit at about 47%, adoption wise. Aside from that, the major deployed DNS servers and client libraries support DNSSEC and so do most recursive resolvers.

.com, .net, .org and .io are all signed (so is .us and aside from Albania and Belarus so are all the European ccTLDs, quite a few in Asia, South America, Australia and New Zealand) which are rather prevalent TLDs when it comes to where packages are hosted on. DNSSEC has become significantly easier to deploy since it was conceived and more and more registrars offer support for it in their control panel, it's just a box you need to tick. Though it does require some testing before enabling it, it's technically not complicated to do so. If you also host your own authoritative servers for your domain then yes, it'll be a bit more work.

If the response does pass DNSSEC validation it should be entirely feasible to follow along with this proposal. If it doesn't, fall back to the current way of doing it. That would allow to gradually upgrade to this proposal without breaking anything current.

daenney commented Aug 4, 2017

Why not allow for both? If your domain does support DNSSEC, and a lot of g/cTLDs do, it's a question of deploying it yourself or having a DNS provider that does. Just b/c the owner of sockpuppet.org doesn't like DNSSEC and happens to have a few blog posts about it doesn't make this shouldn't be pursued. About 89% of TLDs are signed and cTLDs current sit at about 47%, adoption wise. Aside from that, the major deployed DNS servers and client libraries support DNSSEC and so do most recursive resolvers.

.com, .net, .org and .io are all signed (so is .us and aside from Albania and Belarus so are all the European ccTLDs, quite a few in Asia, South America, Australia and New Zealand) which are rather prevalent TLDs when it comes to where packages are hosted on. DNSSEC has become significantly easier to deploy since it was conceived and more and more registrars offer support for it in their control panel, it's just a box you need to tick. Though it does require some testing before enabling it, it's technically not complicated to do so. If you also host your own authoritative servers for your domain then yes, it'll be a bit more work.

If the response does pass DNSSEC validation it should be entirely feasible to follow along with this proposal. If it doesn't, fall back to the current way of doing it. That would allow to gradually upgrade to this proposal without breaking anything current.

@titanous

This comment has been minimized.

Show comment
Hide comment
@titanous

titanous Aug 4, 2017

Member

@daenney The problem is that DNSSEC as currently deployed doesn't do anything for clients. The vast majority of machines use stub resolvers that don't do DNSSEC validation, so this whole thing is moot.

Member

titanous commented Aug 4, 2017

@daenney The problem is that DNSSEC as currently deployed doesn't do anything for clients. The vast majority of machines use stub resolvers that don't do DNSSEC validation, so this whole thing is moot.

@daenney

This comment has been minimized.

Show comment
Hide comment
@daenney

daenney Aug 4, 2017

Oh right, the validation part. Most seem to fallback when it fails 😞. Afaik though you can see if the request did pass DNSSEC validation, and systemd, macOS and Windows 7+ do support it in their client code, which covers a lot of people.

daenney commented Aug 4, 2017

Oh right, the validation part. Most seem to fallback when it fails 😞. Afaik though you can see if the request did pass DNSSEC validation, and systemd, macOS and Windows 7+ do support it in their client code, which covers a lot of people.

@titanous

This comment has been minimized.

Show comment
Hide comment
@titanous

titanous Aug 4, 2017

Member

@daenney The Authenticated Data flag that you are talking about is just a bit in a cleartext DNS packet from the resolver, it can be tampered with/spoofed by any attacker, so it doesn't help security at all.

Member

titanous commented Aug 4, 2017

@daenney The Authenticated Data flag that you are talking about is just a bit in a cleartext DNS packet from the resolver, it can be tampered with/spoofed by any attacker, so it doesn't help security at all.

@daenney

This comment has been minimized.

Show comment
Hide comment
@daenney

daenney Aug 4, 2017

But how does that create a problem? If AD is set your system's stub resolver will validate the response and afaik it's not feasible for an attacker to tamper with the signature. If an attacker removes AD then you simply go for the fallback behaviour. I'm not aware of any (stub) resolver that just blindly trusts the AD flag and doesn't verify the signatures.

daenney commented Aug 4, 2017

But how does that create a problem? If AD is set your system's stub resolver will validate the response and afaik it's not feasible for an attacker to tamper with the signature. If an attacker removes AD then you simply go for the fallback behaviour. I'm not aware of any (stub) resolver that just blindly trusts the AD flag and doesn't verify the signatures.

@titanous

This comment has been minimized.

Show comment
Hide comment
@titanous

titanous Aug 4, 2017

Member

@daenney There are no signatures for the stub resolver to verify, so it does just blindly trust the AD flag in responses from the resolver. This Microsoft documentation has a good explanation.

Member

titanous commented Aug 4, 2017

@daenney There are no signatures for the stub resolver to verify, so it does just blindly trust the AD flag in responses from the resolver. This Microsoft documentation has a good explanation.

@daenney

This comment has been minimized.

Show comment
Hide comment
@daenney

daenney Aug 4, 2017

@titanous Damn. I've always understood it the other way around :|. That sucks. Never mind indeed then.

daenney commented Aug 4, 2017

@titanous Damn. I've always understood it the other way around :|. That sucks. Never mind indeed then.

@golang golang locked and limited conversation to collaborators Aug 4, 2018

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.