Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Domain Connect for self-hosted services #64

Closed
anderspitman opened this issue Dec 16, 2021 · 12 comments
Closed

Domain Connect for self-hosted services #64

anderspitman opened this issue Dec 16, 2021 · 12 comments

Comments

@anderspitman
Copy link
Contributor

Hi @arnoldblinn, I just finished reading your article and the spec. I was very excited to discover Domain Connect this morning. It is in spirit almost exactly what I've been looking for for over a year. I've even spent some time recently implementing what is essentially the synchronous flow. I have a few questions.

My primary goal is to make data ownership and self-hosting accessible to more people. I don't think the average person should need to understand things like TLS certs and DNS records in order to securely self-host a service on a domain they control. LetsEncrypt mostly solves the TLS cert issue. I think something like Domain Connect is the solution to DNS complexity. Here's a concrete use case.

I have an open source project, boringproxy, which is essentially a combination of a reverse proxy with SSH tunneling (think ngrok or CloudFlare Tunnel. I maintain a list of similar services here) and auto TLS built-in. Once running, it presents the user with a simple web UI that lets them map domain names to ports running on specific machines, even if those machines are behind NATs etc, and tunnels requests. This works quite well, but the main missing piece is DNS. Currently the user has to set DNS records manually. Typically this would involve setting a wildcard record *.example.com pointing to the public IP of their boringproxy server. When they start the server they choose a subdomain such as admin.example.com which is used for the UI, and when they create new tunnels they name them things like tunnel1.example.com, etc.

What I want to be able to do is integrate boringproxy with DNS providers to automate this process. In my mind the ideal flow would be:

  1. User starts boringproxy server
  2. The server detects its public IP and asks the user if they want to integrate with a DNS provider
  3. If yes, a list of supported DNS providers is shown
  4. The user selects their provider
  5. A link is printed which takes the user to a consent page which lets them select a domain/subdomain/wildcard domain to use
  6. If the user consents, DNS records are set
  7. The user is redirected to the public IP address with the selected domain in the URL
  8. The boringproxy server uses the selected domain to perform a LetsEncrypt flow and redirects the user to the HTTPS admin page.
  9. The user is able to create tunnels. If a wildcard was selected in the initial flow, they can use that. Or they can perform more DNS flows to add additional domains/subdomains.

With this is mind and after reading the Domain Connect spec, I have the following questions/concerns:

  1. Requiring template publishing seems overly burdensome for this use case. Would every single user of boringproxy have to integrate with DNS providers? Or is there a way I as the project maintainer can register a template usable by anyone running a boringproxy server? In my implementation I simply allow anonymous requests, with the requested records URL-encoded in the request. It's then up to the DNS provider to warn the user of the implications of the suggested changes, and to provide them with the "phishing warning" etc.

  2. It would be nice if the DNS provider allowed the user to select a domain/subdomain on the consent page. So the service provider would simply present a list of supported DNS providers, and the user would select the one they use, and it could then be passed back to the service in the URL on redirect. This way the user doesn't have to type in the domain/subdomain. Is this possible with Domain Connect?

  3. Specifically for the functionality of boringproxy where users create tunnels on demand, is it possible to use a single template to facilitate changing an arbitrary number of hosts? I feel like this has to be possible, but it was a little unclear after reading this section.

Thank you for answering any of these questions that you can, and sorry it's so long!
@arnoldblinn
Copy link
Collaborator

arnoldblinn commented Dec 16, 2021 via email

@anderspitman
Copy link
Contributor Author

anderspitman commented Dec 16, 2021 •
edited
Loading

@arnoldblinn, thank you for the detailed answers! I think my concerns are reduced to 2 questions:

  1. Sounds like I can create a single template for boringproxy, and anyone who hosts their own boringproxy server can use the synchronous flow (assuming async would require OAuth credentials which I can't share)?

  2. Why is there no provision for anonymous synchronous requests, ie without template required or even some "universal" templates for simple actions such as setting a single A Record? It seems like tying everything to templates with specific provider IDs could actually make phishing worse, because when people see the name and/or logo of the provider they assume it's valid. I understand the phishing warnings and that signing requests solves this but in that case why not support fully anonymous requests with phishing warnings? You can't trust anything in the request so it seems like you shouldn't be showing them the logo or provider ID anyway, at which point why is a template necessary?

@arnoldblinn
Copy link
Collaborator

arnoldblinn commented Dec 16, 2021 via email

@pawel-kow
Copy link
Member

Worth considering, although not perfect, is the approach we took with DynDNS client.
So it's in fact async flow, with confidential client and known predefined secret.
Public client with PKCE would have been better, but this was easier to do at a time as no upgrades needed to the auth servers.

More details:
https://github.com/Domain-Connect/DomainConnectDDNS-Windows

There is also a Python client
https://github.com/Domain-Connect/DomainConnectDDNS-Python

In fact if only A/AAAA records are needed one could piggyback on the same template... it's just OAuth in the end of the day.

@arnoldblinn
Copy link
Collaborator

arnoldblinn commented Dec 16, 2021 via email

@anderspitman
Copy link
Contributor Author

@arnoldblinn thanks again. I guess I just don't see the difference between showing a phishing warning with a pre-defined template, and showing a phishing warning with a template you pass in the query itself (anonymously). You can't trust the request in either case. And I would argue that showing the service provider name and/or logo in this case is actually worse for security.

Requiring that all templates be predefined and onboarded seems like it would slow adoption and make it much more difficult for small players and open source projects to get involved. I would try implementing Domain Connect today but knowing that I have to get at least one DNS provider to onboard my template makes me wonder if it would take weeks of back-and-forth with me trying to debug my template. And there's no guarantee any of them would choose to onboard it at all.

@pawel-kow thanks for the suggestion. I'm not sure I see the advantage. The synchronous flow should be fine for my project. It looks like the phishing warning is required either way since the secret is public. I feel like your suggestion that I piggyback on the DynDNS template further illustrates my point. If it doesn't matter that I impersonate someone else's service, why are predefined templates required in the first place?

I'm not trying to be contrary. There's a good chance I'm just missing something here, and I want to understand.

@anderspitman
Copy link
Contributor Author

anderspitman commented Dec 16, 2021 •
edited
Loading

@arnoldblinn I appreciate your participation so far but recognize that you've mostly moved on from this project. I would value your thoughts moving forward, but please don't feel any obligation.

As a followup, I just bought a domain from Godaddy and built a simple Domain Connect URL using the example template1.

It works as intended (kudos on making an elegant API), but this is what I see:

DeepinScreenshot_select-area_20211216161409

It really seems to me that the use of a predefined template is mostly detrimental as implemented here. Even though warnPhishing is set true in the template, I don't see any indication that there's anything for the user to be concerned about here. What's even more concerning, the inclusion of "Stateless Hosting Primary" and "Example Domain Connect Service" show that the user could easily be mislead into believing they are using a completely different service than indicated.

To be clear, I'm not saying that service provider registration is a bad idea. But I'm advocating two things:

  1. Service provider registration should be optional. Anonymous requests with inline templates should be possible, but clearly marked as untrustworthy.
  2. Any flow that displays information from a template to a user must be signed.

Thoughts?

Also @arnoldblinn, it appears that host=* does not work, for Godaddy at least.

@arnoldblinn
Copy link
Collaborator

arnoldblinn commented Dec 16, 2021 via email

@pawel-kow
Copy link
Member

pawel-kow commented Dec 17, 2021 •
edited
Loading

@anderspitman public template repositiry and the review process in there is happening to save the work for the community.
One important factor for the review is in fact to minimize attack surface for phishing, especially by minimizing possibilities of unintended use of a template or applying additional measures (require signing, warnPhishing, minimize variable parts etc.).
We are not yet that far to have a fully automated rule-set that would allow for such verification on the fly and automated.

The template is in the end of the day an API contract between DNS provider and Service provider (in some cases backed by a read legal agreement). Each time it's a distinct decision of a DNS provider whether to onboard the template and whether to trust the community review process. I can tell most of DNS providers have also own review process before onboarding and most are also onboarding templates out-of-band, for internal usage or for confidential partners.

What we never wanted Domain Connect to be was a generic CRUD-like API for any DNS record, which would eventually keep the doors wide open for any kind of misuse. Domain Connect keeps the scope of user consent as narrow as possible to minimize the risk. And I think it's working very fine for the use-cases it was designed for.

The use-case of standalone tools is a bit difficult as many of the security measures don't work, but as dyndns case shows it's possible and can be very successful.
So I think you can still give it a try with a template for your service. If sync flow works out for you then you should have even easier way to get it rolling.

@anderspitman
Copy link
Contributor Author

anderspitman commented Dec 17, 2021 •
edited
Loading

Thanks again both of you.

I will consider implementing Domain Connect for my projects and trying to get DNS providers to accept my templates.

But ultimately I don't think Domain Connect in its current form is really a good fit for self-hosted software. Requiring every open source project to onboard a template with every DNS provider they want to support is simply too much friction. I really think there needs to be support for anonymous requests. Obviously not all DNS providers would have to support them, but I think it should be an option.

Who should I talk to about trying to get this added to Domain Connect? I need this functionality for my projects, but I don't think it makes sense for me to make a completely separate spec that would work basically the same way on a technical level, only with more open/loosely coupled integration policies.

@pawel-kow
Copy link
Member

pawel-kow commented Dec 20, 2021 •
edited
Loading

@anderspitman as mentioned, there are DNS providers that accept templates from the public repo without any review (let' name them group 1). There are ones who do an internal review and accept (group 2) and finally ones which only accept with additional legal agreement (group 3).
I would assume that groups 2&3 won't accept such "template with request" anyways, same as they don't accept templates out of the repository as-is.

No matter if self-hosted or not, the process of making the template public increases the trust and security for the whole community. For the group 1 it actually is functionally equal to passing a template with the request and even opens the door to the group 2 of providers who would accept your template after review. For group 3 you also get a chance, maybe with a little bit more effort and communication, but eventually as examples of DynDNS, ID4me or Plesk (also self-hosted!) show, this is doable and in many cases very successful.

The bottom line - with template approach you can likely reach all 3 groups. With full dynamic - you likely reach only the group 1.

BUT... this is an open community and healthy community lives from active participants, exchange of ideas and open discussions. You are welcome to join the Slack channel and see whether you gain interest from your proposal. I would argue such extension to be a separate specification (similar like "dynamic registration" specifications to OAuth2 or OIDC are separate to the core) but then if few parties are interested to spec it and eventually implement, let it be so.

@anderspitman
Copy link
Contributor Author

anderspitman commented Dec 28, 2021 •
edited
Loading

Sorry for the delay. Thanks for the explanation. I still think things are too tightly coupled and the friction is too high for small projects to integrate, but I understand the design decisions based on the problems you're trying to solve. Thanks @arnoldblinn and @pawel-kow!

Hope you have a happy new year!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@anderspitman @arnoldblinn @pawel-kow