Using wildcard in additional_hostnames prompts for admin privileges

[pyrello]

Is there an existing issue for this?

• I have searched the existing issues

Output of ddev debug test

ddev debug test diagnostic information: https://gist.github.com/pyrello/81b43cba5124f073bd6bcc730c1d55a1

Opening this bug per @rfay in conversation in Discord.

Expected Behavior

Not requiring administrative privileges to run ddev start

Actual Behavior

Running ddev start requires admin privileges with our setup.

There are two bugs here:

1. We shouldn't try to resolve an invalid hostname (*.x.ddev.site)
2. We shouldn't try to add a wildcard to /etc/hosts, as it won't work there anyway.

Probably we should just notify people that if their wildcard is not resolvable (*.*.ddev.site should normally be resolvable) then they need to make it resolvable one way or another.

Steps To Reproduce

ddev config --additional-hostnames="*.junk"
ddev start

Anything else?

In our project, we use DDEV for an application that runs more than 700 sites. We have it setup to provide all the sites locally at an address like *.uiowa.ddev.site. After upgrading to a new computer that does not allow me to have continuous access to administrative privileges, I am running into an issue where I see the following output when I run ddev start:

ddev start
Starting uiowa... 
The hostname *.uiowa.ddev.site is not currently resolvable, trying to add it to the hosts file 
ddev needs to run with administrative privileges.
You may be required to enter your password for sudo or allow escalation. ddev is about to issue the command:
  sudo --preserve-env=HOME /opt/homebrew/bin/ddev hostname *.uiowa.ddev.site 127.0.0.1
 
Password:
Not populating custom commands or hostadditions because running with root privileges
Failed to add hosts entry *.uiowa.ddev.site: hostname is not a valid dns name: *.uiowa.ddev.site

Workaround

Adding a config.local.yaml file with the following settings allowed me to start ddev without it asking for my password:

override_config: true
additional_hostnames:
  - sppa.uiowa

This allows the site https://sppa.uiowa.ddev.site to be available locally. The config.local.yaml file does not get checked in.

[rfay]

Please explain your workaround for people who might run up against this.

[pyrello]

Please explain your workaround for people who might run up against this.

Updated IS with workaround.

[pyrello]

I was running into some seemingly unrelated trouble and as I was going through the troubleshooting guide, I noticed the DDEV Starts but Browser Can’t Access URL section. That seems to indicate that the reason why the problem in this issue might have been happening: My computer was unable to connect to the internet.

While DDEV can create a web server and a Docker network infrastructure for a project, it doesn’t have control of your computer’s name resolution, so its backup technique to make a hostname resolvable by the browser is to add an entry to the hosts file (/etc/hosts on Linux and macOS, C:\Windows\system32\drivers\etc\hosts on traditional Windows).

[rfay]

@gitressa most likely DDEV.exe on the Windows side is prompting you for escalated privs here and you need to give them. But please do ddev.exe --version and see if it's recent. If it's not, use choco upgrade -y ddev in an admin PS window. See if that helps. I don't think your problem has anything to do with this issue.

[rfay]

Thanks, I'm going to mark everything else as off-topic. What seemed to be happening is sudo didn't work???

[mariano-dagostino]

Hi there. I'm having a similar situation (using linux here DDEV version v1.22.7) not sure if my scenario is different. I have a custom DNS server that maps any subdomain of *.websites.ddev.site to an IP: 192.168.1.40.

Then DDEV is trying to run:

sudo --preserve-env=HOME /usr/bin/ddev hostname *.websites.ddev.site 127.0.0.1

Which I think it won't have any effect in the hostname (it doesn't even add a new line to /etc/hosts anyways)

So even with the mentioned workaround of adding this additional_hostnames to the config.local.yaml I think the bug is not to try to attempt to an invalid hostname (*.websites.ddev.site in my case) in the first place.

[rfay]

@mariano-dagostino I think it is the same thing. I imagine if you use explicit names in additional_hostnames or additional_fqdns you have no trouble right?

If you are able to make your DNS server respond to *.websites (I don't think that's possible) it would probably resolve your particular problem.

[mariano-dagostino]

If you are able to make your DNS server respond to *.websites (I don't think that's possible) it would probably resolve your particular problem.

@rfay Thanks. Yes I'm able to make my DNS respond to *.websites I'm using blocky as custom dns provider: 0xERR0R/blocky#74 (comment)

My problem is somehow ddev is not able to figure this out and attempts to alter the hostname and request password for this.

[rfay]

For normal hostnames DDEV tries to look up the hostname, and if it succeeds (if it succeeds in this case looking up the actual record *.websites.ddev.site then it doesn't try to add to /etc/hosts. I haven't studied this, but a PR is certainly welcome.

[mariano-dagostino]

@rfay Thanks. Is IsHostnameInHostsFile the right function to modify or there is something else to look?

[rfay]

No, it's AddHostsEntriesIfNeeded:

ddev/pkg/ddevapp/hostname_mgt.go

Lines 74 to 112 in faff807

	func (app *DdevApp) AddHostsEntriesIfNeeded() error {
	var err error
	dockerIP, err := dockerutil.GetDockerIP()
	if err != nil {
	return fmt.Errorf("could not get Docker IP: %v", err)
	}
	if os.Getenv("DDEV_NONINTERACTIVE") == "true" {
	util.Warning("Not trying to add hostnames because DDEV_NONINTERACTIVE=true")
	return nil
	}
	for _, name := range app.GetHostnames() {
	// If we're able to resolve the hostname via DNS or otherwise we
	// don't have to worry about this. This will allow resolution
	// of *.ddev.site for example
	if app.UseDNSWhenPossible && globalconfig.IsInternetActive() {
	// If they have provided "*.<name>" then look up the suffix
	checkName := strings.TrimPrefix(name, "*.")
	hostIPs, err := net.LookupHost(checkName)
	// If we had successful lookup and dockerIP matches
	// with adding to hosts file.
	if err == nil && len(hostIPs) > 0 && hostIPs[0] == dockerIP {
	continue
	}
	}
	// We likely won't hit the hosts.Has() as true because
	// we already did a lookup. But check anyway.
	exists, err := IsHostnameInHostsFile(name)
	if exists {
	continue
	}
	if err != nil {
	util.Warning("Unable to open hosts file: %v", err)
	continue
	}

The reason you're not being successful with your approach is obvious:

			// If they have provided "*.<name>" then look up the suffix
			checkName := strings.TrimPrefix(name, "*.")
			hostIPs, err := net.LookupHost(checkName)

A PR to sort this out would be great! It's a pretty obscure corner case for DDEV usage so even though it's tagged for a release right now it's going to be hard for this one to really land on our plate.

Lots of folks in the community have made great contributions, and it's not that hard. https://ddev.readthedocs.io/en/latest/developers/building-contributing/

[stasadev]

I'm using NetworkManager + dnsmasq on Linux, all the necessary config for this:

$ cat /etc/NetworkManager/conf.d/dns.conf
[main]
dns=dnsmasq

$ cat /etc/NetworkManager/dnsmasq.d/ddev.site.conf
address=/*.ddev.site/127.0.0.1

I can resolve any subdomain, and ddev start also works for me without the host change request:

$ ping -c1 any.custom.domain.in.ddev.site
PING any.custom.domain.in.ddev.site (127.0.0.1) 56(84) bytes of data.
64 bytes from localhost (127.0.0.1): icmp_seq=1 ttl=64 time=0.033 ms

I might try blocky this weekend to see what it is (I'm interested in adblocking and can test the DNS resolver).

[rfay]

Remember that a proper test here requires using an IP address that is not 127.0.0.1 and is using a wildcard in additional_hostnames.

[stasadev]

It works for me with blocky.

My config /etc/blocky/blocky.yml:

upstream:
  default:
    - 46.182.19.48
    - 80.241.218.68
    - tcp-tls:fdns1.dismail.de:853
    - https://dns.digitale-gesellschaft.ch/dns-query
blocking:
  blackLists:
    ads:
      - https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts
  clientGroupsBlock:
    default:
      - ads
ports:
  dns: 53
  http: 4000
customDNS:
  mapping:
    ddev.site: 127.0.0.1
    test: 127.0.0.1

I started the blocky service and configured NetworkManager to use it (192.168.1.55 is my machine IP).

$ sudo systemctl enable --now blocky.service

$ cat /etc/NetworkManager/conf.d/dns-servers.conf
[global-dns-domain-*]
servers=192.168.1.55

$ sudo systemctl restart NetworkManager.service

$ cat /etc/resolv.conf
# Generated by NetworkManager
nameserver 192.168.1.55

And finally test it with DDEV, it works for me:

$ ping -c1 custom.domain.in.ddev.site
PING custom.domain.in.ddev.site (127.0.0.1) 56(84) bytes of data.
64 bytes from localhost (127.0.0.1): icmp_seq=1 ttl=64 time=0.046 ms

$ ddev config --additional-hostnames="*.custom.domain.in" --project-tld="ddev.site"
$ ddev start
$ xdg-open https://my.custom.domain.in.ddev.site/

Repeat the same with test:

$ ping -c1 custom.domain.in.test
PING custom.domain.in.test (127.0.0.1) 56(84) bytes of data.
64 bytes from localhost (127.0.0.1): icmp_seq=1 ttl=64 time=0.046 ms

$ ddev config --additional-hostnames="*.custom.domain.in" --project-tld="test"
$ ddev start
$ xdg-open https://my.custom.domain.in.test/

[rfay]

Just a note though, you don't need custom DNS to do any subdomain with ddev.site, the normal record published on the internet does fine:

$ nslookup one.two.three.four.ddev.site
Server:		1.1.1.1
Address:	1.1.1.1#53

Non-authoritative answer:
Name:	one.two.three.four.ddev.site
Address: 127.0.0.1

[stasadev]

Yes, it works when you have internet all the time, but when you are offline, it is the only reliable way to make DDEV work without requesting hosts to be edited.

[rfay]

When actually offline, though, you'd normally want to let it create entries in /etc/hosts; that's what I do on a plane or whatever. But I guess you're talking about the situation where you have a local network but no internet reachability.

[rfay]

I think we probably need a whole topic in the docs explaining what name resolution is, why it matters, how it's usually done, etc. And an FAQ pointing to it.

[rfay]

I would appreciate it if you folks interested in this issue could:

• test the PR at fix: handle wildcard hostnames when name resolution not available, fixes #4881 #6025 (comment)
• Review this blog:
• Add article on name resolution in DDEV ddev.com#176
  Read it at https://20240327-name-resolution.ddev-com-front-end.pages.dev/blog/ddev-name-resolution-wildcards/