500s for `tailscale cert`

[philkuz]

What is the issue?

Trying to run tailscale cert command that I've run many times before and today getting service failures

$ sudo tailscale cert <tailscale-domain>.ts.net
500 Internal Server Error: acme: order  status: invalid

Common across other members of my team. Status in https://status.tailscale.com/ is still green, which is not true.

Steps to reproduce

$ sudo tailscale cert <tailscale-domain>.ts.net
500 Internal Server Error: acme: order  status: invalid

Are there any recent changes that introduced the issue?

No response

OS

❯ uname -a
Linux auger 6.8.0-47-generic #47~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Wed Oct  2 16:16:55 UTC 2 x86_64 x86_64 x86_64 GNU/Linux

Tailscale version

❯ tailscale version
1.78.1
  tailscale commit: 8903926f7167f4c4a2e21a6e3c0dae855322f3d6
  other commit: c4163954ed6b6d5e1fa9d6ae9d88e096ed361d71
  go version: go1.23.3

Other software

No response

Bug report

No response

[bradfitz]

This is an ongoing issue with one of our DNS providers: tailscale/issue-status#54

LetsEncrypt is unable to verify the DNS challenges as a result.

[bradfitz]

We should probably add support to tailscale cert for surfacing these errors more nicely, though, with a human readable message that we can set from the control plane.

[lucaguarro]

I am still getting this error despite #14386 being closed

[p-rintz]

I have the same issue and I think it may have to do with there being multiple _acme-challenge TXT entries for my subdomain.

_acme-challenge.sub.domain.ts.net. 300 IN	TXT "pX2kR5tJ9nLq-D3vYw-zH1bF7oUeC8sT"
_acme-challenge.sub.domain.ts.net. 300 IN	TXT "qY8wZ1xS4jRk-L7pBv-mN5cOqUaH9dF"
_acme-challenge.sub.domain.ts.net. 300 IN	TXT "fR3tGnPkXoQwEs-VzLcJiYbDuHaMnWfOtKy5pC2r"
_acme-challenge.sub.domain.ts.net. 300 IN	TXT "zX1wAeSrTyUiOh-PpKlMqJnVcDfGhXyBaZtNw2rE3"
_acme-challenge.sub.domain.ts.net. 300 IN	TXT "fR3tGnPkXoQwEs-VzLcJiYbDuHaMnWfOtKy5pC2r"
_acme-challenge.sub.domain.ts.net. 300 IN	TXT "pC0vBnMsXaYdZw-JkQlTgUrEfHiOpNyVcSw7lX3a"
_acme-challenge.sub.domain.ts.net. 300 IN	TXT "hY8uJkLmNoPtRv-SzXwQeDfGaYbIcUtKoVp4nMs1"
_acme-challenge.sub.domain.ts.net. 300 IN	TXT "vW2qTyUiOpAsDf-GhJkLmNoPqRsTvUxYzBcX8eF1"
_acme-challenge.sub.domain.ts.net. 300 IN	TXT "zX2vUc6YqJtLbNp-FgHsWo9pKi5rDeAx-mY1vBt4cQ"
_acme-challenge.sub.domain.ts.net. 300 IN	TXT "cB1eXn2YqLjRtUo-RyKfZs8gDh9pWeAx-vT0oQx5lH"

Tailscale seemingly sometimes fails to clean-up old entries.

Edit:
2 days later, the TXT entries are still there. This subdomain is basically blocked for acme cert requests.