Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Multi-SAN certificates get renewed even though nothing changed; hitting LE rate limits #4005

Closed
UlfbertWusel opened this issue Mar 30, 2022 · 1 comment
Closed

Multi-SAN certificates get renewed even though nothing changed; hitting LE rate limits #4005

UlfbertWusel opened this issue Mar 30, 2022 · 1 comment

Comments

@UlfbertWusel
Copy link

Hello,

we're using your great tool to automate certificate requests and renewals for quite a lot of domains and so far it has been great.
For a few weeks now we've been experiencing issues, where previously working (in terms of renewals) multi-SAN certs have been hitting the LetsEncrypt Rate Limits. So far we've not been able to find out why this happens. Also, the cert renewal succeeds every time the rate limits reset. Of course no changes are made to the cert configuration to trigger a renewal.
So far we noticed this in three different multi-SAN certificates, other work as expected.

It seems that acme.sh sees changes in the domains even though none have occurred.

Command we run with relevant excerpt of failed request from logfile (real domain name changed for obvious reasons):

/home/gitlab-runner/builds/letsencrypt-test/acme/acme.sh --issue --server letsencrypt --cert-home /home/gitlab-runner/letsencrypt/certs/subdir --dns dns_aws --home /home/gitlab-runner/.acme.sh -d b.r.com -d b.r.one -d b.em.one -d *.b.r.one -d *.b.ec.pro -d *.b.ec.cloud -d *.b.em.one -d ca.r.com -d ca.r.one -d ca.em.one -d *.r.cl.services -d *.ca.r.one -d *.ca.ec.pro -d *.ca.ec.cloud -d *.ca.em.one

[Wed Mar 30 01:01:28 CEST 2022] _main_domain='b.r.com'
[Wed Mar 30 01:01:28 CEST 2022] _alt_domains='b.r.one,b.em.one,*.b.r.one,*.b.r.com,*.b.ec.pro,*.b.ec.cloud,*.b.em.one,ca.r.com,ca.r.one,ca.em.one,*.r.cl.services,*.ca.r.one,*.ca.r.com,*.ca.ec.pro,*.ca.ec.cloud,*.ca.em.one'
[Wed Mar 30 01:01:28 CEST 2022] Using config home:/home/gitlab-runner/.acme.sh
[Wed Mar 30 01:01:28 CEST 2022] ACME_DIRECTORY='https://acme-v02.api.letsencrypt.org/directory'
[Wed Mar 30 01:01:28 CEST 2022] DOMAIN_PATH='/home/gitlab-runner/letsencrypt/certs/subdir/b.r.com'
[Wed Mar 30 01:01:28 CEST 2022] Using ACME_DIRECTORY: https://acme-v02.api.letsencrypt.org/directory
[Wed Mar 30 01:01:28 CEST 2022] _init api for server: https://acme-v02.api.letsencrypt.org/directory
[Wed Mar 30 01:01:28 CEST 2022] GET
[Wed Mar 30 01:01:28 CEST 2022] url='https://acme-v02.api.letsencrypt.org/directory'
[Wed Mar 30 01:01:28 CEST 2022] timeout=
[Wed Mar 30 01:01:28 CEST 2022] _CURL='curl -L --silent --dump-header /home/gitlab-runner/.acme.sh/http.header  -g '
[Wed Mar 30 01:01:29 CEST 2022] ret='0'
[Wed Mar 30 01:01:29 CEST 2022] ACME_KEY_CHANGE='https://acme-v02.api.letsencrypt.org/acme/key-change'
[Wed Mar 30 01:01:29 CEST 2022] ACME_NEW_AUTHZ
[Wed Mar 30 01:01:29 CEST 2022] ACME_NEW_ORDER='https://acme-v02.api.letsencrypt.org/acme/new-order'
[Wed Mar 30 01:01:29 CEST 2022] ACME_NEW_ACCOUNT='https://acme-v02.api.letsencrypt.org/acme/new-acct'
[Wed Mar 30 01:01:29 CEST 2022] ACME_REVOKE_CERT='https://acme-v02.api.letsencrypt.org/acme/revoke-cert'
[Wed Mar 30 01:01:29 CEST 2022] ACME_AGREEMENT='https://letsencrypt.org/documents/LE-SA-v1.2-November-15-2017.pdf'
[Wed Mar 30 01:01:29 CEST 2022] ACME_NEW_NONCE='https://acme-v02.api.letsencrypt.org/acme/new-nonce'
[Wed Mar 30 01:01:29 CEST 2022] ACME_VERSION='2'
[Wed Mar 30 01:01:29 CEST 2022] Le_NextRenewTime='1653649363'
[Wed Mar 30 01:01:29 CEST 2022] _saved_domain='b.r.com'
[Wed Mar 30 01:01:29 CEST 2022] _saved_alt='b.r.one,b.em.one,*.b.r.one,*.b.r.com,*.b.ec.pro,*.b.ec.cloud,*.b.em.one,ca.r.com,ca.r.one,ca.em.one,*.r.cl.services,*.ca.r.one,*.ca.r.com,*.ca.ec.pro,*.ca.ec.cloud'
[Wed Mar 30 01:01:29 CEST 2022] Domains have changed.
[Wed Mar 30 01:01:29 CEST 2022] _on_before_issue
[Wed Mar 30 01:01:29 CEST 2022] _chk_main_domain='b.r.com'
[Wed Mar 30 01:01:29 CEST 2022] _chk_alt_domains='b.r.one,b.em.one,*.b.r.one,*.b.r.com,*.b.ec.pro,*.b.ec.cloud,*.b.em.one,ca.r.com,ca.r.one,ca.em.one,*.r.cl.services,*.ca.r.one,*.ca.r.com,*.ca.ec.pro,*.ca.ec.cloud,*.ca.em.one'
[Wed Mar 30 01:01:29 CEST 2022] Le_LocalAddress
[Wed Mar 30 01:01:29 CEST 2022] d='b.r.com'
[Wed Mar 30 01:01:29 CEST 2022] Check for domain='b.r.com'
[Wed Mar 30 01:01:29 CEST 2022] _currentRoot='dns_aws'
[Wed Mar 30 01:01:29 CEST 2022] d='b.r.one'
[Wed Mar 30 01:01:29 CEST 2022] Check for domain='b.r.one'
[Wed Mar 30 01:01:29 CEST 2022] _currentRoot='dns_aws'
[Wed Mar 30 01:01:29 CEST 2022] d='b.em.one'
[Wed Mar 30 01:01:29 CEST 2022] Check for domain='b.em.one'
[Wed Mar 30 01:01:29 CEST 2022] _currentRoot='dns_aws'
[Wed Mar 30 01:01:29 CEST 2022] d='*.b.r.one'
[Wed Mar 30 01:01:29 CEST 2022] Check for domain='*.b.r.one'
[Wed Mar 30 01:01:29 CEST 2022] _currentRoot='dns_aws'
[Wed Mar 30 01:01:29 CEST 2022] d='*.b.r.com'
[Wed Mar 30 01:01:29 CEST 2022] Check for domain='*.b.r.com'
[Wed Mar 30 01:01:29 CEST 2022] _currentRoot='dns_aws'
[Wed Mar 30 01:01:29 CEST 2022] d='*.b.ec.pro'
[Wed Mar 30 01:01:29 CEST 2022] Check for domain='*.b.ec.pro'
[Wed Mar 30 01:01:29 CEST 2022] _currentRoot='dns_aws'
[Wed Mar 30 01:01:29 CEST 2022] d='*.b.ec.cloud'
[Wed Mar 30 01:01:29 CEST 2022] Check for domain='*.b.ec.cloud'
[Wed Mar 30 01:01:29 CEST 2022] _currentRoot='dns_aws'
[Wed Mar 30 01:01:29 CEST 2022] d='*.b.em.one'
[Wed Mar 30 01:01:29 CEST 2022] Check for domain='*.b.em.one'
[Wed Mar 30 01:01:29 CEST 2022] _currentRoot='dns_aws'
[Wed Mar 30 01:01:29 CEST 2022] d='ca.r.com'
[Wed Mar 30 01:01:29 CEST 2022] Check for domain='ca.r.com'
[Wed Mar 30 01:01:29 CEST 2022] _currentRoot='dns_aws'
[Wed Mar 30 01:01:29 CEST 2022] d='ca.r.one'
[Wed Mar 30 01:01:29 CEST 2022] Check for domain='ca.r.one'
[Wed Mar 30 01:01:29 CEST 2022] _currentRoot='dns_aws'
[Wed Mar 30 01:01:29 CEST 2022] d='ca.em.one'
[Wed Mar 30 01:01:29 CEST 2022] Check for domain='ca.em.one'
[Wed Mar 30 01:01:29 CEST 2022] _currentRoot='dns_aws'
[Wed Mar 30 01:01:29 CEST 2022] d='*.r.cl.services'
[Wed Mar 30 01:01:29 CEST 2022] Check for domain='*.r.cl.services'
[Wed Mar 30 01:01:29 CEST 2022] _currentRoot='dns_aws'
[Wed Mar 30 01:01:29 CEST 2022] d='*.ca.r.one'
[Wed Mar 30 01:01:29 CEST 2022] Check for domain='*.ca.r.one'
[Wed Mar 30 01:01:29 CEST 2022] _currentRoot='dns_aws'
[Wed Mar 30 01:01:29 CEST 2022] d='*.ca.r.com'
[Wed Mar 30 01:01:29 CEST 2022] Check for domain='*.ca.r.com'
[Wed Mar 30 01:01:29 CEST 2022] _currentRoot='dns_aws'
[Wed Mar 30 01:01:29 CEST 2022] d='*.ca.ec.pro'
[Wed Mar 30 01:01:29 CEST 2022] Check for domain='*.ca.ec.pro'
[Wed Mar 30 01:01:29 CEST 2022] _currentRoot='dns_aws'
[Wed Mar 30 01:01:29 CEST 2022] d='*.ca.ec.cloud'
[Wed Mar 30 01:01:29 CEST 2022] Check for domain='*.ca.ec.cloud'
[Wed Mar 30 01:01:29 CEST 2022] _currentRoot='dns_aws'
[Wed Mar 30 01:01:29 CEST 2022] d='*.ca.em.one'
[Wed Mar 30 01:01:29 CEST 2022] Check for domain='*.ca.em.one'
[Wed Mar 30 01:01:29 CEST 2022] _currentRoot='dns_aws'
[Wed Mar 30 01:01:29 CEST 2022] d
[Wed Mar 30 01:01:29 CEST 2022] _saved_account_key_hash is not changed, skip register account.
[Wed Mar 30 01:01:29 CEST 2022] Read key length:
[Wed Mar 30 01:01:29 CEST 2022] _createcsr
[Wed Mar 30 01:01:29 CEST 2022] Multi domain='DNS:b.r.com,DNS:b.r.one,DNS:b.em.one,DNS:*.b.r.one,DNS:*.b.r.com,DNS:*.b.ec.pro,DNS:*.b.ec.cloud,DNS:*.b.em.one,DNS:ca.r.com,DNS:ca.r.one,DNS:ca.em.one,DNS:*.r.cl.services,DNS:*.ca.r.one,DNS:*.ca.r.com,DNS:*.ca.ec.pro,DNS:*.ca.ec.cloud,DNS:*.ca.em.one'
[Wed Mar 30 01:01:29 CEST 2022] Getting domain auth token for each domain
[Wed Mar 30 01:01:29 CEST 2022] d='b.r.one'
[Wed Mar 30 01:01:29 CEST 2022] d='b.em.one'
[Wed Mar 30 01:01:29 CEST 2022] d='*.b.r.one'
[Wed Mar 30 01:01:29 CEST 2022] d='*.b.r.com'
[Wed Mar 30 01:01:29 CEST 2022] d='*.b.ec.pro'
[Wed Mar 30 01:01:29 CEST 2022] d='*.b.ec.cloud'
[Wed Mar 30 01:01:29 CEST 2022] d='*.b.em.one'
[Wed Mar 30 01:01:29 CEST 2022] d='ca.r.com'
[Wed Mar 30 01:01:29 CEST 2022] d='ca.r.one'
[Wed Mar 30 01:01:29 CEST 2022] d='ca.em.one'
[Wed Mar 30 01:01:29 CEST 2022] d='*.r.cl.services'
[Wed Mar 30 01:01:29 CEST 2022] d='*.ca.r.one'
[Wed Mar 30 01:01:29 CEST 2022] d='*.ca.r.com'
[Wed Mar 30 01:01:29 CEST 2022] d='*.ca.ec.pro'
[Wed Mar 30 01:01:29 CEST 2022] d='*.ca.ec.cloud'
[Wed Mar 30 01:01:29 CEST 2022] d='*.ca.em.one'
[Wed Mar 30 01:01:29 CEST 2022] d
[Wed Mar 30 01:01:29 CEST 2022] url='https://acme-v02.api.letsencrypt.org/acme/new-order'
[Wed Mar 30 01:01:29 CEST 2022] payload='{"identifiers": [{"type":"dns","value":"b.r.com"},{"type":"dns","value":"b.r.one"},{"type":"dns","value":"b.em.one"},{"type":"dns","value":"*.b.r.one"},{"type":"dns","value":"*.b.r.com"},{"type":"dns","value":"*.b.ec.pro"},{"type":"dns","value":"*.b.ec.cloud"},{"type":"dns","value":"*.b.em.one"},{"type":"dns","value":"ca.r.com"},{"type":"dns","value":"ca.r.one"},{"type":"dns","value":"ca.em.one"},{"type":"dns","value":"*.r.cl.services"},{"type":"dns","value":"*.ca.r.one"},{"type":"dns","value":"*.ca.r.com"},{"type":"dns","value":"*.ca.ec.pro"},{"type":"dns","value":"*.ca.ec.cloud"},{"type":"dns","value":"*.ca.em.one"}]}'
[Wed Mar 30 01:01:29 CEST 2022] RSA key
[Wed Mar 30 01:01:29 CEST 2022] HEAD
[Wed Mar 30 01:01:29 CEST 2022] _post_url='https://acme-v02.api.letsencrypt.org/acme/new-nonce'
[Wed Mar 30 01:01:29 CEST 2022] _CURL='curl -L --silent --dump-header /home/gitlab-runner/.acme.sh/http.header  -g  -I  '
[Wed Mar 30 01:01:30 CEST 2022] _ret='0'
[Wed Mar 30 01:01:30 CEST 2022] POST
[Wed Mar 30 01:01:30 CEST 2022] _post_url='https://acme-v02.api.letsencrypt.org/acme/new-order'
[Wed Mar 30 01:01:30 CEST 2022] _CURL='curl -L --silent --dump-header /home/gitlab-runner/.acme.sh/http.header  -g '
[Wed Mar 30 01:01:31 CEST 2022] _ret='0'
[Wed Mar 30 01:01:31 CEST 2022] code='429'
[Wed Mar 30 01:01:31 CEST 2022] Le_LinkOrder
[Wed Mar 30 01:01:31 CEST 2022] Le_OrderFinalize
[Wed Mar 30 01:01:31 CEST 2022] Create new order error. Le_OrderFinalize not found. {
  "type": "urn:ietf:params:acme:error:rateLimited",
  "detail": "Error creating new order :: too many certificates (5) already issued for this exact set of domains in the last 168 hours: *.b.ec.cloud,*.b.ec.pro,*.b.em.one,*.b.r.com,*.b.r.one,*.ca.ec.cloud,*.ca.ec.pro,*.ca.em.one,*.ca.r.com,*.ca.r.one,*.r.cl.services,b.em.one,b.r.com,b.r.one,ca.em.one,ca.r.com,ca.r.one: see https://letsencrypt.org/docs/rate-limits/",
  "status": 429
}
[Wed Mar 30 01:01:31 CEST 2022] pid
[Wed Mar 30 01:01:31 CEST 2022] No need to restore nginx, skip.
[Wed Mar 30 01:01:31 CEST 2022] _clearupdns
[Wed Mar 30 01:01:31 CEST 2022] dns_entries
[Wed Mar 30 01:01:31 CEST 2022] skip dns.
[Wed Mar 30 01:01:31 CEST 2022] _on_issue_err
[Wed Mar 30 01:01:31 CEST 2022] Please check log file for more details: /home/gitlab-runner/.acme.sh/acme.sh.log```

If you have any idea how to fix this, please advise.
In case you need any more info, let me know.

Many thanks in advance.
@Neilpang
Copy link
Member

If you changed the order of the domains in the command line, acme.sh will treat as "domain changed", so it tries to request a new cert again.
I'm not sure why the order of the domains was changed, but I just fixed it.

please try again with the latest dev branch:

acme.sh --upgrade -b dev

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@Neilpang @UlfbertWusel