security/acme-client switching from staging to prod leads to account error

[proofy]

I tried with different accounts, even a new one, but with the same result.

What i noticed after some extra digging is that the error states the invalid account URL as "https://acme-staging-v02.api.letsencrypt.org/acme/acct/9468460\" which is clearly pointing at the LE staging environment although i changed my setting to use the production environment.
Is the script trying to get a real certificate from the staging environment??? I can't imagine, otherwise a lot of people would have the same problem.

I also noticed the following lines in the log:

_saved_account_key_hash='+D9eLQaohKgjpS+HZ4kp6KX3tdg15N+i+uCw4TJo3pE='
_saved_account_key_hash is not changed, skip register account.

It is always the same key hash, but i cannot find where it is coming from. I would expect that it changes when i use a new account, but it doesn't.

Steps to reproduce:

• Set up the acme plugin with an account, validation method and certificate and use the staging environment to get a test certificate which works fine.
• Then change in the settings tab the LE environment to 'Production Environment' and save and apply the new setting.
• Then go to the certificates tab and re-issue the same certificate. That is when the error occurs.

Kind regards,
Jack.

Originally posted by @jekare in #1473 (comment)

[luismompohanden]

I can confirm this issue with the same steps to reproduce.
OPNsense 19.7.4_1-amd64
A crash report came with this issue. I forwarded it.

[miodzicho]

Same

[jpawlowski]

Same.

This can be avoided by creating dedicated accounts for STAGING and PROD and to make sure to change those in each and every certificate when changing the global PROD/STAGING setting.

[miodzicho]

Not in my case, even changing account doesn't help :

`

"detail": "KeyID header contained an	invalid account URL: "https://acme-staging-v02.api.letsencrypt.org/acme/acct/11242486\"",
`

[jpawlowski]

You need to start from scratch by cleaning up the artifacts first, otherwise the account mixup will persist.

[billgertz]

Got the exact same error as cited in the bug report. However I used jpawlowski's suggestion.

Switching to a new LE account fixed the problem. But needed to run (re-) issue twice. It seems the first (re-) issue just registered the new LE account and stopped there.

The output below came from acme.sh with a --debug 2 flag. This is recent pull request (#1531 debug change is buried but it's there) not yet integrated into OPNsense. So, yes, I'm running a self-patched system. This level of detail may be different on your system (YMMV).

[Mon Oct  7 01:19:45 CEST 2019] ACME_DIRECTORY='https://acme-v02.api.letsencrypt.org/directory'
[Mon Oct  7 01:19:45 CEST 2019] Use length 4096
[Mon Oct  7 01:19:45 CEST 2019] Using RSA: 4096
[Mon Oct  7 01:19:46 CEST 2019] Create account key ok.
[Mon Oct  7 01:19:46 CEST 2019] ACME_DIRECTORY='https://acme-v02.api.letsencrypt.org/directory'
[Mon Oct  7 01:19:46 CEST 2019] Using config home:/var/etc/acme-client/home
[Mon Oct  7 01:19:46 CEST 2019] ACME_DIRECTORY='https://acme-v02.api.letsencrypt.org/directory'
[Mon Oct  7 01:19:46 CEST 2019] _init api for server: https://acme-v02.api.letsencrypt.org/directory
[Mon Oct  7 01:19:46 CEST 2019] GET
[Mon Oct  7 01:19:46 CEST 2019] url='https://acme-v02.api.letsencrypt.org/directory'
[Mon Oct  7 01:19:46 CEST 2019] timeout=
[Mon Oct  7 01:19:46 CEST 2019] _CURL='curl -L --silent --dump-header /var/etc/acme-client/home/http.header  --trace-ascii /tmp/tmp.uqWfxFQl  -g '
[Mon Oct  7 01:19:46 CEST 2019] ret='0'
[Mon Oct  7 01:19:46 CEST 2019] ACME_KEY_CHANGE='https://acme-v02.api.letsencrypt.org/acme/key-change'
[Mon Oct  7 01:19:46 CEST 2019] ACME_NEW_AUTHZ
[Mon Oct  7 01:19:46 CEST 2019] ACME_NEW_ORDER='https://acme-v02.api.letsencrypt.org/acme/new-order'
[Mon Oct  7 01:19:46 CEST 2019] ACME_NEW_ACCOUNT='https://acme-v02.api.letsencrypt.org/acme/new-acct'
[Mon Oct  7 01:19:46 CEST 2019] ACME_REVOKE_CERT='https://acme-v02.api.letsencrypt.org/acme/revoke-cert'
[Mon Oct  7 01:19:46 CEST 2019] ACME_AGREEMENT='https://letsencrypt.org/documents/LE-SA-v1.2-November-15-2017.pdf'
[Mon Oct  7 01:19:46 CEST 2019] ACME_NEW_NONCE='https://acme-v02.api.letsencrypt.org/acme/new-nonce'
[Mon Oct  7 01:19:46 CEST 2019] ACME_VERSION='2'
[Mon Oct  7 01:19:46 CEST 2019] RSA key
[Mon Oct  7 01:19:47 CEST 2019] Registering account
[Mon Oct  7 01:19:47 CEST 2019] url='https://acme-v02.api.letsencrypt.org/acme/new-acct'
[Mon Oct  7 01:19:47 CEST 2019] payload='{"contact": ["mailto: bill@blockfish.net"], "termsOfServiceAgreed": true}'
[Mon Oct  7 01:19:47 CEST 2019] HEAD
[Mon Oct  7 01:19:47 CEST 2019] _post_url='https://acme-v02.api.letsencrypt.org/acme/new-nonce'
[Mon Oct  7 01:19:47 CEST 2019] _CURL='curl -L --silent --dump-header /var/etc/acme-client/home/http.header  --trace-ascii /tmp/tmp.9YOKeRnv  -g '
[Mon Oct  7 01:19:47 CEST 2019] _ret='0'
[Mon Oct  7 01:19:47 CEST 2019] POST
[Mon Oct  7 01:19:47 CEST 2019] _post_url='https://acme-v02.api.letsencrypt.org/acme/new-acct'
[Mon Oct  7 01:19:47 CEST 2019] _CURL='curl -L --silent --dump-header /var/etc/acme-client/home/http.header  --trace-ascii /tmp/tmp.9YOKeRnv  -g '
[Mon Oct  7 01:19:48 CEST 2019] _ret='0'
[Mon Oct  7 01:19:48 CEST 2019] code='201'
[Mon Oct  7 01:19:48 CEST 2019] Registered
[Mon Oct  7 01:19:48 CEST 2019] _accUri='https://acme-v02.api.letsencrypt.org/acme/acct/68693193'
[Mon Oct  7 01:19:48 CEST 2019] Calc CA_KEY_HASH='ClklQy8f34s/PpiT5z69ca3B1nK5jviaS/H8cNmVYuA='
[Mon Oct  7 01:19:48 CEST 2019] ACCOUNT_THUMBPRINT='Re7J53urvbg2FKxtLgyoJGi-aRvqDMqZZElkMM-sHgo'

Second try using the same LE account as registered above. Did nothing more than hit “Forcefully (re-) issue the selected certificate?” button a second time. Cert issued. Success!

But perhaps there is a bug, that is, it should issue a cert the first time through?

[miodzicho]

I did as well, but unfortunatelly getting this :

[Mon Oct 7 10:29:31 CEST 2019] response='{ "type": "urn:ietf:params:acme:error:malformed", "detail": "Unable to update challenge :: authorization must be pending", "status": 400

[jpawlowski]

That’s just the regular limiter when you have tried too often and there are pending validations. Wait for them to time out, it should usually take 1-2 hours.

[miodzicho]

So. I cannot get this working at all...Created all fresh.

Last Acme Status : validation failed

Log:
[Tue Oct 8 09:07:53 CEST 2019] original='{
"type": "http-01",
"status": "pending",

[fraenki]

@miodzicho I think that's an unrelated problem. Please open a new issue and provide more details about your Let's Encrypt plugin and certificate configuration.

[fraenki]

which is clearly pointing at the LE staging environment although i changed my setting to use the production environment

I think this can be solved. Currently we'll create a new directory for every LE account:

plugins/security/acme-client/src/opnsense/scripts/OPNsense/AcmeClient/certhelper.php

Line 347 in a878826

$account_conf_dir = "/var/etc/acme-client/accounts/" . $acctObj->id;

It should be pretty easy to modify this to not only use the internal Account ID but also include the LE environment in the path. This should allow to use the "same" account for both the LE staging and production environments.

However, this will provide a challenge for all existing configurations, because we have to migrate all accounts to the new (staging or production) path. I'll try to figure this out.

[fraenki]

@jpawlowski I've created a fix in fraenki@5866358, would you do a quick review?

[ErikJStaab]

Patch 5866358 fixed the issue for me, I had restored my config.xml previously and was having trouble getting my existing certificate to renew even after rekeying etc. I was switching around to the staging environment etc. After deleting/recreating everything again I was still having issues. Applied patch, renewed, and it seems to be working now. I was getting the same invalid account URL message from the staging server URL.

[fraenki]

@ErikJStaab Thanks for testing!

I'm closing this issue, the fix will be available in the upcoming release 1.27 of the os-acme-client plugin.