Random badNonce errors

[kelunik]

I'm getting badNonce errors sometimes (infrequently), rerunning the same code solves the problem. As anyone else experienced that problem? I don't have any concurrent requests in my client as far as I know.

[jmhodges]

Got a nonce that you saw that on by chance?

[kelunik]

I'll add logging for all nonces now, so I'll able to provide one next time.

[kelunik]

Maybe related: https://community.letsencrypt.org/t/getting-the-client-sent-an-unacceptable-anti-replay-nonce/9172?u=kelunik

[mhutter]

I get this every time... for log see https://gist.github.com/mhutter/1aa9f4285ec4036a8560

[voltagex]

I get this too, on a Debian box and a FreeBSD 9 box, plus the logs are empty.

[kevinburke]

Here's one. Let me know what other info you need

2016-01-25 03:45:09,296:INFO:requests.packages.urllib3.connectionpool:Starting new HTTPS connection (1): acme-v01.api.letsencrypt.org
2016-01-25 03:45:09,468:DEBUG:requests.packages.urllib3.connectionpool:"POST /acme/new-authz HTTP/1.1" 400 92
2016-01-25 03:45:09,470:DEBUG:root:Received <Response [400]>. Headers: {'Content-Length': '92', 'Server': 'nginx', 'Connection': 'close', 'Date': 'Mon, 25 Jan 2016 03:45:09 GMT', 'Content-Type': 'application/problem+json', 'Replay-Nonce': 'qWPKGNp7kKQxQvVdniRo1JGMywEG3p7qYVFavwiT_-0'}. Content: '{"type":"urn:acme:error:badNonce","detail":"JWS has invalid anti-replay nonce","status":400}'
2016-01-25 03:45:09,471:DEBUG:acme.client:Storing nonce: '\xa9c\xca\x18\xda{\x90\xa41B\xf5]\x9e$h\xd4\x91\x8c\xcb\x01\x06\xde\x9e\xeaaQZ\xbf\x08\x93\xff\xed'
2016-01-25 03:45:09,471:DEBUG:acme.client:Received response <Response [400]> (headers: {'Content-Length': '92', 'Server': 'nginx', 'Connection': 'close', 'Date': 'Mon, 25 Jan 2016 03:45:09 GMT', 'Content-Type': 'application/problem+json', 'Replay-Nonce': 'qWPKGNp7kKQxQvVdniRo1JGMywEG3p7qYVFavwiT_-0'}): '{"type":"urn:acme:error:badNonce","detail":"JWS has invalid anti-replay nonce","status":400}'
2016-01-25 03:45:09,474:DEBUG:letsencrypt.cli:Exiting abnormally:
Traceback (most recent call last):
  File "/home/kevin/.local/share/letsencrypt/bin/letsencrypt", line 11, in <module>
    sys.exit(main())
  File "/home/kevin/.local/share/letsencrypt/local/lib/python2.7/site-packages/letsencrypt/cli.py", line 1398, in main
    return args.func(args, config, plugins)

[NHellFire]

Here's one of mine. I had to retry 6 or so times for it to work.

2016-01-25 03:24:34,699:INFO:requests.packages.urllib3.connectionpool:Starting new HTTPS connection (1): acme-v01.api.letsencrypt.org 2016-01-25 03:24:34,910:DEBUG:requests.packages.urllib3.connectionpool:"POST /acme/new-authz HTTP/1.1" 400 92 2016-01-25 03:24:34,914:DEBUG:root:Received <Response [400]>. Headers: {'Content-Length': '92', 'Server': 'nginx', 'Connection': 'close', 'Date': 'Mon, 25 Jan 2016 03:24:34 GMT', 'Content-Type': 'application/problem+json', 'Replay-Nonce': 'QQJ2H07ZSJg00HSBLrYdJ_3YD9gWMHBkhfrK2oBDAiM'}. Content: '{"type":"urn:acme:error:badNonce","detail":"JWS has invalid anti-replay nonce","status":400}' 2016-01-25 03:24:34,914:DEBUG:acme.client:Storing nonce: "A\x02v\x1fN\xd9H\x984\xd0t\x81.\xb6\x1d'\xfd\xd8\x0f\xd8\x160pd\x85\xfa\xca\xda\x80C\x02#" 2016-01-25 03:24:34,915:DEBUG:acme.client:Received response <Response [400]> (headers: {'Content-Length': '92', 'Server': 'nginx', 'Connection': 'close', 'Date': 'Mon, 25 Jan 2016 03:24:34 GMT', 'Content-Type': 'application/problem+json', 'Replay-Nonce': 'QQJ2H07ZSJg00HSBLrYdJ_3YD9gWMHBkhfrK2oBDAiM'}): '{"type":"urn:acme:error:badNonce","detail":"JWS has invalid anti-replay nonce","status":400}' 2016-01-25 03:24:34,916:DEBUG:letsencrypt.cli:Exiting abnormally:

[marki555]

I tried with version 0.1.1 of LE client and it worked for 2 of 3 domains after few trues, but not for the last one. Now I upgraded to 0.2, but can't make it work (tried about 6 times). Here is the log:

2016-01-25 16:13:40,978:DEBUG:root:Requesting fresh nonce
2016-01-25 16:13:40,978:DEBUG:root:Sending HEAD request to https://acme-v01.api.letsencrypt.org/acme/new-authz. args: (), kwargs: {}
2016-01-25 16:13:40,980:INFO:requests.packages.urllib3.connectionpool:Starting new HTTPS connection (1): acme-v01.api.letsencrypt.org
2016-01-25 16:13:41,054:DEBUG:requests.packages.urllib3.connectionpool:"HEAD /acme/new-authz HTTP/1.1" 405 0
2016-01-25 16:13:41,058:DEBUG:root:Received <Response [405]>. Headers: {'Content-Length': '78', 'Server': 'nginx', 'Connection': 'keep-alive', 'Allow': 'POST', 'Date': 'Mon, 25 Jan 2016 16:13:41 GMT', 'Content-Type': 'application/problem+json', 'Replay-Nonce': 'ObhL3Wb0m4kFZJGZZQFhOxd6tqGzbdOcXxVgddDmf5w'}. Content: ''
2016-01-25 16:13:41,058:DEBUG:acme.client:Storing nonce: '9\xb8K\xddf\xf4\x9b\x89\x05d\x91\x99e\x01a;\x17z\xb6\xa1\xb3m\xd3\x9c_\x15`u\xd0\xe6\x7f\x9c'
2016-01-25 16:13:41,058:DEBUG:acme.jose.json_util:Omitted empty fields: combinations=None, challenges=None, expires=None, status=None
2016-01-25 16:13:41,058:DEBUG:acme.client:Serialized JSON: {"identifier": {"type": "dns", "value": "www.stuzkova.com"}, "resource": "new-authz"}
2016-01-25 16:13:41,061:DEBUG:acme.jose.json_util:Omitted empty fields: x5c=(), cty=None, x5tS256=None, x5u=None, crit=(), x5t=None, kid=None, jwk=None, alg=None, typ=None, jku=None
2016-01-25 16:13:41,073:DEBUG:acme.jose.json_util:Omitted empty fields: x5c=(), cty=None, typ=None, x5tS256=None, x5u=None, crit=(), nonce=None, kid=None, x5t=None, jku=None
2016-01-25 16:13:41,073:DEBUG:root:Sending POST request to https://acme-v01.api.letsencrypt.org/acme/new-authz. args: (), kwargs: {'data': '{"header": {"alg": "RS256", "jwk": {"e": "AQAB", "kty": "RSA", "n": "7sTYz-5lpVbNOPKwLA0h4pz5aHSAx3dMQns6nh4stbPuuKBcREEvClS-2pRMyG_a4YtXyA9R5y8OQ3knS5fmy376an7_p6HI_FWh5zUiN13-uud9QD32vCtWA5qTpXyyl-d-QUzjDFvkOdqXg-xusBSgn4DCHR3lscgdQbvA07oWRrzFyuw0QeMyxVPaqsT_fDzrPLYOCEuhBlj1u8jL21V3Xf0lwTDHxw5CF8yQDKsCBk4n2vIQWoTIz0fTar0XQ19VSdWoFhG0ODtM1My7DMqaIuubAIBfx6UfPFbpyTgMSrkzN8YTkrQIvSUu_i9kcGJUpo_FNO9k_RdWNKuuGw"}}, "protected": "eyJub25jZSI6ICJPYmhMM1diMG00a0ZaSkdaWlFGaE94ZDZ0cUd6YmRPY1h4VmdkZERtZjV3In0", "payload": "eyJpZGVudGlmaWVyIjogeyJ0eXBlIjogImRucyIsICJ2YWx1ZSI6ICJ3d3cuc3R1emtvdmEuY29tIn0sICJyZXNvdXJjZSI6ICJuZXctYXV0aHoifQ", "signature": "JoImr4PpNSTeCwxQSv2wEgbmRcZPJp-VlZLK_aUroUfjVq7S_gBRY6rtQEf4AxWQQnwH2PaHnWzRGNeqw3SskTXrr5RjMF5FHUpKSLHx9bvJR9M2p7LHuJjBUiabfcd-5Mr4oQ7w30oboNT9zAXFq5nrb5W-Cpn6--UCY8YbwEnHKmXyl07ukIU0jOsJjG8CelyNWp1B5B0-lEUqjq7nF1toc8DWqhIJKcl7bkBa5CNOoqRr1JBC-KeJsXe9yHVjZqQ1JiKJWRnrA9SVZrrP1B4bI2xQBZbgKoTJNTR-uPgOjNoOtNaKkV91wGMMDtq3hY4d_i8qILo1-pr70_o1aA"}'}
2016-01-25 16:13:41,074:INFO:requests.packages.urllib3.connectionpool:Starting new HTTPS connection (1): acme-v01.api.letsencrypt.org
2016-01-25 16:13:41,453:DEBUG:requests.packages.urllib3.connectionpool:"POST /acme/new-authz HTTP/1.1" 400 92
2016-01-25 16:13:41,456:DEBUG:root:Received <Response [400]>. Headers: {'Content-Length': '92', 'Server': 'nginx', 'Connection': 'close', 'Date': 'Mon, 25 Jan 2016 16:13:41 GMT', 'Content-Type': 'application/problem+json', 'Replay-Nonce': 'SEQHsm0ZMsCSIVjaJNvshs0kCMDzCKJe92-kI-HM0u8'}. Content: '{"type":"urn:acme:error:badNonce","detail":"JWS has invalid anti-replay nonce","status":400}'
2016-01-25 16:13:41,456:DEBUG:acme.client:Storing nonce: 'HD\x07\xb2m\x192\xc0\x92!X\xda$\xdb\xec\x86\xcd$\x08\xc0\xf3\x08\xa2^\xf7o\xa4#\xe1\xcc\xd2\xef'
2016-01-25 16:13:41,456:DEBUG:acme.client:Received response <Response [400]> (headers: {'Content-Length': '92', 'Server': 'nginx', 'Connection': 'close', 'Date': 'Mon, 25 Jan 2016 16:13:41 GMT', 'Content-Type': 'application/problem+json', 'Replay-Nonce': 'SEQHsm0ZMsCSIVjaJNvshs0kCMDzCKJe92-kI-HM0u8'}): '{"type":"urn:acme:error:badNonce","detail":"JWS has invalid anti-replay nonce","status":400}'

[wturrell]

I needed three attempts with the 0.2.0 client on Debian 8.2. (Ask if you need any more logs.)

[warden]

Same here on Debian 8.3. First 3 tries were unsuccesful, then it started working.

[jmhodges]

We just finished up reverting a CDN config change that was causing this problem. There was caching in some places where there should not be.

In fact, it finished around the time of the previous comment to this one. Enjoy!

[guruvan]

Quick note for those who may run across this issue
in janeczku/rancher-letsencrypt#3 it appears that attempting to use an email address already in use without providing credentials has caused this error. The error message in this case could be more helpful.

[kelunik]

Using an e-mail address already in use with another key isn't an issue and doesn't result in this error.

[guruvan]

@kelunik -I'm sure you are correct, however, my own real-world experience is that

• using rancher-letsencrypt as the client
  1. I had a previously used email address, and repeatedly received anti-replay nonce errors
  2. after many tests, several different domain names:
  3. I switched to using a NEW email address and stopped receiving those errors.
• I tested several of the same names, and each one I obtained certs
  AFAICT it's fully reproducible - I use guruvan@maza.club - fail, use a new address, success. Each time. I could try again in a few days & see if still reproduces

[kelunik]

Can you reproduce it with the same email and another client?

[guruvan]

I do not know - I can give that a try when I run another round later this week I'll check

• official letsencrypt client, with new and previously used addresses (I now have several used :)))
• rancher-letsencrypt with the same

I'lll report back here so we can narrow in on this.

[cmcaine]

I also found that I had to change email address to get letsencrypt to give me new certificates.

Haven't tested with another client - the only other client I use barely has a notion of users and certainly doesn't store user information anywhere locally, as far as I remember.

[catsnsnacks]

Using an automated tool set up by a hosting provider to install/generate certs, I also had to modify my email address to get it to issue a new certificate, otherwise I ran into the badNonce error.

Thankfully, with gmail addresses, emailaddress@gmail is the same as email.address@gmail

[jsha]

What hosting provider did you use?

[catsnsnacks]

Apis Networks

I browsed through this list and decided on Apis:
https://github.com/certbot/certbot/wiki/Web-Hosting-Supporting-LE

[ubershmekel]

I've gotten these errors and fixed them by changing the email address. Does that mean this issue should be opened @jmhodges ? Based on your cdn comment, it seems an issue might need to be opened on a different repo.

2017/02/19 01:46:29 h2_bundle.go:3223: http2: server: error reading preface from client 98.164.240.42:54611: timeout waiting for client preface
time="2017-02-19T02:13:17Z" level=error msg="map[www.tagsyo.com:acme: Error 400 - urn:acme:error:badNonce - JWS has invalid anti-replay nonce 91WN0oA5PjLkDXGV_xe8izfu0GhDAvnE0So4_DQindU]" 
time="2017-02-19T02:13:17Z" level=error msg="Error getting ACME certificates [www.tagsyo.com] : Cannot obtain certificates map[www.tagsyo.com:acme: Error 400 - urn:acme:error:badNonce - JWS has invalid anti-replay nonce 91WN0oA5PjLkDXGV_xe8izfu0GhDAvnE0So4_DQindU]+v"  

[cpu]

@ubershmekel What client are you using? Contact emails are not related to badNonce errors. Can you please open a new issue with the problem you are seeing and we can help troubleshoot? It is unrelated to this closed issue.

Note for others that find this issue: The original CDN problem at the root of this particular issue has been fixed. If you are experiencing a badNonce error you should open a new issue instead of replying here.