New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Random badNonce errors #1217

Closed
kelunik opened this Issue Dec 3, 2015 · 22 comments

Comments

Projects
None yet
@kelunik
Contributor

kelunik commented Dec 3, 2015

I'm getting badNonce errors sometimes (infrequently), rerunning the same code solves the problem. As anyone else experienced that problem? I don't have any concurrent requests in my client as far as I know.

@jmhodges

This comment has been minimized.

Contributor

jmhodges commented Dec 3, 2015

Got a nonce that you saw that on by chance?

@kelunik

This comment has been minimized.

Contributor

kelunik commented Dec 3, 2015

I'll add logging for all nonces now, so I'll able to provide one next time.

@kelunik

This comment has been minimized.

@mhutter

This comment has been minimized.

mhutter commented Jan 21, 2016

I get this every time... for log see https://gist.github.com/mhutter/1aa9f4285ec4036a8560

@voltagex

This comment has been minimized.

voltagex commented Jan 21, 2016

I get this too, on a Debian box and a FreeBSD 9 box, plus the logs are empty.

patch0 added a commit to BytemarkHosting/symbiosis that referenced this issue Jan 21, 2016

jmhodges added a commit that referenced this issue Jan 21, 2016

log nonce to and from client
Also, log when a nonce service error occurs.

Updates #1217

jmhodges added a commit that referenced this issue Jan 21, 2016

log nonce to and from client
Also, log when a nonce service error occurs.

Updates #1217
@kevinburke

This comment has been minimized.

Contributor

kevinburke commented Jan 25, 2016

Here's one. Let me know what other info you need

2016-01-25 03:45:09,296:INFO:requests.packages.urllib3.connectionpool:Starting new HTTPS connection (1): acme-v01.api.letsencrypt.org
2016-01-25 03:45:09,468:DEBUG:requests.packages.urllib3.connectionpool:"POST /acme/new-authz HTTP/1.1" 400 92
2016-01-25 03:45:09,470:DEBUG:root:Received <Response [400]>. Headers: {'Content-Length': '92', 'Server': 'nginx', 'Connection': 'close', 'Date': 'Mon, 25 Jan 2016 03:45:09 GMT', 'Content-Type': 'application/problem+json', 'Replay-Nonce': 'qWPKGNp7kKQxQvVdniRo1JGMywEG3p7qYVFavwiT_-0'}. Content: '{"type":"urn:acme:error:badNonce","detail":"JWS has invalid anti-replay nonce","status":400}'
2016-01-25 03:45:09,471:DEBUG:acme.client:Storing nonce: '\xa9c\xca\x18\xda{\x90\xa41B\xf5]\x9e$h\xd4\x91\x8c\xcb\x01\x06\xde\x9e\xeaaQZ\xbf\x08\x93\xff\xed'
2016-01-25 03:45:09,471:DEBUG:acme.client:Received response <Response [400]> (headers: {'Content-Length': '92', 'Server': 'nginx', 'Connection': 'close', 'Date': 'Mon, 25 Jan 2016 03:45:09 GMT', 'Content-Type': 'application/problem+json', 'Replay-Nonce': 'qWPKGNp7kKQxQvVdniRo1JGMywEG3p7qYVFavwiT_-0'}): '{"type":"urn:acme:error:badNonce","detail":"JWS has invalid anti-replay nonce","status":400}'
2016-01-25 03:45:09,474:DEBUG:letsencrypt.cli:Exiting abnormally:
Traceback (most recent call last):
  File "/home/kevin/.local/share/letsencrypt/bin/letsencrypt", line 11, in <module>
    sys.exit(main())
  File "/home/kevin/.local/share/letsencrypt/local/lib/python2.7/site-packages/letsencrypt/cli.py", line 1398, in main
    return args.func(args, config, plugins)
@NHellFire

This comment has been minimized.

NHellFire commented Jan 25, 2016

Here's one of mine. I had to retry 6 or so times for it to work.

2016-01-25 03:24:34,699:INFO:requests.packages.urllib3.connectionpool:Starting new HTTPS connection (1): acme-v01.api.letsencrypt.org 2016-01-25 03:24:34,910:DEBUG:requests.packages.urllib3.connectionpool:"POST /acme/new-authz HTTP/1.1" 400 92 2016-01-25 03:24:34,914:DEBUG:root:Received <Response [400]>. Headers: {'Content-Length': '92', 'Server': 'nginx', 'Connection': 'close', 'Date': 'Mon, 25 Jan 2016 03:24:34 GMT', 'Content-Type': 'application/problem+json', 'Replay-Nonce': 'QQJ2H07ZSJg00HSBLrYdJ_3YD9gWMHBkhfrK2oBDAiM'}. Content: '{"type":"urn:acme:error:badNonce","detail":"JWS has invalid anti-replay nonce","status":400}' 2016-01-25 03:24:34,914:DEBUG:acme.client:Storing nonce: "A\x02v\x1fN\xd9H\x984\xd0t\x81.\xb6\x1d'\xfd\xd8\x0f\xd8\x160pd\x85\xfa\xca\xda\x80C\x02#" 2016-01-25 03:24:34,915:DEBUG:acme.client:Received response <Response [400]> (headers: {'Content-Length': '92', 'Server': 'nginx', 'Connection': 'close', 'Date': 'Mon, 25 Jan 2016 03:24:34 GMT', 'Content-Type': 'application/problem+json', 'Replay-Nonce': 'QQJ2H07ZSJg00HSBLrYdJ_3YD9gWMHBkhfrK2oBDAiM'}): '{"type":"urn:acme:error:badNonce","detail":"JWS has invalid anti-replay nonce","status":400}' 2016-01-25 03:24:34,916:DEBUG:letsencrypt.cli:Exiting abnormally:

@marki555

This comment has been minimized.

marki555 commented Jan 25, 2016

I tried with version 0.1.1 of LE client and it worked for 2 of 3 domains after few trues, but not for the last one. Now I upgraded to 0.2, but can't make it work (tried about 6 times). Here is the log:

2016-01-25 16:13:40,978:DEBUG:root:Requesting fresh nonce
2016-01-25 16:13:40,978:DEBUG:root:Sending HEAD request to https://acme-v01.api.letsencrypt.org/acme/new-authz. args: (), kwargs: {}
2016-01-25 16:13:40,980:INFO:requests.packages.urllib3.connectionpool:Starting new HTTPS connection (1): acme-v01.api.letsencrypt.org
2016-01-25 16:13:41,054:DEBUG:requests.packages.urllib3.connectionpool:"HEAD /acme/new-authz HTTP/1.1" 405 0
2016-01-25 16:13:41,058:DEBUG:root:Received <Response [405]>. Headers: {'Content-Length': '78', 'Server': 'nginx', 'Connection': 'keep-alive', 'Allow': 'POST', 'Date': 'Mon, 25 Jan 2016 16:13:41 GMT', 'Content-Type': 'application/problem+json', 'Replay-Nonce': 'ObhL3Wb0m4kFZJGZZQFhOxd6tqGzbdOcXxVgddDmf5w'}. Content: ''
2016-01-25 16:13:41,058:DEBUG:acme.client:Storing nonce: '9\xb8K\xddf\xf4\x9b\x89\x05d\x91\x99e\x01a;\x17z\xb6\xa1\xb3m\xd3\x9c_\x15`u\xd0\xe6\x7f\x9c'
2016-01-25 16:13:41,058:DEBUG:acme.jose.json_util:Omitted empty fields: combinations=None, challenges=None, expires=None, status=None
2016-01-25 16:13:41,058:DEBUG:acme.client:Serialized JSON: {"identifier": {"type": "dns", "value": "www.stuzkova.com"}, "resource": "new-authz"}
2016-01-25 16:13:41,061:DEBUG:acme.jose.json_util:Omitted empty fields: x5c=(), cty=None, x5tS256=None, x5u=None, crit=(), x5t=None, kid=None, jwk=None, alg=None, typ=None, jku=None
2016-01-25 16:13:41,073:DEBUG:acme.jose.json_util:Omitted empty fields: x5c=(), cty=None, typ=None, x5tS256=None, x5u=None, crit=(), nonce=None, kid=None, x5t=None, jku=None
2016-01-25 16:13:41,073:DEBUG:root:Sending POST request to https://acme-v01.api.letsencrypt.org/acme/new-authz. args: (), kwargs: {'data': '{"header": {"alg": "RS256", "jwk": {"e": "AQAB", "kty": "RSA", "n": "7sTYz-5lpVbNOPKwLA0h4pz5aHSAx3dMQns6nh4stbPuuKBcREEvClS-2pRMyG_a4YtXyA9R5y8OQ3knS5fmy376an7_p6HI_FWh5zUiN13-uud9QD32vCtWA5qTpXyyl-d-QUzjDFvkOdqXg-xusBSgn4DCHR3lscgdQbvA07oWRrzFyuw0QeMyxVPaqsT_fDzrPLYOCEuhBlj1u8jL21V3Xf0lwTDHxw5CF8yQDKsCBk4n2vIQWoTIz0fTar0XQ19VSdWoFhG0ODtM1My7DMqaIuubAIBfx6UfPFbpyTgMSrkzN8YTkrQIvSUu_i9kcGJUpo_FNO9k_RdWNKuuGw"}}, "protected": "eyJub25jZSI6ICJPYmhMM1diMG00a0ZaSkdaWlFGaE94ZDZ0cUd6YmRPY1h4VmdkZERtZjV3In0", "payload": "eyJpZGVudGlmaWVyIjogeyJ0eXBlIjogImRucyIsICJ2YWx1ZSI6ICJ3d3cuc3R1emtvdmEuY29tIn0sICJyZXNvdXJjZSI6ICJuZXctYXV0aHoifQ", "signature": "JoImr4PpNSTeCwxQSv2wEgbmRcZPJp-VlZLK_aUroUfjVq7S_gBRY6rtQEf4AxWQQnwH2PaHnWzRGNeqw3SskTXrr5RjMF5FHUpKSLHx9bvJR9M2p7LHuJjBUiabfcd-5Mr4oQ7w30oboNT9zAXFq5nrb5W-Cpn6--UCY8YbwEnHKmXyl07ukIU0jOsJjG8CelyNWp1B5B0-lEUqjq7nF1toc8DWqhIJKcl7bkBa5CNOoqRr1JBC-KeJsXe9yHVjZqQ1JiKJWRnrA9SVZrrP1B4bI2xQBZbgKoTJNTR-uPgOjNoOtNaKkV91wGMMDtq3hY4d_i8qILo1-pr70_o1aA"}'}
2016-01-25 16:13:41,074:INFO:requests.packages.urllib3.connectionpool:Starting new HTTPS connection (1): acme-v01.api.letsencrypt.org
2016-01-25 16:13:41,453:DEBUG:requests.packages.urllib3.connectionpool:"POST /acme/new-authz HTTP/1.1" 400 92
2016-01-25 16:13:41,456:DEBUG:root:Received <Response [400]>. Headers: {'Content-Length': '92', 'Server': 'nginx', 'Connection': 'close', 'Date': 'Mon, 25 Jan 2016 16:13:41 GMT', 'Content-Type': 'application/problem+json', 'Replay-Nonce': 'SEQHsm0ZMsCSIVjaJNvshs0kCMDzCKJe92-kI-HM0u8'}. Content: '{"type":"urn:acme:error:badNonce","detail":"JWS has invalid anti-replay nonce","status":400}'
2016-01-25 16:13:41,456:DEBUG:acme.client:Storing nonce: 'HD\x07\xb2m\x192\xc0\x92!X\xda$\xdb\xec\x86\xcd$\x08\xc0\xf3\x08\xa2^\xf7o\xa4#\xe1\xcc\xd2\xef'
2016-01-25 16:13:41,456:DEBUG:acme.client:Received response <Response [400]> (headers: {'Content-Length': '92', 'Server': 'nginx', 'Connection': 'close', 'Date': 'Mon, 25 Jan 2016 16:13:41 GMT', 'Content-Type': 'application/problem+json', 'Replay-Nonce': 'SEQHsm0ZMsCSIVjaJNvshs0kCMDzCKJe92-kI-HM0u8'}): '{"type":"urn:acme:error:badNonce","detail":"JWS has invalid anti-replay nonce","status":400}'
@wturrell

This comment has been minimized.

wturrell commented Jan 26, 2016

I needed three attempts with the 0.2.0 client on Debian 8.2. (Ask if you need any more logs.)

@warden

This comment has been minimized.

warden commented Jan 28, 2016

Same here on Debian 8.3. First 3 tries were unsuccesful, then it started working.

@jmhodges

This comment has been minimized.

Contributor

jmhodges commented Jan 28, 2016

We just finished up reverting a CDN config change that was causing this problem. There was caching in some places where there should not be.

In fact, it finished around the time of the previous comment to this one. Enjoy!

@guruvan

This comment has been minimized.

guruvan commented Mar 26, 2016

Quick note for those who may run across this issue
in janeczku/rancher-letsencrypt#3 it appears that attempting to use an email address already in use without providing credentials has caused this error. The error message in this case could be more helpful.

@kelunik

This comment has been minimized.

Contributor

kelunik commented Mar 26, 2016

Using an e-mail address already in use with another key isn't an issue and doesn't result in this error.

@guruvan

This comment has been minimized.

guruvan commented Mar 29, 2016

@kelunik -I'm sure you are correct, however, my own real-world experience is that

  • using rancher-letsencrypt as the client
    1. I had a previously used email address, and repeatedly received anti-replay nonce errors
    2. after many tests, several different domain names:
    3. I switched to using a NEW email address and stopped receiving those errors.
  • I tested several of the same names, and each one I obtained certs
    AFAICT it's fully reproducible - I use guruvan@maza.club - fail, use a new address, success. Each time. I could try again in a few days & see if still reproduces
@kelunik

This comment has been minimized.

Contributor

kelunik commented Mar 29, 2016

Can you reproduce it with the same email and another client?

@guruvan

This comment has been minimized.

guruvan commented Mar 29, 2016

I do not know - I can give that a try when I run another round later this week I'll check

  • official letsencrypt client, with new and previously used addresses (I now have several used :)))
  • rancher-letsencrypt with the same

I'lll report back here so we can narrow in on this.

@cmcaine

This comment has been minimized.

cmcaine commented Apr 22, 2016

I also found that I had to change email address to get letsencrypt to give me new certificates.

Haven't tested with another client - the only other client I use barely has a notion of users and certainly doesn't store user information anywhere locally, as far as I remember.

@lotus42

This comment has been minimized.

lotus42 commented May 8, 2016

Using an automated tool set up by a hosting provider to install/generate certs, I also had to modify my email address to get it to issue a new certificate, otherwise I ran into the badNonce error.

Thankfully, with gmail addresses, emailaddress@gmail is the same as email.address@gmail

@jsha

This comment has been minimized.

Contributor

jsha commented May 8, 2016

What hosting provider did you use?

@lotus42

This comment has been minimized.

lotus42 commented May 9, 2016

Apis Networks

I browsed through this list and decided on Apis:
https://github.com/certbot/certbot/wiki/Web-Hosting-Supporting-LE

@ubershmekel

This comment has been minimized.

ubershmekel commented Feb 19, 2017

I've gotten these errors and fixed them by changing the email address. Does that mean this issue should be opened @jmhodges ? Based on your cdn comment, it seems an issue might need to be opened on a different repo.

2017/02/19 01:46:29 h2_bundle.go:3223: http2: server: error reading preface from client 98.164.240.42:54611: timeout waiting for client preface
time="2017-02-19T02:13:17Z" level=error msg="map[www.tagsyo.com:acme: Error 400 - urn:acme:error:badNonce - JWS has invalid anti-replay nonce 91WN0oA5PjLkDXGV_xe8izfu0GhDAvnE0So4_DQindU]" 
time="2017-02-19T02:13:17Z" level=error msg="Error getting ACME certificates [www.tagsyo.com] : Cannot obtain certificates map[www.tagsyo.com:acme: Error 400 - urn:acme:error:badNonce - JWS has invalid anti-replay nonce 91WN0oA5PjLkDXGV_xe8izfu0GhDAvnE0So4_DQindU]+v"  
@cpu

This comment has been minimized.

Member

cpu commented Feb 20, 2017

@ubershmekel What client are you using? Contact emails are not related to badNonce errors. Can you please open a new issue with the problem you are seeing and we can help troubleshoot? It is unrelated to this closed issue.

Note for others that find this issue: The original CDN problem at the root of this particular issue has been fixed. If you are experiencing a badNonce error you should open a new issue instead of replying here.

@letsencrypt letsencrypt locked and limited conversation to collaborators Feb 21, 2017

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.