Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Change "localhost" test server certificate #2720

merged 2 commits into from Apr 27, 2017


Copy link

@redapple redapple commented Apr 27, 2017

I had this strange issue this afternoon that I could not understand (failing HTTPS tests even with master branch or latest tag code)
I was seeing tons of

ResponseNeverReceived: [<twisted.python.failure.Failure OpenSSL.SSL.Error: [('SSL routines', 'ssl3_read_bytes', 'tlsv1 alert internal error')]>]

TLS connections were being closed by the test server in tests/; The server was returning ServerHello but followed by an Alert, in the same payload:

Secure Sockets Layer
    TLSv1.2 Record Layer: Handshake Protocol: Server Hello
    TLSv1.2 Record Layer: Handshake Protocol: Certificate
    TLSv1.2 Record Layer: Alert (Level: Fatal, Description: Internal Error)

I eventually logged what was happening within Twisted and got this in :

_checkHandshakeStatus(): Error = Error([('rsa routines', 'RSA_sign', 'digest too big for rsa key'), ('SSL routines', 'ssl3_send_server_key_exchange', 'EVP lib')],)

So I tried regenerating the certificate and key for the "localhost" server and it worked.

I'm not sure what changed on my system to trigger this. But I think it doesn't hurt to use certificate and key that we can (re)generate (hence the accompanying README)

@@ -183,8 +183,8 @@ class HttpTestCase(unittest.TestCase):
download_handler_cls = HTTPDownloadHandler

# only used for HTTPS tests
keyfile = 'keys/cert.pem'
certfile = 'keys/cert.pem'

This comment has been minimized.


kmike Apr 27, 2017

are old files still needed?

This comment has been minimized.


redapple Apr 27, 2017
Author Contributor

Nope. I just removed it and the other reference to it.

Copy link

@codecov-io codecov-io commented Apr 27, 2017

Codecov Report

Merging #2720 into master will not change coverage.
The diff coverage is n/a.

Impacted file tree graph

@@           Coverage Diff           @@
##           master    #2720   +/-   ##
  Coverage   84.66%   84.66%           
  Files         162      162           
  Lines        9122     9122           
  Branches     1353     1353           
  Hits         7723     7723           
  Misses       1141     1141           
  Partials      258      258

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2d66c58...6d14e39. Read the comment docs.

@kmike kmike merged commit 7fc11c1 into scrapy:master Apr 27, 2017
3 checks passed
3 checks passed
codecov/patch Coverage not affected when comparing 2d66c58...6d14e39
codecov/project 84.66% remains the same compared to 2d66c58
continuous-integration/travis-ci/pr The Travis CI build passed
@kmike kmike modified the milestone: v1.5 Dec 22, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

3 participants
You can’t perform that action at this time.