Unknown SSL protocol error in connection to any site through HTTPS #2299
Comments
I can't reproduce that here in Ubuntu. I also tried in Windows using libcurl and each of mbedTLS, OpenSSL, WinSSL and wolfSSL. Is it possible something is intercepting the connection? Can you still reproduce? Do other https requests work? curl 7.52.1 (x86_64-pc-linux-gnu) libcurl/7.52.1 OpenSSL/1.0.2j zlib/1.2.8 nghttp2/1.18.1 librtmp/2.3 |
I don't know how to check it. If you have any suggestions, please share it. This machine connected directly to provider coaxial modem and problem is still there. Actually problem is not new, I was witnessing it at least few months but I've decided to do something with it :) Results of fresh tests:
Windows 10 machine (no problems):
No. No one HTTPS request works. |
#1681 is another report with the same curl on stretch, but that only affected a select host. On the affected computer use wireshark and set capture filter [1]: GitHub only allows certain file types like zip. Also, before zipping it you may want to remove possibly sensitive information from the packet capture such as IP/MAC by using TraceWrangler, but disable remove unknown layers. |
U can try to do it by this solution |
@mrsmallyi, thank you for suggestion but this solution does not help in my case, errors are the same. @jay, I was wrong when I said "No one HTTPS request work". Requests to https://test.com suddenly work. However requests to https://google.com and https://packagist.org still lead to error. Dumps: dump-curl-test-20180211.zip Logs of requests:
Successful request (https://packagist.org using wget):
Successful request (https://test.com using curl)
|
The ClientHello is essentially the same for both the good and the bad connection. For bad conn to packagist.com there is no reply received to the ClientHello, despite several retransmissions over the next minute. Exactly 1 minute after the ClientHello the server initiates a close FIN which is what packagist would do without receiving one. Also if you look at the TSecr (timestamp echo reply) the number is from before the ClientHello was sent, meaning it has not received the ClientHello or the retransmissions, since if it did it could send any one of those as the echoreply. I don't know why it's happening but I doubt it has to do with curl. It's possible something curl does may contribute, for example the ClientHello may be seen as too big and rejected by some middleman? Are you in China and maybe some of these hosts are blocked? That doesn't explain why wget isn't doing that or its ClientHello is otherwise different and can pass through. Try making the ClientHello smaller by only sending a single cipher, for example here is a cipher that connected you to test.com and also works for me for packagist.com:
Also I'm curious what ClientHello wget is sending that works. |
No, I'm in Toronto and in this case it would affect the other devices in the network too. And wget also works...
Yes. it works. But regular request without 'cipher' option still does not work. I'm not sure I totally understand what cipher is and what did I change using this option. I would you very appreciate if you explain it :) Also, do you have any idea, what further actions can I do to solve my problem? Dump: curl-test-packagist-org-cipher-20180212.zip
|
Please try packagist.com not packagist.org. Sometimes you are referring to one and sometimes the other and I examined packagist.com which is in your first trace. Monitor
|
I did this mistake once and then I noticed that and remade tests and edited my comment. If you take a look to all my comments you'll see only packagist.org in them. Anyway, there is no difference in results between any of hosts of both packagist.org and packagist.com.
I did it.
Successful request to https://packagist.com using curl with one cipher:
Successful request to https://packagist.com using wget:
|
Can I use only one cipher as a temporary solution? Is it possible to set this option in config permanently? |
Thanks I downloaded the capture and was unable to reproduce. I extracted the raw ClientHello and sent it like this:
I received a ServerHello in reply. Since I am now sending the exact same ClientHello and receiving a reply I believe there is something on your end interfering with the traffic that prevents the ClientHello from being received by the server. My guess is it has something to do with the size of the ClientHello. Attached are the files I used to reproduce and also several files where I modified the raw data in the ClientHello for making the length of data in the padding extension 1, 0, and also removing the padding extension. In all cases I received a ServerHello in reply. You can try sending the raw data in netcat and monitor in Wireshark to see if you receive a server reply to any of them. (Note nc option -q90 is wait 90 seconds after sending, probably unnecessary and this server should terminate after 1 min if no ServerHello.) ClientHello_sent_via_nc__anon.zip It is possible to stop libcurl from requesting the ClientHello be padded by removing flag SSL_OP_TLSEXT_PADDING from SSL_OP_ALL: diff --git a/lib/vtls/openssl.c b/lib/vtls/openssl.c
index 2a6b3cf..94092d0 100644
--- a/lib/vtls/openssl.c
+++ b/lib/vtls/openssl.c
@@ -2181,6 +2181,9 @@ static CURLcode ossl_connect_step1(struct connectdata *con
ctx_options = SSL_OP_ALL;
+ /* disable padding test */
+ ctx_options &= ~SSL_OP_TLSEXT_PADDING;
+
#ifdef SSL_OP_NO_TICKET
ctx_options |= SSL_OP_NO_TICKET;
#endif
I think that would cause more problems than it would solve if you put it in the curlrc. There's no guarantee an arbitrary server that you connect to will accept that cipher. For that server specifically even it could stop accepting that cipher. |
Impressive research @jay! This clienthello extension extension is even documented in RFC 7685 and the RFC says:
So, while this is clearly not a curl bug we can of course discuss whether to add support for this padding option. |
@jay, I successfully receive response for raw ClientHello packet. However, I still cannot receive response for ClientHello sent by curl.
Is it possible for me to apply this patch somehow? |
@bagder, I would very appreciate if you describe, how I can do that. I installed curl by command
Is that correct sequence? If is so, what should I do on the step 3? |
./configure && make That installs it in /usr/local and you can use that version as a workaround. If you want to repackage curl/libcurl see https://wiki.debian.org/BuildingAPackage#Edit_the_source_code |
Experiencing same in latest Jenkins LTS docker image behind a corporate firewall. wget works while curl fails:
|
curl --version |
We have no reason to believe this is a curl problem. If you have more information that makes you think what you are reporting is different then please open a new issue. |
I'm going to add an item to the TODO about this extension and then move to close this issue. |
The padding AFAIK is added by default by the ssl backend when necessary but the patch here disables that for openssl. Too little is known about why the padding would be causing the issue so I don't think it's appropriate to add even as an option. |
I did this
curl -v https://packagist.org
curl -v --insecure https://packagist.org
I also tried
wget -q -S -O - https://packagist.org
and it works perfectly without any errors.I expected the following
Response from server.
I received the following
curl/libcurl version
operating system
The text was updated successfully, but these errors were encountered: