7.69, 7.70 busy-loops when doing SOCKS connection

[zloi-user]

curl loads the cpu, with many threads this is especially noticeable

on version 7.68.0 the problem is not observed

[bagder]

Then please give us a lot more details about the problem and how to reproduce!

[bagder]

That's not very helpful.

[zloi-user]

I have a libcurl-based proxy checker, when testing part of the bad proxies 7.69 and 7.70 - it loads the system 100%, on version 7.68 only ~2%

sorry for my english, used google translator

perl script example

#!/usr/bin/perl -w
use strict;
use WWW::Curl::Easy;
use POSIX ':sys_wait_h';

use constant MAX_FORKs => 100;
$SIG{CHLD}='IGNORE';

my @apids;

print "Run 'top -p $$' test start in 15 seconds\n";
sleep 15;

my @aproxy=(
'95.216.33.245:9514',
'186.226.172.115:63253',
'181.101.51.44:1080',
'41.164.169.106:61671',
'187.73.68.14:53281',
'177.73.47.110:4145',
'115.88.138.251:4145',
'181.102.25.18:1080',
'125.26.108.12:38150',
'92.245.104.154:49384',
'94.139.176.179:1085',
'134.0.63.134:1723',
'165.16.54.244:31618',
'121.230.103.6:38801',
'186.225.32.97:1080',
'120.28.110.216:4145',
'211.35.73.254:4145',
'200.212.2.125:61283',
'181.102.116.157:7071',
'123.168.67.70:54321',
'117.69.147.113:38801',
'178.150.194.243:4153',
'181.3.111.84:1080',
'138.68.240.218:3128',
'178.218.58.234:4145',
'199.188.93.247:8000',
'95.68.115.202:53281',
'91.247.250.215:4145',
'45.232.249.73:4145',
'183.161.145.168:38801',
'185.134.99.66:51327',
'114.104.182.54:38801',
'200.152.78.48:38543',
'27.22.132.77:38801',
'118.174.220.32:51399',
'196.28.234.66:1080',
'181.5.241.176:7071',
'195.154.106.167:80',
'187.103.15.17:40559',
'175.41.44.43:34079',
'36.66.61.7:46118',
'181.102.28.40:7071',
'184.168.146.10:43227',
'168.181.134.119:41351',
'94.124.193.243:48258',
'41.215.37.230:4145',
'182.52.51.47:51449',
'186.84.172.7:60657',
'91.236.251.131:9251',
'181.102.182.177:1080',
'143.137.126.37:63253',
'181.102.84.217:7071',
'176.107.251.132:4145',
'125.25.165.22:4153',
'109.162.241.109:1085',
'185.85.219.74:61068',
'212.1.97.254:4145',
'103.239.254.114:50409',
'185.162.0.74:10801',
'175.111.15.2:33159',
'190.128.26.99:45412',
'61.145.48.38:4216',
'150.129.52.74:6667',
'178.219.173.193:4145',
'186.225.194.78:34110',
'43.248.73.122:31308',
'209.16.78.27:54321',
'103.210.142.54:4145',
'14.116.208.123:38801',
'181.211.38.62:35069',
'181.3.102.155:1080',
'113.12.202.50:53324',
'112.4.232.37:1080',
'195.22.253.238:4145',
'49.85.97.57:38801',
'86.34.197.6:23500',
'85.15.179.235:4145',
'37.26.136.181:33578',
'165.227.20.194:9050',
'177.125.206.227:4145',
'177.200.159.213:32817',
'178.47.131.202:4145',
'52.68.105.182:80',
'199.188.93.91:9000',
'81.89.113.142:44493',
'60.169.213.35:38801',
'192.158.15.201:50877',
'216.109.197.49:36974',
'45.228.4.90:4145',
'117.65.99.236:38801',
'181.3.63.8:7071',
'181.7.223.7:1080',
'176.192.8.206:61339',
'177.11.156.246:4145',
'124.156.98.172:80',
'203.77.240.75:4145',
'114.33.206.143:36346',
'45.112.11.25:1080',
'190.144.116.34:32877',
'150.129.201.30:6667'
);


for(my $i_pid=0;$i_pid<MAX_FORKs; $i_pid++){
 fork_curl($aproxy[$i_pid]) unless $apids[$i_pid]=fork;
}

for my $cur_pid (@apids){
	while($cur_pid && waitpid($cur_pid,WNOHANG)==0){
		sleep 2
	}
}

sub fork_curl {
 my $addres_porxy=shift;
 my $response_body='';
 my $curl=WWW::Curl::Easy->new;
 $curl->setopt(CURLOPT_WRITEDATA,\$response_body);
 $curl->setopt(CURLOPT_PROXY,'socks4://'.$addres_porxy);
 $curl->setopt(CURLOPT_URL,'https://www.google.com/favicon.ico');
 $curl->perform();
 exit 0;
}

[jay]

Can you reproduce without forking? Is there a specific proxy connection causing the problem?

[zloi-user]

Can you reproduce without forking? Is there a specific proxy connection causing the problem?

try the proxy 115.88.138.251:4145, I have this proxy currently loads the cpu core 100%

curl --proxy 'socks4://115.88.138.251:4145' 'https://www.google.com/favicon.ico' -o - >>/dev/null

if it does not load the processor, try to run several times

[jzakrzewski]

I've quickly tried it out. It doesn't hang each time but it happens

me@machine /tmp $ curl --proxy 'socks4://115.88.138.251:4145' 'https://www.google.com/favicon.ico'
Warning: Binary output can mess up your terminal. Use "--output -" to tell 
Warning: curl to output it to your terminal anyway, or consider "--output 
Warning: <FILE>" to save to a file.
me@machine /tmp $ curl --proxy 'socks4://115.88.138.251:4145' 'https://www.google.com/favicon.ico' -v
*   Trying 115.88.138.251:4145...
* SOCKS4 communication to www.google.com:443
* SOCKS4 connect to IPv4 216.58.215.228 (locally resolved)
^C
me@machine /tmp $ curl -V
curl 7.70.0 (x86_64-pc-linux-gnu) libcurl/7.70.0 OpenSSL/1.1.1g zlib/1.2.11 libssh2/1.9.0_DEV
Release-Date: 2020-04-29
Protocols: dict file ftp ftps http https imap imaps pop3 pop3s rtsp scp sftp smtp smtps tftp 
Features: HTTPS-proxy Largefile libz NTLM SSL TLS-SRP UnixSockets
me@machine /tmp $ 

During the second run I had 96% on one core used by curl.

[jay]

if it does not load the processor, try to run several times

...

I've quickly tried it out. It doesn't hang each time but it happens

I ran it a few twenty times and I don't see anything unusual here. Sometimes the transfer succeeds and sometimes it doesn't due to RST (shown as "Failed receiving connect request ack").

curl 7.71.0-DEV (i386-pc-win32) libcurl/7.71.0-DEV OpenSSL/1.0.2t nghttp2/1.40.0

[zloi-user]

if it does not load the processor, try to run several times

...

I've quickly tried it out. It doesn't hang each time but it happens

I ran it a few twenty times and I don't see anything unusual here. Sometimes the transfer succeeds and sometimes it doesn't due to RST (shown as "Failed receiving connect request ack").

curl 7.71.0-DEV (i386-pc-win32) libcurl/7.71.0-DEV OpenSSL/1.0.2t nghttp2/1.40.0

I tried on windows to reproduce the problem - it did not work, it seems this problem is linux curl implementation

sorry for my english, used google translator

[bagder]

It is system agnostic and happens when the SOCKS proxy closes the connection while curl runs the SOCKS state machine. See my proposed fix in #5542.

[bagder]

@zloi-user: it would be awesome if you could try that patch and tell us if it fixes the problem you see!

[zloi-user]

compiled 7.69.1 with a patch

problem still exists :(

[bagder]

That was the second of the two commits, and the first one was the one that fixed the problem for me. You should rather merge both of them!

[bagder]

I rebased the PR and squashed the two commits into a single one. Try that instead!

[zloi-user]

compiled 7.69.1 with two patches
1,2

cheers, the problem does not occur :)

[jay]

It is system agnostic and happens when the SOCKS proxy closes the connection while curl runs the SOCKS state machine. See my proposed fix in #5542.

Both Windows and Linux recv return 0 repeatedly but then Windows after a second will return -1 WSAECONNABORTED which is why it doesn't busy loop.