Closed
Description
Consider this:
>>> c=pycurl.Curl() # disable dns cache >>> c.setopt(c.DNS_CACHE_TIMEOUT,0) >>> c.setopt(c.VERBOSE,1) # set ip to .102 >>> c.setopt(c.RESOLVE,['ya.ru:80:127.0.0.102']) >>> c.setopt(c.URL,'http://ya.ru') >>> c.perform() * Added ya.ru:80:127.0.0.102 to DNS cache # ^ ok * Rebuilt URL to: http://ya.ru/ * Hostname ya.ru was found in DNS cache * Trying 127.0.0.102... # ^ ok * TCP_NODELAY set * connect to 127.0.0.102 port 80 failed: Connection refused * Failed to connect to ya.ru port 80: Connection refused * Closing connection 0 Traceback (most recent call last): File "", line 1, in pycurl.error: (7, 'Failed to connect to ya.ru port 80: Connection refused') # clear fake dns >>> c.setopt(c.RESOLVE,['-ya.ru:80']) # set fake dns to another ip, .103 >>> c.setopt(c.RESOLVE,['ya.ru:80:127.0.0.103']) >>> c.perform() * Added ya.ru:80:127.0.0.103 to DNS cache # ^ ok * Rebuilt URL to: http://ya.ru/ * Hostname ya.ru was found in DNS cache * Trying 127.0.0.102... # ^ ??? uses old ip * TCP_NODELAY set * connect to 127.0.0.102 port 80 failed: Connection refused * Failed to connect to ya.ru port 80: Connection refused * Closing connection 1 Traceback (most recent call last): File "", line 1, in pycurl.error: (7, 'Failed to connect to ya.ru port 80: Connection refused') >>>
If I remove fake dns entry a request is made to the real ip:
# continuing with the same curl handle >>> c.setopt(c.RESOLVE,['-ya.ru:80']) >>> c.perform() * Rebuilt URL to: http://ya.ru/ * Trying 87.250.250.242... * TCP_NODELAY set * Connected to ya.ru (87.250.250.242) port 80 (#2) > GET / HTTP/1.1 Host: ya.ru User-Agent: PycURL/7.43.0.1 libcurl/7.56.1 OpenSSL/1.0.2l zlib/1.2.8 libidn2/2.0.2 libpsl/0.18.0 (+libidn2/2.0.2) libssh2/1.8.0 nghttp2/1.26.0 librtmp/2.3 Accept: */* < HTTP/1.1 302 Found < Date: Sat, 09 Dec 2017 05:52:48 GMT < Cache-Control: no-cache,no-store,max-age=0,must-revalidate < Location: https://ya.ru/ < Expires: Sat, 09 Dec 2017 05:52:49 GMT < Last-Modified: Sat, 09 Dec 2017 05:52:49 GMT < P3P: policyref="/w3c/p3p.xml", CP="NON DSP ADM DEV PSD IVDo OUR IND STP PHY PRE NAV UNI" < Set-Cookie: yandexuid=4809395281512798769; Expires=Tue, 07-Dec-2027 05:52:48 GMT; Domain=.ya.ru; Path=/ < X-XSS-Protection: 1; mode=block < X-Content-Type-Options: nosniff < Content-Length: 0 < * Connection #2 to host ya.ru left intact >>>
But now if I set another fake ip, curl makes a connection to the real ip:
# still using the same handle >>> c.setopt(c.RESOLVE,['ya.ru:80:127.0.0.104']) >>> c.perform() * Added ya.ru:80:127.0.0.104 to DNS cache # ^ ok * Rebuilt URL to: http://ya.ru/ * Found bundle for host ya.ru: 0x559567269060 [can pipeline] * Re-using existing connection! (#2) with host ya.ru * Connected to ya.ru (87.250.250.242) port 80 (#2) > GET / HTTP/1.1 Host: ya.ru User-Agent: PycURL/7.43.0.1 libcurl/7.56.1 OpenSSL/1.0.2l zlib/1.2.8 libidn2/2.0.2 libpsl/0.18.0 (+libidn2/2.0.2) libssh2/1.8.0 nghttp2/1.26.0 librtmp/2.3 Accept: */* < HTTP/1.1 302 Found < Date: Sat, 09 Dec 2017 05:53:51 GMT < Cache-Control: no-cache,no-store,max-age=0,must-revalidate < Location: https://ya.ru/ < Expires: Sat, 09 Dec 2017 05:53:52 GMT < Last-Modified: Sat, 09 Dec 2017 05:53:52 GMT < P3P: policyref="/w3c/p3p.xml", CP="NON DSP ADM DEV PSD IVDo OUR IND STP PHY PRE NAV UNI" < Set-Cookie: yandexuid=5327304491512798831; Expires=Tue, 07-Dec-2027 05:53:51 GMT; Domain=.ya.ru; Path=/ < X-XSS-Protection: 1; mode=block < X-Content-Type-Options: nosniff < Content-Length: 0 < * Connection #2 to host ya.ru left intact >>>
OK, maybe I need to also set FORBID_REUSE. Let's try that.
# still on the same curl handle >>> c.setopt(c.FORBID_REUSE,1) >>> c.perform() * Rebuilt URL to: http://ya.ru/ * Found bundle for host ya.ru: 0x559567269060 [can pipeline] * Re-using existing connection! (#2) with host ya.ru # ^ ??? Didn't I forbid reuse? * Connected to ya.ru (87.250.250.242) port 80 (#2) > GET / HTTP/1.1 Host: ya.ru User-Agent: PycURL/7.43.0.1 libcurl/7.56.1 OpenSSL/1.0.2l zlib/1.2.8 libidn2/2.0.2 libpsl/0.18.0 (+libidn2/2.0.2) libssh2/1.8.0 nghttp2/1.26.0 librtmp/2.3 Accept: */* < HTTP/1.1 302 Found < Date: Sat, 09 Dec 2017 05:55:05 GMT < Cache-Control: no-cache,no-store,max-age=0,must-revalidate < Location: https://ya.ru/ < Expires: Sat, 09 Dec 2017 05:55:05 GMT < Last-Modified: Sat, 09 Dec 2017 05:55:05 GMT < P3P: policyref="/w3c/p3p.xml", CP="NON DSP ADM DEV PSD IVDo OUR IND STP PHY PRE NAV UNI" < Set-Cookie: yandexuid=3103926351512798905; Expires=Tue, 07-Dec-2027 05:55:05 GMT; Domain=.ya.ru; Path=/ < X-XSS-Protection: 1; mode=block < X-Content-Type-Options: nosniff < Content-Length: 0 < * Closing connection 2 >>>
Let's try this all again:
>>> c=pycurl.Curl() >>> c.setopt(c.VERBOSE,1) >>> c.setopt(c.FORBID_REUSE,1) >>> c.setopt(c.DNS_CACHE_TIMEOUT,0) >>> c.setopt(c.RESOLVE,['ya.ru:80:127.0.0.104']) >>> c.setopt(c.URL,'http://ya.ru') >>> c.perform() * Added ya.ru:80:127.0.0.104 to DNS cache * Rebuilt URL to: http://ya.ru/ * Hostname ya.ru was found in DNS cache * Trying 127.0.0.104... * TCP_NODELAY set * connect to 127.0.0.104 port 80 failed: Connection refused * Failed to connect to ya.ru port 80: Connection refused * Closing connection 0 Traceback (most recent call last): File "", line 1, in pycurl.error: (7, 'Failed to connect to ya.ru port 80: Connection refused') >>> c.setopt(c.RESOLVE,['-ya.ru:80']) >>> c.setopt(c.RESOLVE,['ya.ru:80:127.0.0.105']) >>> c.perform() * Added ya.ru:80:127.0.0.105 to DNS cache * Rebuilt URL to: http://ya.ru/ * Hostname ya.ru was found in DNS cache * Trying 127.0.0.104... # ^ shouldn't it try to connect to .105? * TCP_NODELAY set * connect to 127.0.0.104 port 80 failed: Connection refused * Failed to connect to ya.ru port 80: Connection refused * Closing connection 1 Traceback (most recent call last): File "", line 1, in pycurl.error: (7, 'Failed to connect to ya.ru port 80: Connection refused') >>>
I think something in the DNS cache/connection reuse does not interact properly with RESOLVE option.
This came out of me investigating pycurl/pycurl#443 which, strictly speaking, appears to be a user error as the user did not disable DNS cache and connection reuse.
Metadata
Metadata
Assignees
Labels
No labels