Getting "TypeError: can't concat str to bytes" while using proxy_type=PROXY_TYPE_SOCKS5

[mahirchavda]

Stacktrace:

$ C:/Python37/python3.exe c:/Users/myuser/Desktop/r.py
Traceback (most recent call last):
  File "c:/Users/myuser/Desktop/r.py", line 86, in <module>
    test_sock5_httplib2()
  File "c:/Users/myuser/Desktop/r.py", line 80, in test_sock5_httplib2
    resp, content = http.request("https://somesite.com", "GET")
  File "C:\Python37\lib\site-packages\httplib2\__init__.py", line 1953, in request
    cachekey,
  File "C:\Python37\lib\site-packages\httplib2\__init__.py", line 1618, in _request
    conn, request_uri, method, body, headers
  File "C:\Python37\lib\site-packages\httplib2\__init__.py", line 1524, in _conn_request
    conn.connect()
  File "C:\Python37\lib\site-packages\httplib2\__init__.py", line 1305, in connect
    sock.connect((self.host, self.port))
  File "C:\Python37\lib\site-packages\httplib2\socks.py", line 488, in connect
    self.__negotiatesocks5(destpair[0], destpair[1])
  File "C:\Python37\lib\site-packages\httplib2\socks.py", line 281, in __negotiatesocks5
    + self.__proxy[5]
TypeError: can't concat str to bytes

System Detail:

>>> import sys
>>> sys.version
'3.7.1 (v3.7.1:260ec2c36a, Oct 20 2018, 14:05:16) [MSC v.1915 32 bit (Intel)]'
>>> import httplib2
>>> httplib2.__version__
'0.13.1'
>>> 

Source code(r.py):

def test_sock5_httplib2():
    from httplib2 import Http, socks, ProxyInfo

    proxy = ProxyInfo(proxy_type=socks.PROXY_TYPE_SOCKS5,
                                   proxy_host="IP",
                                   proxy_port=1080,
                                   proxy_user="user",
                                   proxy_pass="password",
                                   proxy_rdns=False)

    http = Http(proxy_info = proxy)
    # http = Http()
    resp, content = http.request("https://somesite.com", "GET")
    print(resp.status)

if __name__ == "__main__":
    test_sock5_httplib2()

[temoto]

@mahirchavda hello. Thanks for great bug report with test. fc5986c this is fix if you need super fast. Our merge/release process typically takes 1-3 calendar days.

[temoto]

Oh and if you just use bytes for proxy user/pass in your code - that's kind of solution too.

[mahirchavda]

@temoto Thanks for quick fix

[mahirchavda]

Hi @temoto, I'm still getting same error.
Could you please try to run r.py file in your environment to reproduce the issue.

I performed below step in order to test the latest code

$ python3 -m pip uninstall httplib2
$ git clone https://github.com/httplib2/httplib2.git
$ cd httplib2
$ git checkout proxy-auth-bytes
$ python3 setup.py install
$ C:/Python37/python3.exe c:/Users/myuser/Desktop/r.py

The same stack trace:

Traceback (most recent call last):
  File "c:/Users/myuser/Desktop/r.py", line 93, in <module>
    test_sock5_httplib2()
  File "c:/Users/myuser/Desktop/r.py", line 87, in test_sock5_httplib2
    resp, content = http.request("https://somesite.com", "GET")
  File "C:\Python37\lib\site-packages\httplib2-0.13.1-py3.7.egg\httplib2\__init__.py", line 1957, in request
    cachekey,
  File "C:\Python37\lib\site-packages\httplib2-0.13.1-py3.7.egg\httplib2\__init__.py", line 1622, in _request
    conn, request_uri, method, body, headers
  File "C:\Python37\lib\site-packages\httplib2-0.13.1-py3.7.egg\httplib2\__init__.py", line 1528, in _conn_request
    conn.connect()
  File "C:\Python37\lib\site-packages\httplib2-0.13.1-py3.7.egg\httplib2\__init__.py", line 1309, in connect
    sock.connect((self.host, self.port))
  File "C:\Python37\lib\site-packages\httplib2-0.13.1-py3.7.egg\httplib2\socks.py", line 482, in connect
    self.__negotiatesocks5(destpair[0], destpair[1])
  File "C:\Python37\lib\site-packages\httplib2-0.13.1-py3.7.egg\httplib2\socks.py", line 275, in __negotiatesocks5
    + self.__proxy[5]
TypeError: can't concat str to bytes 

[mahirchavda]

I also tried to use bytes for proxy user/pass. It is throwing the same error.

[temoto]

Sorry, I thought there is one obvious code location that could influence this and didn't run your test. Will do more thorough job this time.

…

On Sat, Sep 14, 2019, 02:39 mahirchavda ***@***.***> wrote: I also tried to use bytes for proxy user/pass. It is throwing the same error. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#144?email_source=notifications&email_token=AAAGTMN4GIK5JF3F5WNUO2LQJQCBNA5CNFSM4IWPIEZKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD6WIHQY#issuecomment-531399619>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAAGTMLOY3LKFXBOA7V5KSTQJQCBNANCNFSM4IWPIEZA> .

[temoto]

@mahirchavda there were two places with incorrect str/bytes type, this version works eb7190b

[mahirchavda]

It is working now. Thanks for the help @temoto.

[mahirchavda]

@temoto Is it ok to apply this patch in our codebase?

[temoto]

@mahirchavda yes, the bug fix part of this patch is fine and will not change.

We are discussing test related nuances that may change before merge into master.

[mahirchavda]

I see. Thank You.

[mahirchavda]

Out of curiosity, Why it is trying to import PySocks first in the python3? PySocks don't have support for proxy_type=PROXY_TYPE_HTTP_NO_TUNNEL.

# File: python3/httplib2/__init__.py
try:
    import socks
except ImportError:
    # TODO: remove this fallback and copypasted socksipy module upon py2/3 merge,
    # idea is to have soft-dependency on any compatible module called socks
    from . import socks

[temoto]

I remember one request to use system-wide socks. It kinda makes sense to remain small HTTP library and offload proxy job.

[mahirchavda]

@temoto Can I close this issue?

[temoto]

@mahirchavda if you feel that the problem is solved, of course go ahead.

[mahirchavda]

Thanks. It is resolved in v0.14.0 Hence closing this issue.