Unable to dispatch client connection over multiprocessing queue

[SeanZhang1212]

it was working fine with commits on Jun 4, 2019, but now throws below error when accessing http://httpbin.org/ip

Traceback (most recent call last):
  File "/usr/lib64/python2.7/multiprocessing/queues.py", line 266, in _feed
    send(obj)
TypeError: expected string or Unicode object, NoneType found

[SeanZhang1212]

DEBUG output when accessing http://httpbin.org/ip with Jun 4's commits:

2019-07-30 10:36:12,079 - INFO - run:653 - Starting server on port 8899
2019-07-30 10:36:16,317 - DEBUG - run:619 - Proxying connection <socket._socketobject object at 0x7fae32a9d360>
2019-07-30 10:36:16,327 - DEBUG - _process_rlist:551 - client is ready for reads, reading
2019-07-30 10:36:16,327 - DEBUG - recv:351 - rcvd 134 bytes from client
2019-07-30 10:36:16,327 - DEBUG - _process_request:470 - request parser is in state complete
2019-07-30 10:36:16,327 - DEBUG - _process_request:490 - connecting to server httpbin.org:80
2019-07-30 10:36:16,594 - DEBUG - _process_request:492 - connected to server httpbin.org:80
2019-07-30 10:36:16,594 - DEBUG - _process_wlist:545 - server is ready for writes, flushing server buffer
2019-07-30 10:36:16,595 - DEBUG - flush:377 - flushed 140 bytes to server
2019-07-30 10:36:16,806 - DEBUG - _process_rlist:568 - server is ready for reads, reading
2019-07-30 10:36:16,806 - DEBUG - recv:351 - rcvd 393 bytes from server
2019-07-30 10:36:16,807 - DEBUG - _process_wlist:541 - client is ready for writes, flushing client buffer
2019-07-30 10:36:16,807 - DEBUG - flush:377 - flushed 393 bytes to client
2019-07-30 10:36:16,807 - DEBUG - _process_rlist:568 - server is ready for reads, reading
2019-07-30 10:36:16,807 - DEBUG - recv:349 - rcvd 0 bytes from server
2019-07-30 10:36:16,807 - DEBUG - _process_rlist:573 - server closed connection
2019-07-30 10:36:16,807 - DEBUG - _process:604 - client buffer is empty and response state is complete, breaking
2019-07-30 10:36:16,807 - DEBUG - run:628 - closing client connection with pending client buffer size 0 bytes
2019-07-30 10:36:16,807 - DEBUG - run:632 - closed client connection with pending server buffer size 0 bytes
2019-07-30 10:36:16,807 - INFO - _access_log:527 - 223.104.212.61:5202 - GET httpbin.org:80/ip - 200 OK - 393 bytes
2019-07-30 10:36:16,807 - DEBUG - run:634 - Closing proxy for connection <socket._socketobject object at 0x7fae32a9d360> at address ('223.104.212.61', 5202)

[SeanZhang1212]

And output when accessing http://httpbin.org/ip with Jul 24's commits:

[root@10-10-133-28 proxy.py]# python proxy_Jul24.py --hostname 0.0.0.0 --port 8899 --log-level DEBUG --ipv4
2019-07-30 10:40:21,087 - INFO - pid:15820 - setup:723 - Starting 2 workers
2019-07-30 10:40:21,090 - INFO - pid:15820 - run:686 - Started server on port 8899
Traceback (most recent call last):
  File "/usr/lib64/python2.7/multiprocessing/queues.py", line 266, in _feed
    send(obj)
TypeError: expected string or Unicode object, NoneType found

[abhinavsingh]

Thanks @SeanZhang1212 for flagging it. This is most likely caused by #56 which introduced concept of multiple worker threads. I did successfully test the changes on Mac with PY3. Will check on Linux and other OSes. I see you are running on a Linux box. Thanks again.

[abhinavsingh]

Can confirm it only affects Python 2.7 version.

Root cause is inability to send accepted client connection over multiprocessing queue, which is no longer the case with Python 3+. Please upgrade to version 3+, proxy.py will soon deprecate support for 2.7 version.

2.7 will also be officially deprecated in 2020.

[abhinavsingh]

Following commit will no longer allow Python 2.7 users to run proxy.py from develop branch.

[abhinavsingh]

Marking as won't fix as supporting Python2.7 is not the effort at this stage. Specially since proxy.py can run within silo Python3+ virtual environments.