Fixed a problem with proxy connection: [Errno 115] Operation now in progress

[ov7a]

Setup

A simple script:

from telethon.sync import TelegramClient
import socks
import logging
logging.basicConfig(level=logging.DEBUG)

api_id = 11111
api_hash = '...'
proxy=(socks.SOCKS4, '127.0.0.1', 9050)

with TelegramClient('anon', api_id, api_hash, proxy=proxy, timeout=60) as client:
	print(client.get_me().stringify())

The socks proxy is Tor, so it is quite slow.

Expected behavior

The script connect to telegram through the proxy and prints user info.

Actual behavior

The script fails to connect to the proxy. Log:

DEBUG:asyncio:Using selector: EpollSelector
INFO:telethon.network.mtprotosender:Connecting to 149.154.167.51:443/TcpFull...
DEBUG:telethon.network.mtprotosender:Connection attempt 1...
WARNING:telethon.network.mtprotosender:Attempt 1 at connecting failed: ProxyConnectionError: Error connecting to SOCKS4 proxy 127.0.0.1:9050: [Errno 115] Operation now in progress
DEBUG:telethon.network.mtprotosender:Connection attempt 2...
WARNING:telethon.network.mtprotosender:Attempt 2 at connecting failed: ProxyConnectionError: Error connecting to SOCKS4 proxy 127.0.0.1:9050: [Errno 115] Operation now in progress
...

The problem

According to python docs:

The connect() operation is also subject to the timeout setting, and in general it is recommended to call settimeout() before calling connect() or pass a timeout parameter to create_connection(). However, the system network stack may also return a connection timeout error of its own regardless of any Python socket timeout setting.

The error description for code 115 says:

EINPROGRESS
The socket is nonblocking and the connection cannot be completed
immediately. It is possible to select(2) or poll(2) for completion by
selecting the socket for writing. After select(2) indicates
writability, use getsockopt(2) to read the SO_ERROR option at level
SOL_SOCKET to determine whether connect() completed successfully
(SO_ERROR is zero) or unsuccessfully (SO_ERROR is one of the usual
error codes listed here, explaining the reason for the failure).

It seems that the tcp handshake with proxy is not being finished with time.

The solution

I've changed setblocking to settimeout and it worked great for me.

[Lonami]

The problem with this change is that now it blocks the entire thread, right? So other asyncio tasks cannot make progress while a sender is connecting via proxy.

I think switching to an async alternative of pysocks would work, but that would be a breaking change unless we translate the values from it to said new library now that I think about it.

[ov7a]

I understand your concern about thread blocking, but you are already doing it for ssl wrapping, aren't you? I thought it would be ok to block the thread just to establish a connection properly.

I've lurked the pysocks repo more thoroughly and found your issue about blocking behavior. It seems like there are indeed problems with async and pysocks. But don't known about any decent alternatives for socks proxy connection. Do you know any?

[Lonami]

but you are already doing it for ssl wrapping

That's a good observation, and yes, we are currently doing it for SSL which is unfortunate. Preferably, we would avoid any more blocking than strictly necessary.

But don't known about any decent alternatives for socks proxy connection. Do you know any?

Back when I tried to prepare the library for v2, I found pproxy (pproxy on PyPi) which seems somewhat popular.

[ov7a]

I think this issue essentially blocks Telethon usage for users from countries where telegram is blocked. At least the error should be more descriptive. Even in case when proxy works fast and a connection can be established successfully. One day it can randomly begin to operate slightly slower and a user can get a hard-to-debug problem.

I think it's better to have reliable but blocking solution rather than asynchronous but unreliable solution.

As for v2, branches v2 and v2-old do not look very alive for me. Do you still work on them?

[Lonami]

I only pointed out the v2 branches to explain where I tried pproxy, and nothing else.

I guess I can accept this patch since it doesn't affect people not using proxy and helps people using them.

[ov7a]

Awesome! Thank you!