Initialization hangs when using threads

[nicorac]

I'm making some test with the treading branch and I'm having an infinite loop during initialization.

This is an excerpt of my code:

$settings['threading']['allow_threading'] = true;
$settings['app_info']['api_id'] = API_ID;
$settings['app_info']['api_hash'] = API_HASH;
$settings['logger']['logger_level'] = Logger::ULTRA_VERBOSE;

echo 'New instance of MadelineProto...'.PHP_EOL;
$MadelineProto = new API($settings);
echo 'ok'.PHP_EOL;

where the last echo 'ok' is never printed.

This is what it's logged in ULTRA_VERBOSE mode:

New instance of MadelineProto...
MTProto:                Istantiating DataCenter...
MTProto:                Loading RSA key...
RSA:                    Istantiating \phpseclib\Crypt\RSA...
RSA:                    Loading key...
RSA:                    Computing fingerprint...
RSA:                    Istantiating \phpseclib\Crypt\RSA...
RSA:                    Loading key...
RSA:                    Computing fingerprint...
RSA:                    Istantiating \phpseclib\Crypt\RSA...
RSA:                    Loading key...
RSA:                    Computing fingerprint...
RSA:                    Istantiating \phpseclib\Crypt\RSA...
RSA:                    Loading key...
RSA:                    Computing fingerprint...
MTProto:                Translating tl schemas...
TL:                     Loading TL schemes...
TL:                     Parsing TL_mtproto_v1.json...
TL:                     Translating objects...
TL:                     Translating methods...
TL:                     Parsing TL_telegram_v66.tl...
TL:                     Translating objects...
TL:                     Translating methods...
TL:                     Parsing TL_secret.tl...
TL:                     Translating objects...
TL:                     Translating methods...
TL:                     Parsing TL_calls.tl...
TL:                     CRC32 mismatch (e8201c08, dbf948c1) for decryptedDataBlock#dbf948c1 random_id:long random_bytes
:string flags:# voice_call_id:flags.2?int128 in_seq_no:flags.4?int out_seq_no:flags.4?int recent_received_mask:flags.5?int proto:flags.3?int extra:flags.1?string raw_data:flags.0?string = DecryptedDataBlock;
TL:                     CRC32 mismatch (d20a6f5c, cc0d0e76) for simpleDataBlock#cc0d0e76 random_id:long random_bytes:string raw_data:string = DecryptedDataBlock;
TL:                     Translating objects...
TL:                     Translating methods...
TL:                     Parsing TL_botAPI.tl...
TL:                     Translating objects...
TL:                     Translating methods...
MTProto:                Executing dh_prime checks (0/3)...
DataCenter:             Connecting to DC 2 (main server, ipv4, tcp_full)...
MTProto:                THREADING IS ENABLED
MTProto:                Socket reader on DC 2: CREATING
MTProto:                Socket reader on DC 2: SUBMITTING
MTProto:                Socket reader on DC 2: WAITING
MTProto:                Socket reader on DC 2: READY
MTProto:                Generating permanent authorization key for DC 2...
AuthKeyHandler:         Requesting pq
CallHandler:            Calling method (try number 1 for req_pq)...
CallHandler:            Getting response (try number 1 for req_pq)...
MTProto:                Socket reader on DC 2: WORKING
CallHandler:            Getting response (try number 1 for req_pq)...
MTProto:                Socket reader on DC 2: SUBMITTING
MTProto:                Socket reader on DC 2: WAITING
MTProto:                Socket reader on DC 2: READY
SocketReader (t):       Shutting down reader pool 2
CallHandler:            Getting response (try number 1 for req_pq)...
MTProto:                Socket reader on DC 2: SUBMITTING
MTProto:                Socket reader on DC 2: WAITING
MTProto:                Socket reader on DC 2: READY
SocketReader (t):       Shutting down reader pool 2
CallHandler:            Getting response (try number 1 for req_pq)...
MTProto:                Socket reader on DC 2: SUBMITTING
MTProto:                Socket reader on DC 2: WAITING
MTProto:                Socket reader on DC 2: READY
SocketReader (t):       Shutting down reader pool 2
CallHandler:            Getting response (try number 1 for req_pq)...
MTProto:                Socket reader on DC 2: SUBMITTING
MTProto:                Socket reader on DC 2: WAITING
MTProto:                Socket reader on DC 2: READY
SocketReader (t):       Shutting down reader pool 2

the last 5 lines are repeated forever untill I kill the process from another console (CTRL+C doesn't work).

It seems that MTProto::start_threads() never pass the while() loop that waits for the socket to become ready.

[danog]

You must compile the feature/nested-volatile branch of https://github.com/SirSnyder/pthreads in order to make basic threading work with MadelineProto (it is now thread safe, but there are still some bugs I've yet to fix)

[nicorac]

Thanks, haven't seen in documentation.
Cloned that repo and rebuilt PHP successfully.
Now the initialization goes further on but then raises an Exception:

CallHandler:            Getting response (try number 1 for auth.bindTempAuthKey)...
MTProto:                Socket reader on DC 2: WORKING

Fatal error: Uncaught RuntimeException: pthreads detected an attempt to connect to an object which has already been destroyed in /MadelineProto/src/danog/MadelineProto/MTProtoTools/CallHandler.php on line 83

RuntimeException: pthreads detected an attempt to connect to an object which has already been destroyed in /MadelineProto/src/danog/MadelineProto/MTProtoTools/CallHandler.php on line 83

Call Stack:
    0.0006     394728   1. {main}() /myTelegram.php:0
    0.0072     725184   2. danog\MadelineProto\API->__construct() /myTelegram.php:71
    0.0072     725456   3. danog\MadelineProto\API->___construct() /MadelineProto/vendor/danog/magicalserializer/src/danog/Serializable.php:35
    0.0239    2344016   4. danog\MadelineProto\MTProto->__construct() /MadelineProto/src/danog/MadelineProto/API.php:27
    0.0239    2344288   5. danog\MadelineProto\MTProto->___construct() /MadelineProto/vendor/danog/magicalserializer/src/danog/Serializable.php:35
    0.5256    4839744   6. danog\MadelineProto\MTProto->connect_to_all_dcs() /MadelineProto/src/danog/MadelineProto/MTProto.php:96
    0.6130    4931600   7. danog\MadelineProto\MTProto->init_authorization() /MadelineProto/src/danog/MadelineProto/MTProto.php:438
    7.2869    6249696   8. danog\MadelineProto\MTProto->bind_temp_auth_key() /MadelineProto/src/danog/MadelineProto/MTProto.php:467
    7.2943    6354776   9. danog\MadelineProto\MTProto->method_call() /MadelineProto/src/danog/MadelineProto/MTProtoTools/AuthKeyHandler.php:554

[danog]

And that's where I'm stuck on atm

[nicorac]

Ok, I haven't got that threading is not usable atm.

I need to continue reading incoming message while doing a long operation, like downloading a large media.
My first try was to extend Files::download_to_stream() and allow it to download the large file in small resumable parts, then alternate update reading and media download.

Will get back to that meanwhile, but if you have a better suggestion I'm all ears...

[danog]

Let's move this to #4