Skip to content

Unhandled exception futures.TimeoutError() #221

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
6 of 13 tasks
am11y opened this issue Nov 29, 2021 · 16 comments
Closed
6 of 13 tasks

Unhandled exception futures.TimeoutError() #221

am11y opened this issue Nov 29, 2021 · 16 comments

Comments

@am11y
Copy link

am11y commented Nov 29, 2021

Check this or we will delete your issue. (fill in the checkbox with an X like so: [x])

  • I have searched for other issues with the same problem or similar feature requests.

Select one:

  • Bug
  • Feature Request
  • Technical Help
  • Other

Environment

  • Are you using the module on a VPS or other Cloud hosting?
  • Are you using the module on a Raspberry Pi?

What kind of internet connection do you have?

Optics.

Average system load (CPU)

6 × AMD EPYC 7282 16-Core Processor
10% avg load

Hardware specification

Include here a description of the server hardware.

Operating System? (include version)

  • macOS
  • Windows
  • Linux : Debian GNU/Linux 10

Options

  • stream_buffer
  • process_stream_data

Which endpoint do you connect?

binance.com-futures

Python Version Requirement

  • I am using Python 3.7 or above

Exact Python Version?

3.7.3

Pip Version?

18.1

Dependencies

https://pastebin.com/6uDicgT9

UNICORN Binance WebSocket API Version?

1.34.2

Description of your issue

I get the exceptions below on regular basis, many times consecutively in a row:
Exception in thread Thread-1800:
Traceback (most recent call last):
  File "/usr/lib/python3.7/threading.py", line 917, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.7/threading.py", line 865, in run
    self._target(*self._args, **self._kwargs)
  File "/home/user/.local/lib/python3.7/site-packages/unicorn_binance_websocket_api/unicorn_binance_websocket_api_manager.py", line 473, in _create_stream_thread
    loop.run_until_complete(socket.start_socket())
  File "/usr/lib/python3.7/asyncio/base_events.py", line 584, in run_until_complete
    return future.result()
  File "/home/user/.local/lib/python3.7/site-packages/unicorn_binance_websocket_api/unicorn_binance_websocket_api_socket.py", line 68, in start_socket
    symbols=self.symbols) as websocket:
  File "/home/user/.local/lib/python3.7/site-packages/unicorn_binance_websocket_api/unicorn_binance_websocket_api_connection.py", line 147, in __aenter__
    self.manager.websocket_list[self.stream_id] = await self._conn.__aenter__()
  File "/home/user/.local/lib/python3.7/site-packages/websockets/legacy/client.py", line 632, in __aenter__
    return await self
  File "/home/user/.local/lib/python3.7/site-packages/websockets/legacy/client.py", line 649, in __await_impl_timeout__
    return await asyncio.wait_for(self.__await_impl__(), self.open_timeout)
  File "/usr/lib/python3.7/asyncio/tasks.py", line 423, in wait_for
    raise futures.TimeoutError()
concurrent.futures._base.TimeoutError

@am11y
Copy link
Author

am11y commented Dec 1, 2021

Hello, anybody here? I get this error constantly and it is very annoying. Does it happen to anyone else too? What should be done to fix it?

@oliver-zehentleitner
Copy link
Member

In old python versions i never had that issue in 2 years.

We have similar new open issues related to this issue.

Please add answers to the questions we ask when you created this new issue. This information help me to solve this issue.

I am planing a bigger release at the end of this year with a fix for this bug.

@am11y
Copy link
Author

am11y commented Dec 1, 2021

Please add answers to the questions

Fixed

@oliver-zehentleitner
Copy link
Member

stream_buffer or process_stream_data?

@am11y
Copy link
Author

am11y commented Dec 1, 2021

Sorry, I don't know what is this.
Here is the initialization code for stream:

self.__socket_manager_fapi = BinanceWebSocketApiManager( exchange = "binance.com-futures" )
self.__fapi_thread = threading.Thread( target = self.__callback_message, args = ( self.__socket_manager_fapi, callbacks_fapi, ) )
self.__fapi_thread.start()

def __callback_message( self, socket, callback_fcns ):
        while True:
            if socket.is_manager_stopping():
                exit( 0 )
            stream_data = socket.pop_stream_data_from_stream_buffer()
            if stream_data is False:
                time.sleep( 0.01 )
            else:
                data = json.loads( stream_data )
                if not "data" in data:
                    printJson( data )
                    continue
                kline_data = data["data"]["k"]

@oliver-zehentleitner
Copy link
Member

Then you use the stream_buffer! Thanks, this infos help!

@am11y
Copy link
Author

am11y commented Dec 1, 2021

Could you please point me to the docs which explain what is the difference between stream_buffer // process_stream_data ?

@oliver-zehentleitner
Copy link
Member

@am11y
Copy link
Author

am11y commented Dec 7, 2021

Hello Oliver,
this problem completely destroys my application. I get spammed with these errors and stream is unusable. Could you please point out to which version of unicorn-bn-websocket package should I revert? Perhaps to the one that does not use async.io? How can I temporarily avoid this until your fix is ready?
Thanks.

@oliver-zehentleitner
Copy link
Member

oliver-zehentleitner commented Dec 7, 2021

Hello!

I understand your problem and your needs, i am just very busy atm :/

I think the queue of awaits is getting to long and then timeouting. How many subscriptions do you have?

You must know, Python is really cool, but it sucks with concurrency, even if you have 6 cpu sockets, each with 16 cores. You just use the resources of one cpu core. Read here about the reason: https://realpython.com/python-gil/

I wanted to study your log file, but did not find it. Wasnt it already posted?

Your python version is well tested with UBWA, i guess the root cause for the error you get is an overloaded application.

I can release a fix to reconnect, but you would still not be stable in cause of all the unneccessary reconnects.

There is no version without asyn.io and never was...

@am11y
Copy link
Author

am11y commented Dec 7, 2021

I have ~50 different streams.

That's why I asked this question: #220

Is it of any use if I try to pack these 50 subscriptions in single stream?

So there is no old version that I can fall back to in order to rescue the situation temporarily?

@am11y
Copy link
Author

am11y commented Dec 8, 2021

Here is the log file:
https://pastebin.com/AfSTmYiQ

@oliver-zehentleitner
Copy link
Member

The link does not work. you can drag&drop the log file to the text box of a new comment.

@oliver-zehentleitner
Copy link
Member

You have 50 streams with one subscription each? If yes, make one stream for each channel like trade, kline_1m, ... and use them for up to 1024 subscriptions per stream. Futures is 200 subscriptions per stream: https://github.com/oliver-zehentleitner/unicorn-binance-websocket-api/wiki/Binance-websocket-endpoint-configuration-overview

@am11y
Copy link
Author

am11y commented Dec 8, 2021

asdfasdf.log

@oliver-zehentleitner
Copy link
Member

Seven-112 pushed a commit to Seven-112/unicorn-binance-websocket-api that referenced this issue May 16, 2023
Seven-112 pushed a commit to Seven-112/unicorn-binance-websocket-api that referenced this issue May 16, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants