This is just a question to try to understand the internals surrounding the connection streams between Client and Scheduler.
The client seems to use two different streams to communicate with the scheduler- the first is the RPC object Client.scheduler, which is used by functions such as get_dataset/publish_dataset/etc. The other is Client.scheduler_stream, which is created inside Client.ensure_connected()
Is there any fundamental reason for using two separate streams? It seems Client.scheduler_stream uses BatchedSend and is used for sending/receiving small messages, and Client.scheduler is the only one used for receiving potentially large amounts of data.
On the Scheduler side, I see how the handlers are separated into self.handlers and self.client_handlers which correspond to the two different streams.
This is just a question to try to understand the internals surrounding the connection streams between Client and Scheduler.
The client seems to use two different streams to communicate with the scheduler- the first is the RPC object Client.scheduler, which is used by functions such as get_dataset/publish_dataset/etc. The other is Client.scheduler_stream, which is created inside Client.ensure_connected()
Is there any fundamental reason for using two separate streams? It seems Client.scheduler_stream uses BatchedSend and is used for sending/receiving small messages, and Client.scheduler is the only one used for receiving potentially large amounts of data.
On the Scheduler side, I see how the handlers are separated into self.handlers and self.client_handlers which correspond to the two different streams.