Continuous Websocket Message Reading on Readiness Event

[Jayanring]

• Read message repeatly when receive a readiness event

[linear]

P-894 Bitacross-Worker Stuck When Handling Concurrent Requests

[Kailai-Wang]

Good catch, thanks!

Should identity worker (in tee-worker/) be changed too?

[BillyWooo]

Should identity worker (in tee-worker/) be changed too?

I think it should.

[kziemianek]

ws server is single threaded so draining connection will block it until it's finished.
This might be good enough for bitacross-worker but not for tee-worker.

However Poll::Level is used so mio events should be emitted as long as there is data in buffer and draining strategy should be used only in case of Poll::Edge. Currenlty we don't know why it behaves this way and this looks only like a workaround.

[Kailai-Wang]

There could be two possible reasons:

• the tungstenite read_message somehow reads all messages in one go and/or clear the socket reading state, so that mio thinks there's nothing left
• the re-register of mio is buggy (delay? race condition?) and doesn't trigger events even if there're still remaining data that were sent quickly after the first message

Either way - can this be reproduced?
If so we could try to dig further and find out when the first message is handled, if there's anything left in the socket just before re-registering

[Kailai-Wang]

Had a discussion with Kasper - we'll keep this workaround only for bitacross-worker.

Root cause is not exaclty known but it's likely the old mio crate is buggy - we see messages being kept in the read buffer in the socket but no mio event is triggered.