WebSocket reconnect - ws provider will miss some events.

[VladoDemcak]

Web3 v1.2.7

We are testing new web3 reconnect feature which has been added in v1.2.7. It works fine but there is one problem we have found.

Setup. WS provider with auto reconnect = true and 3sec delay.

1. We send events 1,2,3,4
2. close WS, now wait delay 3 sec for reconnect
3. Within the delay gap - let's send events 5 and 6
4. WS will reconnect after 3 sec delay and start subscription again. However we will miss events 5 and 6.

We can set small delay but I think there might be still a time gap where we can miss some events.

We tried to workaround this with keeping fromBlock once we processed an event and when ws is reconnected we recreated subscriptions. but web3's requestManager will create it's own subscriptions https://github.com/ethereum/web3.js/blob/v1.2.7/packages/web3-core-requestmanager/src/index.js#L113 and we will have 2 separated subscriptions. so we would process one event twice.

Image below is showing what was happening in the test case.

npm run server -> without WS reconnection we have all events
npm run listen -> WS has been closed and reconnected. we missed events from blocks: 3255 and 3256
npm run send -> you can see when we sent BasicEvents and compare timestamps.

The red rectangles show the "black hole".

You can use again my https://github.com/VladoDemcak/eth-subscribe-debug repository with steps how to reproduce the issue.

[cgewecke]

@VladoDemcak Ah interesting, that makes sense. Will look into this.

[VladoDemcak]

@cgewecke Thanks. I had few minutes to play with this.

Honestly I didn't have time to analyze web3-core-subscription but I wanted to try to add a checkpoint for reconnection. You can look at my PR https://github.com/ethereum/web3.js/pull/3493/files

I tried to run web3 mocha tests. Result: 2731 passing (1m) 1 pending. so maybe the "fix" caused the test failure.
if u will need any help please let me know. fingers crossed.

[Welziewagers2015]

I ran it a few times myself passing on my end.

…

On Tue, Apr 28, 2020, 4:23 PM VladoDemcak ***@***.***> wrote: @cgewecke <https://github.com/cgewecke> Thanks. I had few minutes to play with this. Honestly I didn't have time to analyze web3-core-subscription but I wanted to try to add a checkpoint for reconnection. You can look at my PR https://github.com/ethereum/web3.js/pull/3493/files I tried to run web3 mocha tests. Result: 2731 passing (1m) 1 pending. so maybe the "fix" caused the test failure. if u will need any help please let me know. fingers crossed. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#3491 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AIP3VW3ZWDK4FPKM2DE6PGDRO43LBANCNFSM4MSYAJ6Q> .