SymbolOrderBook connection unstable when subscribing to many Symbols

[PsykotropyK]

Describe the bug

1. when subscribing to 1 Symbol, the OrderBook sync easily and remain synced without any reconnection attemps
2. when I add some symbols (let's say 10) I have :

• if I use default SocketClient (i.e. not specifying any option in BitfinexSymbolOrderBook) connection can be very unstable on some Symbols, having many OnStatusChange events (OrderBookStatus.Reconnecting <> OrderBookStatus.Synced). On the other hand some Symbol are synced and stable
• if i us a SocketClient shared among all instances of BitfinexSymbolOrderBook the stream seems to pause at the first reconnection attempts, i.e. some Symbols go from "Synced" to "Reconnecting" and then nothing else happens for a while and suddenly I have a few updates (i.e. some goes from reconnecting to sync other the other way, with very few OrderBook having received new updates

3. if I subscribe to many symbols (100) I have :

• with default SocketClient : only ~15 Symbols manage to sync, with the same instability as described above
• with shared SocketClient, 15/20% remains under "Connecting" status, most are always under "Reconnecting". The Synced one got order book updates. Also, on some run, everything stops (i.e. no more book updates, nor status updates) without anything logged nor exception raised

Moreover, when trying to connect, the Symbols goes from "Disconnected" to "Connecting" one after another, and hangs at "Connecting" until they are all under this state (which takes a while for 100)

To Reproduce
Client : Client = new BitfinexClient(new BitfinexClientOptions() { LogLevel = LogLevel.Debug, ApiCredentials = new ApiCredentials("key", "secret") });
Socket : SocketClient = new BitfinexSocketClient();
OrderBookOptions : Options = new BitfinexOrderBookOptions(SocketClient);
OrderBook

• new BitfinexSymbolOrderBook(symbol.Name, Precision.PrecisionLevel0, 25);
• new BitfinexSymbolOrderBook(symbol.Name, Precision.PrecisionLevel0, 25, Options);

I monitor the behavior by bindgin a custom class to a ListBox having 3 properties bound: Name (name of the Symbol) Status (connection status) and Count (number of time OnOrderBookUpdate was triggered).

See the attached Project for the complete implementation
ProjectTester.zip

Expected behavior
Lack of stability may not come from your code but from the API, though it should be checked
Socket should be able to resync
If using the default SocketClient is not recommended, defaulting it should not be possible

Debug logging
The log (Trace) with 100 symbols and a single SocketClient shared is attached as .txt.
Some stats on the log:
Out of 814 log lines :

• 415 Warnings (251 out of sync, 99 checksum error, 65 connection lost)
• 398 Info (linked to connection and status updates)
  No apparent errors

logs.txt

[JKorf]

I found an issue with the checksum calculation. When calculating our local checksum a small decimal like 0.0000012 is written as 1.2E-06, which makes the checksum string invalid. I'm looking into a solution for this, decimal handling in Json.Net and json in general is a bit of a pain

[PsykotropyK]

If I follow you, the checksum issue makes that it thinks the order book is unsynced, and forces to resync. That would explain the lack of stability, but not why it can't resync. Or is it then on the broker's side that the high amount of request bans my ip?

[PsykotropyK]

@JKorf I understand that the values used in the checksum comes from the OrderBookEntryConverter class. When you return BitfinexOrderBookEntry maybe use ((decimal)price).ToString(), instead of price.Value<string>() and ((decimal)quantity).ToString(), instead of quantity.Value<string>() as the decimal should not have the scientific notation.

Else, maybe using something like String.Format("{0:F20}", [value]); can help?
As seen here

[JKorf]

Hi, I've done a fair amount of testing and released a new version with various fixes for the checksum validation, and also improved stability/speed on the socket connections. Can you have another go with the latest version?

[PsykotropyK]

I updated and run, the result is great. I'll test more in depth later but running it shows as expected results. 96 out of 100 are connected and synced, remaining 4 stay disconnected. But that's not an issue, I'll have to check them specifically (as it's always the same that remain disconnected), it can be due to many things.

Also I see great improvement in the time it takes to go from disconnected --> connecting --> synced. It used to take a while for 100 symbol before, now it's a matter of seconds.