[roq-core] MbP needs heuristics to determine tick_size when no reference data exists

[thraneh]

This has recently started happening for Deribit where there are new instruments (options) being published at 8am UTC. It appears market data maybe arrives before reference data.

Without reference data, the prices will be rounded or even truncated thereby potentially causing BadState (bid>=ask).

Workaround

Use --mbp_allow_price_inversion=true

[thraneh]

Now logging if no reference data when creating MbP. The pattern to look for is NO REFERENCE DATA symbol="{}"

[thraneh]

Another reason for getting bad state is that reference data can be received, but either the tick-size is missing or it's plain wrong.

Heuristics has now been added to check for a possible initial tick-size based on the prices observed in the initial snapshot.

This issue will not be closed until we have observed the behaviour for all gateways.

[thraneh]

The new logic was flawed and didn't deal with ReferenceData.tick_size potentially being NaN. This has been fixed.

[thraneh]

Today it worked with Deribit. Still needs extensive testing before we can say it's good.

[thraneh]

More changes were required due to HitBTC allowing extreme price ranges.

Short version:

• heuristics detected smaller tick-size than reported by reference data
• this caused high asks to overflow when converted into integer representation
• this was not caught at the gateway level
• these overflowing values then passed through to clients causing "BadState" exceptions

Example, tick size was detected as 1e-9 and maximum ask price was 1e15, converting to integer would require 1e24 causing overflow because int64_t has a maximum of roughly 1e19.

The solution: drop all prices causing overflow (essentially truncating the ask-side of the order book).

[thraneh]

Seems to work now. Will test for a few more days before closing this issue.

[thraneh]

Seeing issues most likely relating to heuristics working differently between gateway and client.

A current idea, and possible solution, is to have the gateway publish current price/quantity decimal digits with MbP and use those on the client side for any scaling required. This would remove any uncertainty from the client-side. (This would also remove the constraint to pass ReferenceData to MbP objects.)

[thraneh]

Seems to work now