Extremely slow periodic database operation

[gustavocoding]

I noticed blockbook will run an extremely IO intensive function periodically:

func (d *RocksDB) ComputeInternalStateColumnStats(stopCompute chan os.Signal) 

This is causing all the IO of my server to be taken, rendering all other processes unresponsive.

What is the reason for this? Is this operation just for statistics purpose, can it be made optional?

[gustavocoding]

I have tested in another faster server, and this periodic stats gathering is running for hours with no way to know when it will finish (it's still in the first table), and it will run again tomorrow since the period is roughly 24h.

My understanding is this is used only to provide info about the db columns for the internal api (*:9030)? If so, this is extremely wasteful and the user couldn't care less, so please, could you stop doing it? 😃

I'd suggest an extra startup param db_stats which is false by default and can be activated if someone needs it.

[martinboehm]

Sure, it makes sense to add the parameter that controls the stats collection. However, the parameter will specify the period of stats collection in hours with default of 24. If you set it to 0, the stats will not be collected.

Also, it is surprising to me that the stats collection is killing your server as it is done only in a single thread. We run multiple instances of Blockbook on our servers and do not notice it at all.

[gustavocoding]

Thanks for the quick fix!
I run bitcoin core and lnd in a dual core, spinning disk $100 NUC, and they run really well, until I installed blockbook, and during the stats collection period, everything would be brought to its knees, but run fine otherwise (when not collecting stats)