Fix LRU deadlock + regression test and improve comments / logging

[richardartoul]

• Fix LRU deadlock that could occur when the events channel was filled up. Achieved this by making the series object call WiredList.Update() outside of a lock as well as adding the concept of a NonBlockingUpdate. Also added a regression test.
• Made LRU events channel size configurable for stress testing purposes.
• Improve "invariant violated" style logging so that metrics are emitted if wired list invariants are violated.
• Added a new "CloseIfFromDisk" method to the database block as an additional layer of protection.

[codecov]

Codecov Report

Merging #862 into master will decrease coverage by 0.06%.
The diff coverage is 62.19%.

@@            Coverage Diff             @@
##           master     #862      +/-   ##
==========================================
- Coverage   78.43%   78.36%   -0.07%     
==========================================
  Files         391      391              
  Lines       33222    33268      +46     
==========================================
+ Hits        26058    26072      +14     
- Misses       5372     5392      +20     
- Partials     1792     1804      +12

Flag	Coverage Δ
#dbnode	81.28% <62.19%> (-0.09%)	⬇️
#m3ninx	71.99% <ø> (ø)	⬆️
#query	67.81% <ø> (-0.04%)	⬇️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 8e93c11...45cbf94. Read the comment docs.

[robskillington]

Could we refactor this to avoid multiple unlock calls? Perhaps something like:

var (
  b block.DatabaseBlock
  updateList *block.WiredList
)

s.Lock()
defer func() {
  s.Unlock()
  if b != nil && updateList != nil {
    updateList.BlockingUpdate(b) // Update outside of lock
  }
}()

[richardartoul]

oh yeah thats better thanks

[robskillington]

LGTM