Emerald: tokens analyzer

[pro-wh]

here's a new tokens analyzer. notes from the demo:

~70% speedup in block scanning

before

after

---

---

[pro-wh]

current state:

• trying this thing where instead of a dedicated tokens-to-query table, I'm adding a "discovery state" column to the tokens table. we can come up with a state machine like (row nonexistent) -> discovered -> info populated <-> need total supply update etc

[pro-wh]

readying this for review, there are some issues with this though

[mitjat]

Nice! It's a good amount of boilerplate to figure out; thank you!

I think this is overall good; I have a few small style/docs suggestions, and one higher-level one about focusing the analyzer on the EVM module, not Emerald.

[pro-wh]

designing new table to use three columns: first seen, last round when mutable info changed, last round when we analyzed it. diagram of the latter two columns:

source

digraph {
    start [shape=none]
    node [shape=box]
    start -> no_row
    no_row [peripheries=2]
    null_round [peripheries=2]
    round_round [peripheries=2]
    // block analyzer
    edge [color=blue]
    no_row -> null_null [label="see transfer"]
    no_row -> round_null [label="see mint"]
    null_null -> round_null [label="see mint"]
    null_round -> high_low [label="see mint"]
    round_round -> high_low [label="see mint"]
    // evm token analyzer
    edge [color=green]
    null_null -> null_round [label="download"]
    null_null -> high_low [label="stale download"]
    round_null -> high_low [label="stale download"]
    round_null -> round_round [label="download"]
    high_low -> round_round [label="download mutable"]
}

blue is what happens in the block analyzer. green is what happens in the evm token analyzer. self edges omitted.

double boxes are consistent states. following only green edges should allow the system to settle into one of these states.

"stale download" is what happens when the evm token analyzer reads one "last round when mutable info changed" into its goroutine, then the block analyzer detects something that would further change the mutable info while the evm token analyzer is downloading info for an out of date round, then the evm token analyzer saves that out of date info.

[pro-wh]

description updated. re-opening for review

[mitjat]

Looking good!
Consider this pretty much approved from my end, I'm just holding off because I'm not sure if we are on the same page re/ dead reckoning for mints and burns.

[mitjat]

🐑 it!