Blockchain State to Simulation

[rigelrozanski]

debugging #3019 would be hecka' lot easier if we had a way to export a everything from a running blockchain to a json file of:

• transactions,
• proposer, and
• block signers

at each height. In addition we would of course also need the genesis file of this blockchain.

Using this information we could run a simulations which should in theory be identical to the running blockchain. From this point we could easily test out code changes to identify whether or not they resolved the issue, add additional debug points further upstream etc.

I really see this as the ultimate way we ought to be debugging problems from live chains

CC @jaekwon @cwgoes

[alexanderbez]

Would love to help out here!

[alexanderbez]

Also, is this something that could work in conjunction with the replayer @jaekwon? Is there overlap here?

[rigelrozanski]

@alexanderbez is there an issue for the replayer?

[alexanderbez]

Issue? Hmmm not sure. I just know @jaekwon has a replayer binary/tool where there might be some overlap here. Either way, I think this is a super useful export to have!

[cwgoes]

Are you referring to gaiareplay?

Hopefully that could be combined with this tool and the existing simulation code.

[alexanderbez]

Precisely!

[alexanderbez]

So I've created a (WIP) PR #3081. This will create a JSON file with the needed exported data to essentially fully replay all txs from a chain (assuming we're also given a genesis file).

So how do you guys envision the actual replaying? From what I see, there really is nothing to "simulate" unless I'm missing something...? It seems to me we should just have a simple binary/command that naively creates a Gaia app and replays all the txs from this file and a genesis file.

[cwgoes]

I wonder if we should really be doing this SDK-side. Replaying blocks and transactions seems a task best done by Tendermint, which already has the historical transaction bytes, block heights, proposers, etc etc. and existing functions for state sync / normal operation to apply transactions in blocks. Besides, this seems like a feature which would be useful for ABCI applications more generally (beyond the Cosmos SDK).

cc @rigelrozanski Do you think Tendermint could encapsulate the featureset you want or are there particular elements of the SDK / Gaia state machine which Tendermint could not easily interact with that justify duplicating parts of the Tendermint state machine in the SDK?

[rigelrozanski]

No I think we should be replaying this on SDK side, this should be a state-machine simulation not a tendermint simulation - for the simulation we should pass in the pre-commit validators from the actual execution and incorporate that into the calculation for proposer rewards etc.

It's possible that another form of Tendermint simulation should exist, or a combined simulation with tendermint and this simulator, but I think the first step is to incorporate the existing "mock-tendermint" we have in our existing simulator into this feature set.

________ edit - more thoughts (maybe reversing this above statement)

@cwgoes are you suggesting that we could use tendermint replay to act as this simulation? The whole point is we want to have control over the contents of these blocks such that these blocks, possibly with modified block transaction field values could be replayed over a modified state machine to simulate if a modification to the statemachine would have caused a blockchain with the same sequence of transactions to nolonger-fail some invariant.

As a part of this we want debugging abilities to let's say run invariants after every single message instead of just once per block - the current random simulator allows us to do this for our random scenarios. - there is also all sorts of other debugging capabilities we have in the simulator not currently available to us in the replay (if I'm not mistaken)

Here is an example:

• let's say there was a problem with how transactions were being created, first we would want to export all the transactions and blocks.
• then modify say certain transactions to have the corrected value for the transaction input.
• then rerun all these transactions over the statemachine to make sure something didn't fail.

Another example

• there is a problem with begin-block logic
• we want to rerun all the transactions with a new begin block state machine logic

etc. I think you get the drift - I get the sense that we want to be able to run things independant of just using tendermint-replay - but maybe my understanding is lacking, and more functionality is actually available to us!

[rigelrozanski]

@alexanderbez what's needs to be simulated is certain information which would normally be passed from tendermint into things like BeginBlock on the sdk side - namely precommit validators. The simulation should be running BeginBlocks and EndBlocks with appropriate transactions bundled together as if it was the real blockchain. This also means that the simulation should also be executing empty blocks (assuming the real blockchain execution contained empty blocks)

[cwgoes]

@rigelrozanski We're on the same page I think.

My question is whether we should architect a "replay mode" in Tendermint to do precisely this:

• Load a past sequence of blocks, headers (including precommits), transactions
• Offer some way to modify transactions (probably through some SDK interface, if we want to modify the txs)
• Replay all blocks from n to m through standard ABCI calls (BeginBlock, EndBlock, DeliverTx)

It doesn't need to be "Tendermint" per se, more like an "ABCI simulator" - but Tendermint might be the most natural place in our current stack to put this logic.

[alexanderbez]

I don't really follow here, but it seems you guys have a good grasp on what needs to be done. I closed #3081 as I don't think it is needed.

[jackzampolin]

I think this issue has been substantially addressed. Closing it as complete.

[rigelrozanski]

@fedekunze has this actually been addressed? I do not believe that it truly has. We cannot export the first 10K blocks of a testnet and re-run this as a simulation correct? If this does exist, I would like to see the documented re-produceable steps as to how to accomplish this

[fedekunze]

It hasn’t been addressed yet. Let’s discuss next weeks the next priorities for the simulator

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.