Store tx final application state on block

[igormcoelho]

I think it's important to persist on blocks the final state of the application transactions, for example, FAULT or HALT.
It's a single extra byte per tx, but the gains with this are huge. The capability of evolving the chain will also be larger, as past faulted/halted events will be guaranteed to stay in the same state.

As discussed on other threads, this complements audit capabilities, including MPT for storages, MPT for past transactions (including return value) and MPT for past blocks. Notifications can perhaps be kept on another MPT.

[shargon]

Agree!

[vncoelho]

We are doing a draft of this implementation, but only during export and import.
In this sense, we are not really including this state into the blocks.

[shargon]

Any advance @vncoelho ? can I help you with this ?

We have two alternatives, change public Transaction[] Transactions; to TransactionState -> With transaction and state, or create a new property VMState[] TransactionStates with the state of every TX, i think that is better the first one, because we could want to store the result, or something more in the future.

[vncoelho]

I had thought about just during export and import.
But if the idea is to store direct in the block let's proceed with it.
In the future, If the MPT provides new tools enabling it to be removed we revert it.
However, we should be aware that it would be a breaking change.

[vncoelho]

@igormcoelho, following our recent discussion, is this still being considered?

[shargon]

For me this is very important

[igormcoelho]

This is a thing we need to decide... solid states and less bug fixing possibilities, or else? 🤔
another problem is that tx results require actually executing all applications at the time block is made, or before... so, less tps too..

[Tommo-L]

Could we store the stateRoot and receiptRoot in the header, is it better than this?

[igormcoelho]

Could we store the stateRoot and receiptRoot in the header, is it better than this?

@Tommo-L we are going in a slightly different direction... it's better than stateRoot on header. I'll explain.

We have discussed in other posts, alternatives to definitive solid states on blocks, that may cause network forks when bugs are fixed. From a "code-is-law" perspective, codes and bugs are "absorbed" into protocol, but we (at least myself) are defending a "protocol-is-law" perspective, where code bugs are treated as bugs, not protocol decisions, so they need fixing without causing network forks.

If we put a stateRoot field, two bad things will happen:

1. Bugs will never be fixed, as long as they affect blockchain state (and they usually do)
2. The CN needs to execute all transaction applications (including storage updates, etc) during block proposal time, and other CN also need to execute the whole thing (to verify stateRoot), before even responding if they agree or not to the block on dBFT. This takes time, and will strongly affect high TPS capabilities of Neo.

So, this path we are proposing is very novel on blockchain world, in my opinion, and may require three things in my opinion:

1. Guarantee that applications finish execution after block is persisted, before next block proposal: Dettached Verification + Application: ensure delay is at most one block #965 (this is easy)
2. Distribute stateRoot via P2P (not on blocks), after every node finishes its execution (so that other nodes may audit their own implementations): [Neo3] Distribute state hash via p2p messages #901
3. To create a solid verification mechanism, I'm defending in this PR that we store the FAULT state on the current or next block, so that a failed tx never becomes "unfailed" in the future. I'll have to adjust the text up there, because it's slightly different now... I think that non-FAULT tx may be faulted on the future, if some bug is discovered, although this is not a desirable thing.
4. Users can defend themselves naturally, by simply adopting a Solid State Transfer NEP (even if blockchain doesn't include storage audits by state tries): Solid State Transfers proposals#97

I think we need to better discuss this, to find a "perfect" solution, because depending on next block to be sure on a FAULT state is like looking to the future, to decide the present 😂 and the future may not be available...
We can also decide to execute everything on block proposal/dBFT time, as long as we know that it's feasible and very fast on practice, for hundreds/thousands of tx. It would be nice to reduce dBFT time too, from 15 seconds to much less... 10... or 5 secs.. so this tightens even more this decision here.

What do you think @Tommo-L , is it going in a reasonable direction?

[shargon]

I think that this was done

[lock9]

We think this is done. I'm closing it but if you think this is a mistake, please reopen this issue.