ABCI ++ tracking issue

[tessr]

The upcoming ABCI++ work is currently scattered across teams, repos, etc. It will probably continue to be fairly disperse, but here's a tracking issue anyways.

Original RFC: tendermint/spec#254

UPDATE:
We have shifted to using this project board to divide and track remaining work for implementing ABCI++:
https://github.com/orgs/tendermint/projects/9

[ebuchman]

I just want to highlight that it seems the three components of ABCI++ (PrepareProposal, ProcessProposal, and VoteExtensions) are relatively independent and could and maybe should all be implemented independently, as they each unblock other kinds of work and I'm not sure if there's a good reason why they must all come together.

I would think that VoteExtensions is the most straight forward and should just proceed, as it will then unblock things like aggregate signatures and native oracles.

Probably next would be PrepareProposal, which should also be relatively straightforward, but for it to actually be useful will require significant lift from the application side to handle it meaningfully.

Finally ProcessProposal, which will introduce more latency into the consensus (executing blocks before prevoting) or else has some subtle impacts on correctness if we want to run that concurrently with voting (see https://github.com/tendermint/spec/pull/283/files#r628424695).

@tessr @ValarDragon does that make sense ?

[ValarDragon]

I think VoteExtensions is the least straightforward personally, since it alters many core Tendermint data structures. (Also its orthogonal to aggregating signatures, the state machine wouldn't be the one doing that aggregation!) Agreed that it would allow for validator-based price oracles, if your view is that it would be the most immediate demand happy to switch focus to that.

I was currently working on a PR for process proposal with everything in sync. It on its own doesn't require executing the block prior to consensus, but does move when you do the communication between state machine and application for communicating the block data.

Agreed with PrepareProposal being relatively straightforward. One downside is that I don't think we should merge it until after vote extensions are down though, since we also want to allow for prepares of vote extensions and not immediately break the API.

Happy to switch focus to vote extensions though!

[ebuchman]

Right isn't Vote Extensions the main breaking changes in Tendermint data structures? Need to review the details again as to why it's less straight forward but seemed it was more a data structure issue vs more complex interactions with the app.

I don't have strong preference on the order just wanted to clarify that we could do the pieces relatively independently and get some clarity on what the current perception of the ordering was going to be.

It on its own doesn't require executing the block prior to consensus, but does move when you do the communication between state machine and application for communicating the block data.

I didn't understand this, can you clarify?

[ValarDragon]

Yeah! ProcessProposal gives the application the block data at prevote time, and the application then can optionally do some validity checks before returning. I just meant that process proposal on its own doesn't enforce you execute the block then, and is useful with very quick (single-pass) checks on the data as well. (such as checking that every tx is well formed)

Agreed that ProcessProposal and VoteExtensions can be done in parallel. PrepareProposal can be done in parallel, but would need adapting again after VoteExtensions

[ebuchman]

Ah yes of course whether the full block is executed is up to the app during ProcessProposal.

Sounds good - sounds like we should pursue ProcessProposal and VoteExtensions and then PrepareProposal. Thanks

[cmwaters]

Created a project board to track ongoing ABCI++ work: https://github.com/orgs/tendermint/projects/9