Expose proposer selection over ABCI with new field in ResponseEndBlock

[ebuchman]

We have recently done a lot of work on getting the proposer selection algorithm right in tendermint. See for instance tendermint/tendermint#3049, tendermint/tendermint#3140, tendermint/tendermint#3222. There are still some open issues and a new proposer-selection label to tag them, but it seems we've addressed most of the concerns and have settled on a reasonably fair algorithm that can handle large validator set changes. Great work team!

One thing we have discussed is exposing the proposer selection to the applications. Currently, the only control apps have is to change the validator set, but otherwise all selection is left to Tendermint.

We should consider adding a new field to ResponseEndBlock which gives applications more control over proposer selection. This could be a list of {PubKey, Priority}, for instance, that would otherwise use the same Tendermint algorithm, but where the app specifies with every block who can be a proposer for the next block and what their respective initial priorities are.

This will need more design work and an ADR, and we need to specify when and how the default behaviour kicks in. But otherwise could be quite valuable to applications, and would for instance allow them to implement randomness in their proposer selection before we do it in Tendermint (#763), and would allow for proposers who aren't even in the validator set, which might be useful for certain configurations.

[hxrts]

linking to the Tendermint issue mentioned above for discoverability https://github.com/tendermint/tendermint/issues/763

[hxrts]

also related f-o-a-m/kepler#123

[cmwaters]

I think it's better that Tendermint implements one good protocol for leader election than allow a way for the application to control it. For me this is an abstraction that lies completely on the Tendermint side of the fence. I'm going to close this but speak up if you disagree