Summary of Upcoming Breaking Changes

[ebuchman]

Proposed summary of impending breaking changes.

The goal is to get as many of the breaking data structure changes into the next breaking release as possible :)

• Change all RIPEMD160 to SHA256[:20]

  • Tx and Validator hashing - types: update tx hash to use SHA256 and no amino #1547, WIP Bucky/hash #1606
  • IAVL (doesn't effect Tendermint)
  • ED25519 address generation - PR update ed25519 address scheme go-crypto#112

• Don't Amino encode basic byte arrays before hashing

  • Txs - types: update tx hash to use SHA256 and no amino #1547
  • pubkeys when computing addresses - tendermint/go-crypto/pubkey based p2p handshake because ips are flaky #103, PR update ed25519 address scheme go-crypto#112

• Proposals/Votes - consensus: spec and impl for BFT time #1146, WIP Jae/bft time #1615

  • don't sign Timestamp
  • instead, we can do BFT time just through the proposer Jae/bft time #1615

• Header

  • Add proposer (and round?) - Add proposer to header #1134
  • Add tags to results hash - Add events to results hash #1007
  • NextValSetHash - NextValidatorSetHash #1313, WIP WIP: Delaying a validator set update by 1 block. #1638

• ABCI

  • AbsentValidators should be a list of addresses, not indices
  • Update header
  • Include Round number in BeginBlock (possibly in header as "LastCommitRound" ? )
  • Begin/EndBlock tags - Add tags to Begin/Endblock and support subscriptions to them #915
  • ResponseInitChain takes GenesisBytes instead of AppStateBytes - Extend RequestInitChain to contain the entire genesis.json file abci#216

• Validator set changes

  • Validators only change at H+2 - NextValidatorSetHash #1313 , WIP WIP: Delaying a validator set update by 1 block. #1638

• P2P

  • Update SecretConnection - Improve SecretConnection/STS KDF? #1165, Use ChaCha20Poly1305 instead of Salsa20Poly1305 for SecretConnection? #1124, WIP BREAKING CHANGE Replaces hash based key derivation with hkdf  #1563
  • Add ValidatorAddress to NodeInfo - p2p: add UnverifiedValidatorAddress to the NodeInfo #1516, WIP (BREAKING) p2p: add UnverifiedValidatorAddress to the NodeInfo #1604
  • pubkey -> id in p2p query

[ebuchman]

Note also from #1502:

• SignatureKey -> Signature in amino - SignatureKey -> Sig go-crypto#84
• Reconcile Query requirements

[ebuchman]

Another point still up for debate is what to do about pubkeys in the ABCI (ie. whether or not to involve Amino) - #1524

[ebuchman]

Maybe change LastBlockID to LastBlockHash in the header? #1605

[ebuchman]

Ideally the encoding of the header would be compatible with a protobuf encoding - #1605 (comment)

Would be nice to not build these SimpleMap structures ... just noticed the discrepency between how we do it for Header and for ConsensusParams ...

[cwgoes]

Can we include ABCI evidence updates - tendermint/abci#243 - and the associated Tendermint changes - #1582 - in this changeset?

edit: Also inclusion of both present and absent validators, and all validator data, in BeginBlock per SDK meeting (with future optimizations possible).

[jaekwon]

While we can have both present and absent validators in BeginBlock, it should be noted that validator updates apply to the next validator set, not the one received via BeginBlock. This means that ABCI developers should take note to persist the validator set somewhere...

Which kind of defeats the point of using addresses vs indices, imo, since it's easy enough to sort a validator set once you have one. But that's fine. I just prefer not to send both validator sets to the app via ABCI, because that's "helping" the ABCI developer a little too much to the detriment of performance.

[ebuchman]

More changes discussed:

• version in header
• merkle tree of tags for efficient proofs of inclusion/exclusion of tags in a block (rather than putting the tags in the result merkle tree) - see Add events to results hash #1007

[ebuchman]

Which kind of defeats the point of using addresses vs indices, imo, since it's easy enough to sort a validator set once you have one. But that's fine. I just prefer not to send both validator sets to the app via ABCI, because that's "helping" the ABCI developer a little too much to the detriment of performance.

This might turn out to be really unfortunate for devs ... hmm

[ebuchman]

Summary of state of this:

ABCI

• Alot of this was released in v0.11
• We still need to:
  • make the Header match the Tendermint header
  • dont send the pubkey in RequestBeginBlock (see updated ADR Bucky/adrs #1711)

Everything else

Everything else is basically in open PRs.

More

The last thing to work on is how we deal with versioning to facilitate upgrades - both on the blockchain datastructures and in the p2p layer.

I think we need to introduce versions for both these, but not exactly sure how. We should prepare an ADR for how to do this.

[ebuchman]

Everything in here is either complete or summarized in open issues that are well tagged.

[ttmc]

@ebuchman I realize you closed this issue, but we were using it to get a sense of how many breaking changes we can expect in the future. Could you check the boxes beside the things that are now done and merged? It would also be helpful to check the boxes beside the things that won't ever be done.

[ebuchman]

Updated. We're now using the "breaking" tag: https://github.com/tendermint/tendermint/issues?q=is%3Aopen+is%3Aissue+label%3Abreaking

[ttmc]

Thanks!