Join mainnet fail with panic "Wrong Block.Header.AppHash"

[jaybxyz]

Summary of Bug

This issue is the same issue as #4254.

ERROR: Error during handshake: Error on replay: Wrong Block.Header.AppHash.  Expected 1D7585E157CDA8EC6801BEF7CC41D21741520C924CA1F86CBE1CD105881F7A05, got 574DC85FED13F0146D33D4A807C7A6D5C96F1FCAB87E9EE5237688DE85030F4A

Version

$ gaiad version --long
cosmos-sdk: 0.35.0
git commit: 028bbef48d093242a2b9adb9bab34eecc97bf3fe
vendor hash: f60176672270c09455c01e9d880079ba36130df4f5cd89df58b6701f50b13aad
build tags: netgo ledger
go version go1.12.9 linux/amd64

Steps to Reproduce

Two of our public nodes are down due to the following error today.

Following seed nodes are the ones that two of our public nodes used when they are down

seeds = "ba3bacc714817218562f743178228f23678b2873@public-seed-node.cosmoshub.certus.one:26656, 2626942148fd39830cb7a3acccb235fab0332d86@173.212.199.36:26656"

Configured these seed nodes that @alexanderbez suggested to try in this #4254 to see if it works, but both of our public nodes stopped at height=16590.

seeds = "3e16af0cead27979e1fc3dac57d03df3c7a77acc@3.87.179.235:26656,2626942148fd39830cb7a3acccb235fab0332d86@173.212.199.36:26656,3028c6ee9be21f0d34be3e97a59b093e15ec0658@91.205.173.168:26656,89e4b72625c0a13d6f62e3cd9d40bfc444cbfa77@34.65.6.52:26656"

---

For Admin Use

• Not duplicate issue
• Appropriate labels applied
• Appropriate contributors tagged
• Contributor assigned/self-assigned

[tac0turtle]

you have to use cosmos-sdk version 0.34.7. 0.35 was a downstream fix, not usable on mainnet

[jaybxyz]

@marbar3778 Let me try with Cosmos SDK version v0.34.7 with the same seed nodes.

[alexanderbez]

Closing. Please reopen if you see issues @kogisin