Fix for consensus stuck. #4307
Merged
Fix for consensus stuck. #4307
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
@Frozen could you update the PR description explaining the why, how, this PR help. |
|
I agree with @sophoah. All my tests and local deployments work well with this change but still a little confused about how this fixes the issue. |
|
Can you perhaps provide an example to reproduce the issue in localnet? You suggestion on Notion that the bug can be replicated by creating "empty blocks to send". |
|
I can't reproduce same error on localnet. |
MaxMustermann2
approved these changes
Nov 24, 2022
|
Starts new view change if block invalid. |
MaxMustermann2
approved these changes
Nov 29, 2022
Frozen
force-pushed
the
fix-consensus-stuck
branch
3 times, most recently
from
43b0b9b
to
e889fa5
Compare
Frozen
force-pushed
the
fix-consensus-stuck
branch
from
e0d7f17
to
6a78993
Compare
ONECasey
added a commit
that referenced
this pull request
Jan 10, 2023
* Rebase dev branch to current main branch (#4318) * add openssl compatibility on m2 chips using darwin (#4302) Adds support for OpenSSL on MacOS Ventura using m2 chips. * [dumpdb] ensure each cross link is dumped (#4311) * bump libp2p to version 0.24.0 and update its dependencies and relevant tests (#4315) * Removed legacy syncing peer provider. (#4260) * Removed legacy syncing peer provider. * Fix localnet. * Fix migrate version. * Rebased on main. * Fix formatting. * Remove blockchain dependency from engine. (#4310) * Consensus doesn't require anymore `Node` as a circular dependency. * Rebased upon main. * Removed engine beacon chain dependency. * Fixed nil error. * Fixed error. * bump libp2p to version 0.24.0 and update its dependencies and relevant tests * fix format, remove wrongly added configs * add back wrongly deleted comment * fix travis go checker Co-authored-by: Konstantin <355847+Frozen@users.noreply.github.com> Co-authored-by: “GheisMohammadi” <“Gheis.Mohammadi@gmail.com”> * bump libp2p to version 0.24.0 and update its dependencies and relevant tests (#4315) * Removed legacy syncing peer provider. (#4260) * Removed legacy syncing peer provider. * Fix localnet. * Fix migrate version. * Rebased on main. * Fix formatting. * Remove blockchain dependency from engine. (#4310) * Consensus doesn't require anymore `Node` as a circular dependency. * Rebased upon main. * Removed engine beacon chain dependency. * Fixed nil error. * Fixed error. * bump libp2p to version 0.24.0 and update its dependencies and relevant tests * fix format, remove wrongly added configs * add back wrongly deleted comment * fix travis go checker Co-authored-by: Konstantin <355847+Frozen@users.noreply.github.com> Co-authored-by: “GheisMohammadi” <“Gheis.Mohammadi@gmail.com”> * Fix for consensus stuck. (#4307) * Added check for block validity. * Starts new view change if block invalid. * Revert "Starts new view change if block invalid." This reverts commit e889fa5. * staged dns sync v1.0 (#4316) * staged dns sync v1.0 * enabled stream downloader for localnet * fix code review issues * remove extra lock Co-authored-by: “GheisMohammadi” <“Gheis.Mohammadi@gmail.com”> * add description for closing client and change randomize process to ma… (#4276) * add description for closing client and change randomize process to make sure only online nodes are added to sync config * fix sync test * fix legacy limitNumPeers test * add WaitForEachPeerToConnect to node configs to make parallel peer connection optional Co-authored-by: “GheisMohammadi” <“Gheis.Mohammadi@gmail.com”> * Small fixes and code cleanup for network stack. (#4320) * staged dns sync v1.0 * enabled stream downloader for localnet * fix code review issues * remove extra lock * staged dns sync v1.0 * Fixed, code clean up and other. * Fixed, code clean up and other. * Fixed, code clean up and other. * Fix config. Co-authored-by: “GheisMohammadi” <“Gheis.Mohammadi@gmail.com”> * Fix not disable cache in archival mode (#4322) * Feature registry (#4324) * Registry for services. * Test. * Reverted comment. * Fix. * Slash fix (#4284) * Implementation of new slashing rate calculation * Write tests for then new slashing rate calculation * Add engine.applySlashing tests * fix #4059 Co-authored-by: Alex Brezas <abresas@gmail.com> Co-authored-by: Dimitris Lamprinos <pkakelas@gmail.com> * Bump github.com/aws/aws-sdk-go from 1.30.1 to 1.33.0 (#4325) (#4328) Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.30.1 to 1.33.0. - [Release notes](https://github.com/aws/aws-sdk-go/releases) - [Changelog](https://github.com/aws/aws-sdk-go/blob/v1.33.0/CHANGELOG.md) - [Commits](aws/aws-sdk-go@v1.30.1...v1.33.0) --- updated-dependencies: - dependency-name: github.com/aws/aws-sdk-go dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump github.com/btcsuite/btcd from 0.21.0-beta to 0.23.2 (#4327) (#4329) Bumps [github.com/btcsuite/btcd](https://github.com/btcsuite/btcd) from 0.21.0-beta to 0.23.2. - [Release notes](https://github.com/btcsuite/btcd/releases) - [Changelog](https://github.com/btcsuite/btcd/blob/master/CHANGES) - [Commits](btcsuite/btcd@v0.21.0-beta...v0.23.2) --- updated-dependencies: - dependency-name: github.com/btcsuite/btcd dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: Max <82761650+MaxMustermann2@users.noreply.github.com> Co-authored-by: Gheis <36589218+GheisMohammadi@users.noreply.github.com> Co-authored-by: Konstantin <355847+Frozen@users.noreply.github.com> Co-authored-by: “GheisMohammadi” <“Gheis.Mohammadi@gmail.com”> Co-authored-by: Danny Willis <102543677+dannyposi@users.noreply.github.com> Co-authored-by: PeekPI <894646171@QQ.COM> Co-authored-by: Alex Brezas <abresas@gmail.com> Co-authored-by: Dimitris Lamprinos <pkakelas@gmail.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
ONECasey
added a commit
that referenced
this pull request
Jan 12, 2023
* Rebase dev branch to current main branch (#4318) * add openssl compatibility on m2 chips using darwin (#4302) Adds support for OpenSSL on MacOS Ventura using m2 chips. * [dumpdb] ensure each cross link is dumped (#4311) * bump libp2p to version 0.24.0 and update its dependencies and relevant tests (#4315) * Removed legacy syncing peer provider. (#4260) * Removed legacy syncing peer provider. * Fix localnet. * Fix migrate version. * Rebased on main. * Fix formatting. * Remove blockchain dependency from engine. (#4310) * Consensus doesn't require anymore `Node` as a circular dependency. * Rebased upon main. * Removed engine beacon chain dependency. * Fixed nil error. * Fixed error. * bump libp2p to version 0.24.0 and update its dependencies and relevant tests * fix format, remove wrongly added configs * add back wrongly deleted comment * fix travis go checker Co-authored-by: Konstantin <355847+Frozen@users.noreply.github.com> Co-authored-by: “GheisMohammadi” <“Gheis.Mohammadi@gmail.com”> * bump libp2p to version 0.24.0 and update its dependencies and relevant tests (#4315) * Removed legacy syncing peer provider. (#4260) * Removed legacy syncing peer provider. * Fix localnet. * Fix migrate version. * Rebased on main. * Fix formatting. * Remove blockchain dependency from engine. (#4310) * Consensus doesn't require anymore `Node` as a circular dependency. * Rebased upon main. * Removed engine beacon chain dependency. * Fixed nil error. * Fixed error. * bump libp2p to version 0.24.0 and update its dependencies and relevant tests * fix format, remove wrongly added configs * add back wrongly deleted comment * fix travis go checker Co-authored-by: Konstantin <355847+Frozen@users.noreply.github.com> Co-authored-by: “GheisMohammadi” <“Gheis.Mohammadi@gmail.com”> * Fix for consensus stuck. (#4307) * Added check for block validity. * Starts new view change if block invalid. * Revert "Starts new view change if block invalid." This reverts commit e889fa5. * staged dns sync v1.0 (#4316) * staged dns sync v1.0 * enabled stream downloader for localnet * fix code review issues * remove extra lock Co-authored-by: “GheisMohammadi” <“Gheis.Mohammadi@gmail.com”> * add description for closing client and change randomize process to ma… (#4276) * add description for closing client and change randomize process to make sure only online nodes are added to sync config * fix sync test * fix legacy limitNumPeers test * add WaitForEachPeerToConnect to node configs to make parallel peer connection optional Co-authored-by: “GheisMohammadi” <“Gheis.Mohammadi@gmail.com”> * Small fixes and code cleanup for network stack. (#4320) * staged dns sync v1.0 * enabled stream downloader for localnet * fix code review issues * remove extra lock * staged dns sync v1.0 * Fixed, code clean up and other. * Fixed, code clean up and other. * Fixed, code clean up and other. * Fix config. Co-authored-by: “GheisMohammadi” <“Gheis.Mohammadi@gmail.com”> * Fix not disable cache in archival mode (#4322) * Feature registry (#4324) * Registry for services. * Test. * Reverted comment. * Fix. * Slash fix (#4284) * Implementation of new slashing rate calculation * Write tests for then new slashing rate calculation * Add engine.applySlashing tests * fix #4059 Co-authored-by: Alex Brezas <abresas@gmail.com> Co-authored-by: Dimitris Lamprinos <pkakelas@gmail.com> * Bump github.com/aws/aws-sdk-go from 1.30.1 to 1.33.0 (#4325) (#4328) Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.30.1 to 1.33.0. - [Release notes](https://github.com/aws/aws-sdk-go/releases) - [Changelog](https://github.com/aws/aws-sdk-go/blob/v1.33.0/CHANGELOG.md) - [Commits](aws/aws-sdk-go@v1.30.1...v1.33.0) --- updated-dependencies: - dependency-name: github.com/aws/aws-sdk-go dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump github.com/btcsuite/btcd from 0.21.0-beta to 0.23.2 (#4327) (#4329) Bumps [github.com/btcsuite/btcd](https://github.com/btcsuite/btcd) from 0.21.0-beta to 0.23.2. - [Release notes](https://github.com/btcsuite/btcd/releases) - [Changelog](https://github.com/btcsuite/btcd/blob/master/CHANGES) - [Commits](btcsuite/btcd@v0.21.0-beta...v0.23.2) --- updated-dependencies: - dependency-name: github.com/btcsuite/btcd dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * fix epoch chain initialization issue (#4331) * Fix getting epoch number. (#4279) * feat: update dockerfile with some enhacement (#4250) * feat: update dockerfile with some enhancement * [docker] fix: update golang version Co-authored-by: MaxMustermann2 <82761650+MaxMustermann2@users.noreply.github.com> * [build] github action update (#4336) * [ops] update github action files * [ops] add debug message in github action * [ops] fix GPG action variable * [ops] fix macos-12 build * [ops] fix macos-12 build * [ops] fix macos-12 build * [ops] fix macos-12 build * [ops] fix macos-12 build * [ops] fix macos-12 build * [cmd] update year version (#4334) * chore(build): upgrade golang to 1.19 (#4335) * chore(build): upgrade golang to 1.19 * chore(build): run `go mod tidy` * chore(build): run `goimports -w -e ${file}` * chore(build): revert github ci changes * chore(build): pin golang version to 1.19.5 * chore(build): fix protoc version on gen files * chore(build): fix protoc-gen-go to v1.26.0 (#4337) Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: Max <82761650+MaxMustermann2@users.noreply.github.com> Co-authored-by: Gheis <36589218+GheisMohammadi@users.noreply.github.com> Co-authored-by: Konstantin <355847+Frozen@users.noreply.github.com> Co-authored-by: “GheisMohammadi” <“Gheis.Mohammadi@gmail.com”> Co-authored-by: Danny Willis <102543677+dannyposi@users.noreply.github.com> Co-authored-by: PeekPI <894646171@QQ.COM> Co-authored-by: Alex Brezas <abresas@gmail.com> Co-authored-by: Dimitris Lamprinos <pkakelas@gmail.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Gheis Mohammadi <Gheis.Mohammadi@gmail.com> Co-authored-by: Jason Yi <90701258+jasonyic@users.noreply.github.com> Co-authored-by: Soph <35721420+sophoah@users.noreply.github.com>
ONECasey
added a commit
that referenced
this pull request
Jan 17, 2023
* Rebase dev branch to current main branch (#4318) * add openssl compatibility on m2 chips using darwin (#4302) Adds support for OpenSSL on MacOS Ventura using m2 chips. * [dumpdb] ensure each cross link is dumped (#4311) * bump libp2p to version 0.24.0 and update its dependencies and relevant tests (#4315) * Removed legacy syncing peer provider. (#4260) * Removed legacy syncing peer provider. * Fix localnet. * Fix migrate version. * Rebased on main. * Fix formatting. * Remove blockchain dependency from engine. (#4310) * Consensus doesn't require anymore `Node` as a circular dependency. * Rebased upon main. * Removed engine beacon chain dependency. * Fixed nil error. * Fixed error. * bump libp2p to version 0.24.0 and update its dependencies and relevant tests * fix format, remove wrongly added configs * add back wrongly deleted comment * fix travis go checker Co-authored-by: Konstantin <355847+Frozen@users.noreply.github.com> Co-authored-by: “GheisMohammadi” <“Gheis.Mohammadi@gmail.com”> * bump libp2p to version 0.24.0 and update its dependencies and relevant tests (#4315) * Removed legacy syncing peer provider. (#4260) * Removed legacy syncing peer provider. * Fix localnet. * Fix migrate version. * Rebased on main. * Fix formatting. * Remove blockchain dependency from engine. (#4310) * Consensus doesn't require anymore `Node` as a circular dependency. * Rebased upon main. * Removed engine beacon chain dependency. * Fixed nil error. * Fixed error. * bump libp2p to version 0.24.0 and update its dependencies and relevant tests * fix format, remove wrongly added configs * add back wrongly deleted comment * fix travis go checker Co-authored-by: Konstantin <355847+Frozen@users.noreply.github.com> Co-authored-by: “GheisMohammadi” <“Gheis.Mohammadi@gmail.com”> * Fix for consensus stuck. (#4307) * Added check for block validity. * Starts new view change if block invalid. * Revert "Starts new view change if block invalid." This reverts commit e889fa5. * staged dns sync v1.0 (#4316) * staged dns sync v1.0 * enabled stream downloader for localnet * fix code review issues * remove extra lock Co-authored-by: “GheisMohammadi” <“Gheis.Mohammadi@gmail.com”> * add description for closing client and change randomize process to ma… (#4276) * add description for closing client and change randomize process to make sure only online nodes are added to sync config * fix sync test * fix legacy limitNumPeers test * add WaitForEachPeerToConnect to node configs to make parallel peer connection optional Co-authored-by: “GheisMohammadi” <“Gheis.Mohammadi@gmail.com”> * Small fixes and code cleanup for network stack. (#4320) * staged dns sync v1.0 * enabled stream downloader for localnet * fix code review issues * remove extra lock * staged dns sync v1.0 * Fixed, code clean up and other. * Fixed, code clean up and other. * Fixed, code clean up and other. * Fix config. Co-authored-by: “GheisMohammadi” <“Gheis.Mohammadi@gmail.com”> * Fix not disable cache in archival mode (#4322) * Feature registry (#4324) * Registry for services. * Test. * Reverted comment. * Fix. * Slash fix (#4284) * Implementation of new slashing rate calculation * Write tests for then new slashing rate calculation * Add engine.applySlashing tests * fix #4059 Co-authored-by: Alex Brezas <abresas@gmail.com> Co-authored-by: Dimitris Lamprinos <pkakelas@gmail.com> * Bump github.com/aws/aws-sdk-go from 1.30.1 to 1.33.0 (#4325) (#4328) Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.30.1 to 1.33.0. - [Release notes](https://github.com/aws/aws-sdk-go/releases) - [Changelog](https://github.com/aws/aws-sdk-go/blob/v1.33.0/CHANGELOG.md) - [Commits](aws/aws-sdk-go@v1.30.1...v1.33.0) --- updated-dependencies: - dependency-name: github.com/aws/aws-sdk-go dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump github.com/btcsuite/btcd from 0.21.0-beta to 0.23.2 (#4327) (#4329) Bumps [github.com/btcsuite/btcd](https://github.com/btcsuite/btcd) from 0.21.0-beta to 0.23.2. - [Release notes](https://github.com/btcsuite/btcd/releases) - [Changelog](https://github.com/btcsuite/btcd/blob/master/CHANGES) - [Commits](btcsuite/btcd@v0.21.0-beta...v0.23.2) --- updated-dependencies: - dependency-name: github.com/btcsuite/btcd dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * fix epoch chain initialization issue (#4331) * Fix getting epoch number. (#4279) * feat: update dockerfile with some enhacement (#4250) * feat: update dockerfile with some enhancement * [docker] fix: update golang version Co-authored-by: MaxMustermann2 <82761650+MaxMustermann2@users.noreply.github.com> * [build] github action update (#4336) * [ops] update github action files * [ops] add debug message in github action * [ops] fix GPG action variable * [ops] fix macos-12 build * [ops] fix macos-12 build * [ops] fix macos-12 build * [ops] fix macos-12 build * [ops] fix macos-12 build * [ops] fix macos-12 build * [cmd] update year version (#4334) * chore(build): upgrade golang to 1.19 (#4335) * chore(build): upgrade golang to 1.19 * chore(build): run `go mod tidy` * chore(build): run `goimports -w -e ${file}` * chore(build): revert github ci changes * chore(build): pin golang version to 1.19.5 * chore(build): fix protoc version on gen files * chore(build): fix protoc-gen-go to v1.26.0 (#4337) * fix config migration issue (#4338) Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: Max <82761650+MaxMustermann2@users.noreply.github.com> Co-authored-by: Gheis <36589218+GheisMohammadi@users.noreply.github.com> Co-authored-by: Konstantin <355847+Frozen@users.noreply.github.com> Co-authored-by: “GheisMohammadi” <“Gheis.Mohammadi@gmail.com”> Co-authored-by: Danny Willis <102543677+dannyposi@users.noreply.github.com> Co-authored-by: PeekPI <894646171@QQ.COM> Co-authored-by: Alex Brezas <abresas@gmail.com> Co-authored-by: Dimitris Lamprinos <pkakelas@gmail.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Gheis Mohammadi <Gheis.Mohammadi@gmail.com> Co-authored-by: Jason Yi <90701258+jasonyic@users.noreply.github.com> Co-authored-by: Soph <35721420+sophoah@users.noreply.github.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
We can see that error happened on
onNewView:{"level":"warn","myBlock":33980416,"myViewID":33984024,"phase":"Commit","mode":"ViewChanging","error":"Block verification failed: crosslink already exist","caller":"/home/sp/harmony/harmony/consen sus/view_change.go:496","time":"2022-11-15T05:44:05.852088781Z","message":"[onNewView] Verify New View Msg Failed"}onNewViewis final step of view change mechanism, and block is already invalid. Previous step isstartNewView, so i added check there for no unchecked blocks sending.