Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

every other block missing in chain #10954

Open
Tracked by #46
wacban opened this issue Apr 5, 2024 · 2 comments
Open
Tracked by #46

every other block missing in chain #10954

wacban opened this issue Apr 5, 2024 · 2 comments
Labels
A-stateless-validation Area: stateless validation Near Core

Comments

@wacban
Copy link
Contributor

wacban commented Apr 5, 2024

This issue happened in statelessnet. Every other block was missing despite seemingly enough stake being online. The issue appeared both with multiple validators and even with just a single validator. The repro steps are unclear.

There are two painful side effects of every other block missing:

  • no block can get finalized
  • chain cannot move on to the next epoch

The chain was rather troubled before the issue was triggered.

  • Validators were crashing due to bugs
  • The chain was stalled for a long time due to not enough stake online

There are not immediate next steps as statelessnet is down now. This is just for reference in case the issue happens again.

@andrei-near
Copy link
Contributor

It happened again, similar scenario: troubled network with >30% missed chunks and network being stopped for ~24h:
Next epoch - in -2761 blocks

@andrei-near
Copy link
Contributor

This seem related to #10558
near_final_block_height 119519718
near_block_height_head 119525916

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-stateless-validation Area: stateless validation Near Core
Projects
None yet
Development

No branches or pull requests

3 participants