Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add bootstrap timeout #13747

Merged
merged 7 commits into from
Aug 7, 2023
Merged

Conversation

psteckler
Copy link
Member

@psteckler psteckler commented Jul 27, 2023

Add a bootstrap timeout to the Incremental observer. After 120 minutes, crash the daemon.

Tested by

  • temporarily inverting the is_seed condition in create_sync_status_observer
  • temporarily setting the timeout to less than a second
  • running the daemon as a seed
  • temporarily not stopping the bootstrap timeout when the status changes to Synced (which is almost instantaneous in the local seed node)

Saw:

[FATAL] *** Mina daemon has been stuck in bootstrap for too long ***
*** Shutting down ***

Closes #13728.

@psteckler psteckler requested a review from a team as a code owner July 27, 2023 01:00
@psteckler
Copy link
Member Author

!c--build-me

@psteckler
Copy link
Member Author

!ci-build-me

@psteckler
Copy link
Member Author

!ci-build-me

@psteckler
Copy link
Member Author

!ci-build-me

1 similar comment
@psteckler
Copy link
Member Author

!ci-build-me

| None ->
()
in
let handle_status_change state status =
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doesn't status here include Bootstrap based on how incremental_status is defined here ? How is state tracked?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I got fooled by the type that merlin was reporting. We only need status.

@psteckler
Copy link
Member Author

!ci-build-me

@psteckler
Copy link
Member Author

!ci-build-me

@deepthiskumar
Copy link
Member

!ci-build-me

@deepthiskumar
Copy link
Member

!approved-for-mainnet

@deepthiskumar deepthiskumar merged commit d257f0b into berkeley Aug 7, 2023
47 checks passed
@deepthiskumar deepthiskumar deleted the feature/bootstrap-stuck-timeout branch August 7, 2023 17:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants