Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adds more info to panic message in AccountsHashVerifier #35353

Merged
merged 1 commit into from
Feb 28, 2024

Conversation

brooksprumo
Copy link
Contributor

Problem

There have been reports on Discord where a node panics shortly after starting up with fastboot. The panic in particular is in AccountsHashVerifier, when calculating the incremental accounts hash, and it says the node doesn't have the necessary accounts hash from the base slot (i.e. full snapshot).

So far this issue has been difficult to debug, as it often depends on the state of the machine (and filesystem) when the panic occurs. If we end up in this exceptional scenario, we could log more information that could help the debugging effort.

Related to #35350 and #35190.

Summary of Changes

Adds more information to the panic message, for debugging.

@brooksprumo brooksprumo added v1.17 PRs that should be backported to v1.17 v1.18 PRs that should be backported to v1.18 labels Feb 28, 2024
@brooksprumo brooksprumo self-assigned this Feb 28, 2024
Copy link
Contributor

mergify bot commented Feb 28, 2024

Backports to the stable branch are to be avoided unless absolutely necessary for fixing bugs, security issues, and perf regressions. Changes intended for backport should be structured such that a minimum effective diff can be committed separately from any refactoring, plumbing, cleanup, etc that are not strictly necessary to achieve the goal. Any of the latter should go only into master and ride the normal stabilization schedule.

Copy link
Contributor

mergify bot commented Feb 28, 2024

Backports to the beta branch are to be avoided unless absolutely necessary for fixing bugs, security issues, and perf regressions. Changes intended for backport should be structured such that a minimum effective diff can be committed separately from any refactoring, plumbing, cleanup, etc that are not strictly necessary to achieve the goal. Any of the latter should go only into master and ride the normal stabilization schedule. Exceptions include CI/metrics changes, CLI improvements and documentation updates on a case by case basis.

@brooksprumo brooksprumo marked this pull request as ready for review February 28, 2024 20:07
@@ -7479,6 +7484,13 @@ impl AccountsDb {
.cloned()
}

/// Get all incremental accounts hashes
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure it requires any special "markup", but currently, we only plan to use this function (and the other new one) to get extra information before we panic, right? I guess it is a separate file AND crate, so these have to be pub

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, exactly.

Copy link
Contributor

@jeffwashington jeffwashington left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

Copy link

codecov bot commented Feb 28, 2024

Codecov Report

Attention: Patch coverage is 0% with 14 lines in your changes are missing coverage. Please review.

Project coverage is 81.7%. Comparing base (8ad125d) to head (dac9759).
Report is 15 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff            @@
##           master   #35353     +/-   ##
=========================================
- Coverage    81.7%    81.7%   -0.1%     
=========================================
  Files         834      834             
  Lines      224235   224248     +13     
=========================================
- Hits       183390   183371     -19     
- Misses      40845    40877     +32     

@brooksprumo brooksprumo merged commit 6aaaf85 into solana-labs:master Feb 28, 2024
37 checks passed
@brooksprumo brooksprumo deleted the fastboot/debug branch February 28, 2024 20:55
mergify bot pushed a commit that referenced this pull request Feb 28, 2024
mergify bot pushed a commit that referenced this pull request Feb 28, 2024
brooksprumo added a commit that referenced this pull request Mar 1, 2024
…ort of #35353) (#35359)

Adds more info to panic message in AccountsHashVerifier (#35353)

(cherry picked from commit 6aaaf85)

Co-authored-by: Brooks <brooks@solana.com>
brooksprumo added a commit that referenced this pull request Mar 1, 2024
…ort of #35353) (#35358)

Adds more info to panic message in AccountsHashVerifier (#35353)

(cherry picked from commit 6aaaf85)

Co-authored-by: Brooks <brooks@solana.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
v1.17 PRs that should be backported to v1.17 v1.18 PRs that should be backported to v1.18
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants