fix: remote timeline client shutdown trips circuit breaker #8495
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
the price to pay is one .expect() where previously we would bail into an anyhow error
We can simplify because the rationale from #5880 doesn't apply anymore. VirtualFile doesn't have transient failures. Private DM link with Joonas: https://neondb.slack.com/archives/D049K7HJ9JM/p1721836424615799
… to CompactionError::ShuttingDown This PR is the product of removing the implicit #[from] conversion from anyhow => Other
downloaded.get() can be a simple map to CompactionError::Other, no need for that refactoring This reverts commit e017aca.
3126 tests run: 3005 passed, 0 failed, 121 skipped (full report)Code coverage* (full report)
* collected from Rust tests only The comment gets automatically updated with the latest test results
6148081 at 2024-07-24T17:54:42.518Z :recycle: |
problame
requested review from
VladLazar and
jcsp
and removed request for
VladLazar
+100 |
jcsp
approved these changes
Jul 25, 2024
koivunej
reviewed
Jul 25, 2024
Comment on lines
771
to
773
| return Err(LoadError::Io( | ||
| anyhow::Error::new(e).context("open layer file"), | ||
| )); |
There was a problem hiding this comment.
This could be LoadError::Open, second one could be LoadError::Read, third could be LoadError::Corruption, none need an anyhow...
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Before this PR
1.The circuit breaker would trip on CompactionError::Shutdown. That's wrong, we want to ignore those cases.
2. remote timeline client shutdown would not be mapped to CompactionError::Shutdown in all circumstances.
We observed this in staging, see https://neondb.slack.com/archives/C033RQ5SPDH/p1721829745384449
This PR fixes (1) with a simple
matchstatement, and (2) by switching a bunch ofanyhowusage over to distinguished errors that ultimately get mapped toCompactionError::Shutdown.I removed the implicit
#[from]conversion fromanyhow::ErrortoCompactionError::Otherto discover all the places that were mapping remote timeline client shutdown toanyhow::Error.In my opinion
#[from]is an antipattern and we should avoid it, especially foranyhow::Error. If some callee is going to return anyhow, the very least the caller should to is to acknowledge, through amap_err(MyError::Other)that they're conflating different failure reasons.