Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[qob] maybe retry GSFS reads #13013

Merged
merged 1 commit into from
May 11, 2023
Merged

[qob] maybe retry GSFS reads #13013

merged 1 commit into from
May 11, 2023

Conversation

danking
Copy link
Contributor

@danking danking commented May 9, 2023

Fixes #12983


After an FSSeekableInputStream method (successfully) returns, getPosition always represents theintended location within the object. We can entirely eliminate lazyPosition because it tracks the
same value as getPosition when reader == null.

For retryable reads, we just seek back to the known correct location of the stream before attempting to read again.

After an `FSSeekableInputStream` method (successfully) returns, `getPosition` always represents the
intended location within the object. We can entirely eliminate `lazyPosition` because it tracks the
same value as `getPosition` when `reader == null`.

For retryable reads, we just seek back to the known correct location of the stream before attempting
to read again.
Copy link
Contributor

@daniel-goldstein daniel-goldstein left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like the reset on the retryTransientErrors

@danking danking merged commit 3b04c08 into hail-is:main May 11, 2023
daniel-goldstein pushed a commit to daniel-goldstein/hail that referenced this pull request May 11, 2023
Fixes hail-is#12983

---

After an `FSSeekableInputStream` method (successfully) returns,
`getPosition` always represents theintended location within the object.
We can entirely eliminate `lazyPosition` because it tracks the
same value as `getPosition` when `reader == null`.

For retryable reads, we just seek back to the known correct location of
the stream before attempting to read again.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[GoogleStorageFS] We should retry calls to the Google libraries' read
2 participants