Fix harbinger timeout bug and _multipleFileReader docstring (resolves #953, resolves #949) #954
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
hannes-ucsc
changed the title
hannes-ucsc
changed the title
|
|
| pid = int(open(harbingerFileName).read()) | ||
| except ValueError: | ||
| # There's a chance the downloading process died before it could write | ||
| # into the harbinger file |
There was a problem hiding this comment.
Then you have to create the file atomically. How do you know that it didn't write half of the pid which would still parse as an int?
There was a problem hiding this comment.
Why is this still here? With the atomic creation you added, it is highly unlikely that you'll get a ValueError, correct? If so, I would rather let the ValueError through.
arkal
force-pushed
the
issues/953-fix-harbinger-timeout-bug
branch
from
36ee67f
to
c99eeae
Compare
|
@hannes-ucsc i implemented the suggested changes |
|
Build failure tracked as #955. |
|
@arkal, see line notes. |
resolves DataBiosphere#953 The harbinger file now stores the PID of the downloading process and other requesting jobs can handle a situation where the downloading job dies without deleting the harbinger file.
arkal
force-pushed
the
issues/953-fix-harbinger-timeout-bug
branch
from
c99eeae
to
5496501
Compare
|
@hannes-ucsc i implemented the suggested changes |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Resolves #953
Resolves #949
Harbinger timeout bug was fixed by adding logic for jobs to ensure that the process downloading the
requested file from the job store is still alive, and to handle the case where it isn't.
Docstring in AbstractCacheTest._multipleFileReader was updated to reflect the changes made to it since
the function was written.