Retrieval Tool: Indicates "complete" before all transfers finish #185
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
dbernstein
requested changes
Oct 24, 2023
| } catch (InterruptedException e) { | ||
| // Exit wait on interruption | ||
| } | ||
| phaser.arriveAndAwaitAdvance(); |
There was a problem hiding this comment.
If you wouldn't mind adding a log message before arriveAndAwaitAdvance indicating that the application is waiting for the last retrievals to complete (and that it may take a few minutes depending on the size of the retrieval and the connection speed), I think that would be an improvment..
|
Spaces can be very large (millions of items) so preventing growing list of futures is a good idea. As long as this change allows parallel downloads I think this is the right solution. |
|
I gave this a test: it works great. Thanks @mikejritter |
dbernstein
approved these changes
Oct 26, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
JIRA Ticket: https://duracloud.atlassian.net/browse/DURACLOUD-1069
What does this Pull Request do?
This updates the shutdown logic of the RetrievalManager to use a phaser to track work completion instead of using awaitTermination.
How should this be tested?
Additional Notes:
I was considering using
submitto return a future and do a get on each to emulate joining the threads, but wasn't sure what the typical space would look like and didn't want a growing list of futures. Not a big deal just an alternate solution which might be a little bit simpler.Interested parties
Tag (@ mention) interested parties or, if unsure, @duracloud/committers