New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Uploader refactoring and additional attempts for immediate uploaders #1724

Merged
merged 6 commits into from Feb 26, 2018

Conversation

Projects
None yet
2 participants
@ssalinas
Member

ssalinas commented Feb 20, 2018

@darcatron I'm folding #1714 into this as well to avoid too many merge conflicts. Will close the other when they are both in hs_qa

There were a few cases where an immediate uploader could miss files. Particularly, if an uploader was immediate, but then something attempted to recreate it (wrote the file again), before the original one expired and was removed. This updates the uploader driver to:

  • Trigger additional immediately upload attempts when the metadata is written to
  • Check for any remaining files matching the glob on immediate uploader expiration, uploading them if some are found
  • Refactor a number of things to java8-ify them and store immediate uploaders in a way that better matches regular uploaders

ssalinas added some commits Feb 20, 2018

@darcatron

This comment has been minimized.

Show comment
Hide comment
@darcatron

darcatron Feb 20, 2018

Contributor

lgtm 🚢

Contributor

darcatron commented Feb 20, 2018

lgtm 🚢

@ssalinas ssalinas referenced this pull request Feb 20, 2018

Closed

removed unused path set #1714

@ssalinas ssalinas added this to the 0.20.0 milestone Feb 20, 2018

@ssalinas ssalinas added the hs_stable label Feb 21, 2018

@ssalinas ssalinas merged commit 48f5435 into master Feb 26, 2018

2 checks passed

continuous-integration/travis-ci/pr The Travis CI build passed
Details
continuous-integration/travis-ci/push The Travis CI build passed
Details

@ssalinas ssalinas deleted the uploader_fixes branch Feb 26, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment