[FLINK-7056][blob] add API to allow job-related BLOBs to be stored #4237
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This was referenced Jun 30, 2017
added 4 commits
July 14, 2017 14:49
This lets BlobClientSslTest extend BlobClientTest as most of its implementation came from there and was simply copied.
This replaces the use of some temporary directory where it is not guaranteed that it will be deleted after the test.
This was only used in tests where it is avoidable but if used anywhere else, it may have caused cleanup issues.
NicoK
force-pushed
the
flink-6916-7056
branch
2 times, most recently
from
f76ba38
to
24bdfce
Compare
added 5 commits
July 16, 2017 23:13
The fact that we always returned URL objects is a relic of the BlobServer's only use for URLClassLoader. Since we'd like to extend its use, returning File objects instead is more generic.
For a cleaner API, instead of having a nullable jobId parameter, use two methods: one for job-related BLOBs, another for job-unrelated ones.
tillrohrmann
approved these changes
Aug 11, 2017
There was a problem hiding this comment.
Thanks for your contributions @NicoK. Changes look good to me. I've rebased it and once Travis gives green light, I'll merge it.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
To ease cleanup, we will make job-related BLOBs be reflected in the blob storage so that they may be removed along with the job. This adds the
jobIdto many methods similar to the previous code from theNAME_ADDRESSABLEmode. This job-based BLOB mode will become the future default for JARs,TaskDeploymentDescriptordata, andDistributedCachedata.Please note that we will rework/reorganise the whole BLOB store APIs completely in a future request. This PR focusses on the added functionality.
This PR is based upon #4236 in a series to implement FLINK-6916.