New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[BEAM-59] Add FileSystems#matchNewResource #2728
Conversation
R: @peihe @ssisk @davorbonaci |
eb33428
to
12e6d90
Compare
Update the TODO / documentation https://github.com/apache/beam/blob/master/sdks/java/core/src/main/java/org/apache/beam/sdk/io/fs/ResourceId.java#L30 |
d0cc6ce
to
cf85719
Compare
Changes Unknown when pulling cf85719 on dhalperi:filesystem-matchnewresource into ** on apache:master**. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
The new FileSystems API needs a way to generate a ResourceId for a resource that does not exist. This does not come up in sources, because we typically are just matching existing files. However, sinks need the ability to reference a new directory (say, in order to create it). Couldn't think of anything better than a simple function that says "treat this string as a full resource path with the specified type", which is what FileSystems#matchNewResource is.
cf85719
to
f93a277
Compare
The new FileSystems API needs a way to generate a ResourceId for a
resource that does not exist. This does not come up in sources, because
we typically are just matching existing files. However, sinks need the
ability to reference a new directory (say, in order to create it).
Couldn't think of anything better than a simple function that says
"treat this string as a full resource path with the specified type",
which is what FileSystems#matchNewResource is.