Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BEAM-951] FileBasedSink: merge FileOperations into IOChannelFactory. #1329

Closed
wants to merge 2 commits into from

Conversation

peihe
Copy link
Contributor

@peihe peihe commented Nov 10, 2016

Be sure to do all of the following to help us incorporate your contribution
quickly and easily:

  • Make sure the PR title is formatted like:
    [BEAM-<Jira issue #>] Description of pull request
  • Make sure tests pass via mvn clean verify. (Even better, enable
    Travis-CI on your fork and ensure the whole test matrix passes).
  • Replace <Jira issue #> in the title with the actual Jira issue
    number, if there is one.
  • If this contribution is large, please file an Apache
    Individual Contributor License Agreement.

@peihe
Copy link
Contributor Author

peihe commented Nov 10, 2016

R: @tgroh

@peihe
Copy link
Contributor Author

peihe commented Nov 10, 2016

needs some refactoring to resolve conflicts

@dhalperi
Copy link
Contributor

Ack. tgroh@ and davorbonaci@ can review in my absence, once you're ready.

R: -@dhalperi @davorbonaci

@peihe peihe force-pushed the rm-file-operations branch 2 times, most recently from fb1433d to 6ef4c3c Compare November 15, 2016 01:42
@peihe
Copy link
Contributor Author

peihe commented Nov 15, 2016

PTAL

PR synced

}
}

private void copyOne(String source, String destination) throws IOException {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can this be inlined?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

private void copyOne(String source, String destination) throws IOException {
try {
// Copy the source file, replacing the existing destination.
// Paths.get(x) will not work on win cause of the ":" after the drive letter
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"on Windows OSes"?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

StandardCopyOption.REPLACE_EXISTING);
} catch (NoSuchFileException e) {
LOG.debug("{} does not exist.", source);
// Suppress exception if file does not exist.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems as though it will be surprising.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree.
However, improving the IOChannelFactory interface is out of the scope of the PR.

*/
@Deprecated
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems @Experimental rather than @Deprecated

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Experimental only means no compatibility guarantees, and sdk/util already suggests that.
I think we also want to discourage people from using it, since we are deprecating it right now.

Copy link
Contributor Author

@peihe peihe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PTAL

}
}

private void copyOne(String source, String destination) throws IOException {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

private void copyOne(String source, String destination) throws IOException {
try {
// Copy the source file, replacing the existing destination.
// Paths.get(x) will not work on win cause of the ":" after the drive letter
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

StandardCopyOption.REPLACE_EXISTING);
} catch (NoSuchFileException e) {
LOG.debug("{} does not exist.", source);
// Suppress exception if file does not exist.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree.
However, improving the IOChannelFactory interface is out of the scope of the PR.

*/
@Deprecated
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Experimental only means no compatibility guarantees, and sdk/util already suggests that.
I think we also want to discourage people from using it, since we are deprecating it right now.

@peihe
Copy link
Contributor Author

peihe commented Nov 16, 2016

rebased for Jenkins license issue

@tgroh
Copy link
Member

tgroh commented Nov 16, 2016

LGTM

@davorbonaci if you could take a look I'd appreciate it.

@davorbonaci
Copy link
Member

LGTM

@asfgit asfgit closed this in 1543ea9 Nov 17, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants