Skip to content

[BEAM-3268] Reshuffle filenames before returning them from WriteFilesResult#5159

Merged
jkff merged 1 commit intoapache:masterfrom
jkff:reshuffle-filenames
Apr 20, 2018
Merged

[BEAM-3268] Reshuffle filenames before returning them from WriteFilesResult#5159
jkff merged 1 commit intoapache:masterfrom
jkff:reshuffle-filenames

Conversation

@jkff
Copy link
Contributor

@jkff jkff commented Apr 17, 2018

@jkff
Copy link
Contributor Author

jkff commented Apr 18, 2018

retest this please

Copy link
Contributor

@lgajowy lgajowy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this PR! I added my suggestions.

.withPrefix("birds")
.withSuffix(".xml"))
.getPerDestinationOutputFilenames()
.apply("Prevent fusion", Reshuffle.viaRandomKey())
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please delete the unused Reshuffle's import (checkstyle fails).

// Verify that files exist.
for (String filename : values) {
FileSystems.match(filename, EmptyMatchTreatment.DISALLOW);
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wouldn't it be better to have it tested in WriteFilesTest instead of TextIOWriteTest?

@jkff jkff force-pushed the reshuffle-filenames branch from 0e2cf38 to 8bd2035 Compare April 18, 2018 22:16
@jkff
Copy link
Contributor Author

jkff commented Apr 18, 2018

Thanks Lukasz! I deleted the import and moved verification of file existence into WriteFilesTest (and removed from TextIOWriteTest as it's no longer responsibility of that test to test this)

@lgajowy
Copy link
Contributor

lgajowy commented Apr 19, 2018

retest this please

@jkff
Copy link
Contributor Author

jkff commented Apr 19, 2018

OK seems tests went through this time.

Copy link
Contributor

@lgajowy lgajowy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks!

@lgajowy
Copy link
Contributor

lgajowy commented Apr 20, 2018

(I have no write access so I cannot merge this :) )

@jkff jkff merged commit 468b3c7 into apache:master Apr 20, 2018
@jkff jkff deleted the reshuffle-filenames branch April 20, 2018 18:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants