This repository has been archived by the owner. It is now read-only.

draft of bootstrapping input splits, also clean up input split interface #528

Merged
merged 8 commits into from Mar 23, 2018

Conversation

Projects
None yet
1 participant
@agibsonccc
Copy link
Member

agibsonccc commented Mar 22, 2018

What changes were proposed in this pull request?

Clean up input split interface, no longer extending Writable.
Add bootstrapping concept to input splits, this allows usage of input splits
with RecordWriters.
Add Partitioner for allowing partition of writes to 1 or more files and output streams.
Allow record writers to use configurations and input splits.
(Please fill in changes proposed in this fix)

How was this patch tested?

(Please explain how this patch was tested. E.g. unit tests, integration tests, manual tests)

Please review
https://github.com/deeplearning4j/deeplearning4j/blob/master/CONTRIBUTING.md before opening a pull request.

@agibsonccc agibsonccc requested a review from AlexDBlack Mar 22, 2018

@agibsonccc agibsonccc merged commit 3678a7e into master Mar 23, 2018

1 check was pending

continuous-integration/jenkins/pr-merge This commit is being built
Details

@agibsonccc agibsonccc deleted the inputsplitpartition branch Mar 23, 2018

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.