Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added key/value mappers for WriteHdfsP #328

Merged
merged 1 commit into from Feb 28, 2017

Conversation

Projects
None yet
4 participants
@gurbuzali
Copy link
Member

commented Feb 27, 2017

No description provided.

@gurbuzali gurbuzali added this to the 0.3.1 milestone Feb 27, 2017

@gurbuzali gurbuzali self-assigned this Feb 27, 2017

@gurbuzali gurbuzali requested a review from cangencer Feb 27, 2017

@mtopolnik

This comment has been minimized.

Copy link
Contributor

commented Feb 27, 2017

Not sure if this is the direction we want to take. It's a single-purpose solution to the problem we intend to solve in general, by providing optimized one-to-one edges which would merge the processors into a single tasklet.

@cangencer

This comment has been minimized.

Copy link
Collaborator

commented Feb 27, 2017

Agree with @mtopolnik - but I think the aim is to make it immediately useful. In general, there is a need for mapping when dealing with sources and sinks and having to create yet another vertex for this purpose seems cumbersome.

* @param valueMapper mapper which can be used to map the value to another value
* @return {@link ProcessorMetaSupplier} supplier
*/
public static <T, R, K, V> ProcessorMetaSupplier writeHdfs(JobConf jobConf,

This comment has been minimized.

Copy link
@cangencer

cangencer Feb 27, 2017

Collaborator

it would be better if type parameters were named "K, V, RK, RV" or something similar.

Javadoc for params is also missing

This comment has been minimized.

Copy link
@cangencer

cangencer Feb 27, 2017

Collaborator

Javadoc should also include comments about Writable requirements in certain cases.

This comment has been minimized.

Copy link
@gurbuzali

gurbuzali Feb 27, 2017

Author Member

👍

@gurbuzali gurbuzali force-pushed the gurbuzali:write-hdfs-mapper branch from 8e5cd17 to 1c0924d Feb 27, 2017

@devOpsHazelcast

This comment has been minimized.

Copy link
Collaborator

commented Feb 27, 2017

Test PASSed.

@cangencer cangencer merged commit efca6c1 into hazelcast:master Feb 28, 2017

1 check passed

default Build finished.
Details

@cangencer cangencer added the hadoop label Mar 8, 2017

@gurbuzali gurbuzali deleted the gurbuzali:write-hdfs-mapper branch Mar 14, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.