New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BEAM-1158]HCatalog example using HadoopIputFormatIO #249

Closed
wants to merge 3 commits into
base: asf-site
from

Conversation

Projects
None yet
5 participants
@seshadri-cr

seshadri-cr commented May 23, 2017

This is an example of reading through HCatalog using HadoopInputFormatIO. Reference JIRA - https://issues.apache.org/jira/browse/BEAM-1158
Please let me know if any further details are required.

@asfbot

This comment has been minimized.

asfbot commented May 23, 2017

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/beam_PreCommit_Website_Stage/496/

Jenkins built the site at commit id aea1194 with Jekyll and staged it here. Happy reviewing.

Note that any previous site has been deleted. This staged site will be automatically deleted after its TTL expires. Push any commit to the pull request branch or re-trigger the build to get it staged again.

@aaltay

This comment has been minimized.

aaltay commented May 23, 2017

R: @ssisk

p.apply("read",
HadoopInputFormatIO.<Long, String>read()
.withConfiguration(hcatConf)
.withValueTranslation(hcatOutputValueType);

This comment has been minimized.

@iemejia

iemejia May 30, 2017

Member

I think you can remove this withValueTranslation and the function because the most important part for a user is to get the PCollection of HCatRecord, then it is up to him to decide if he maps it to strings or to his own object type.

This comment has been minimized.

@seshadri-cr

seshadri-cr May 30, 2017

makes sense, removed the valueTranslation part in the latest commit !

@asfbot

This comment has been minimized.

asfbot commented May 30, 2017

Build finished.
--none--

@asfbot

This comment has been minimized.

asfbot commented May 30, 2017

Build finished.

Jenkins built the site at commit id 192878c with Jekyll and staged it here. Happy reviewing.

Note that any previous site has been deleted. This staged site will be automatically deleted after its TTL expires. Push any commit to the pull request branch or re-trigger the build to get it staged again.

@asfbot

This comment has been minimized.

asfbot commented May 30, 2017

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/beam_PreCommit_Website_Stage/518/

Jenkins built the site at commit id 192878c with Jekyll and staged it here. Happy reviewing.

Note that any previous site has been deleted. This staged site will be automatically deleted after its TTL expires. Push any commit to the pull request branch or re-trigger the build to get it staged again.

@seshadri-cr

This comment has been minimized.

seshadri-cr commented Jun 1, 2017

@iemejia , since we have reached a consensus that HIFIO examples for datastores that do not have a native IO implementation should go to the beam-site, hope we can go ahead and merge this PR(if there are no more review comments) and subsequently close the JIRA - https://issues.apache.org/jira/browse/BEAM-1158, WDYT.

@ssisk

ssisk approved these changes Jun 5, 2017

This change is in line with the other comments, looks good to me!

@iemejia

This comment has been minimized.

Member

iemejia commented Jun 6, 2017

LGTM too, merging

@asfgit asfgit closed this in 07a32b3 Jun 6, 2017

@seshadri-cr seshadri-cr deleted the seshadri-cr:hcatalog_hifio_example branch Jun 22, 2017

robertwb pushed a commit to robertwb/incubator-beam that referenced this pull request Jun 5, 2018

melap pushed a commit to apache/beam that referenced this pull request Jun 20, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment