[BEAM-1158]HCatalog example using HadoopIputFormatIO #249
Conversation
|
Refer to this link for build results (access rights to CI server needed): Jenkins built the site at commit id aea1194 with Jekyll and staged it here. Happy reviewing. Note that any previous site has been deleted. This staged site will be automatically deleted after its TTL expires. Push any commit to the pull request branch or re-trigger the build to get it staged again. |
|
R: @ssisk |
| p.apply("read", | ||
| HadoopInputFormatIO.<Long, String>read() | ||
| .withConfiguration(hcatConf) | ||
| .withValueTranslation(hcatOutputValueType); |
There was a problem hiding this comment.
I think you can remove this withValueTranslation and the function because the most important part for a user is to get the PCollection of HCatRecord, then it is up to him to decide if he maps it to strings or to his own object type.
There was a problem hiding this comment.
makes sense, removed the valueTranslation part in the latest commit !
|
Build finished. |
|
Build finished. Jenkins built the site at commit id 192878c with Jekyll and staged it here. Happy reviewing. Note that any previous site has been deleted. This staged site will be automatically deleted after its TTL expires. Push any commit to the pull request branch or re-trigger the build to get it staged again. |
|
Refer to this link for build results (access rights to CI server needed): Jenkins built the site at commit id 192878c with Jekyll and staged it here. Happy reviewing. Note that any previous site has been deleted. This staged site will be automatically deleted after its TTL expires. Push any commit to the pull request branch or re-trigger the build to get it staged again. |
|
@iemejia , since we have reached a consensus that HIFIO examples for datastores that do not have a native IO implementation should go to the beam-site, hope we can go ahead and merge this PR(if there are no more review comments) and subsequently close the JIRA - https://issues.apache.org/jira/browse/BEAM-1158, WDYT. |
|
LGTM too, merging |
This is an example of reading through HCatalog using HadoopInputFormatIO. Reference JIRA - https://issues.apache.org/jira/browse/BEAM-1158
Please let me know if any further details are required.