Skip to content

Commit

Permalink
Code review fix - generalize to HCatRecord instead of DefaultHCatRecord
Browse files Browse the repository at this point in the history
  • Loading branch information
seshadri-cr committed May 24, 2017
1 parent aea1194 commit 0cee404
Showing 1 changed file with 4 additions and 4 deletions.
8 changes: 4 additions & 4 deletions src/documentation/io/built-in-hadoop.md
Original file line number Diff line number Diff line change
Expand Up @@ -204,7 +204,7 @@ To read data using HCatalog, use `org.apache.hive.hcatalog.mapreduce.HCatInputFo
Configuration hcatConf = new Configuration();
hcatConf.setClass("mapreduce.job.inputformat.class", HCatInputFormat.class, InputFormat.class);
hcatConf.setClass("key.class", LongWritable.class, Object.class);
hcatConf.setClass("value.class", DefaultHCatRecord.class, Object.class);
hcatConf.setClass("value.class", HCatRecord.class, Object.class);
hcatConf.set("hive.metastore.uris", "thrift://metastore-host:port");
org.apache.hive.hcatalog.mapreduce.HCatInputFormat.setInput(hcatConf, "my_database", "my_table", "my_filter");
Expand All @@ -228,12 +228,12 @@ PCollection<KV<Long, String>> hcatData =
# The Beam SDK for Python does not support Hadoop InputFormat IO.
```
The `HCatInputFormat` key class is `java.lang.Long` `Long`, which has a Beam `Coder`. The `HCatInputFormat` value class is `org.apache.hive.hcatalog.data.DefaultHCatRecord` `DefaultHCatRecord`, which does not have a Beam `Coder`. Rather than write a new coder, you can provide your own translation method, as follows:
The `HCatInputFormat` key class is `java.lang.Long` `Long`, which has a Beam `Coder`. The `HCatInputFormat` value class is `org.apache.hive.hcatalog.data.HCatRecord` `HCatRecord`, which does not have a Beam `Coder`. Rather than write a new coder, you can provide your own translation method, as follows:
```java
SimpleFunction<DefaultHCatRecord, String> hcatOutputValueType = SimpleFunction<DefaultHCatRecord, String>()
SimpleFunction<HCatRecord, String> hcatOutputValueType = SimpleFunction<HCatRecord, String>()
{
public String apply(DefaultHCatRecord record) {
public String apply(HCatRecord record) {
return record.toString();
}
};
Expand Down

0 comments on commit 0cee404

Please sign in to comment.