Skip to content

Commit

Permalink
Removed valueTranslation part
Browse files Browse the repository at this point in the history
  • Loading branch information
seshadri-cr committed May 30, 2017
1 parent 0cee404 commit 192878c
Showing 1 changed file with 4 additions and 20 deletions.
24 changes: 4 additions & 20 deletions src/documentation/io/built-in-hadoop.md
Expand Up @@ -217,28 +217,12 @@ org.apache.hive.hcatalog.mapreduce.HCatInputFormat.setInput(hcatConf, "my_databa
Call Read transform as follows:
```java
PCollection<KV<Long, String>> hcatData =
PCollection<KV<Long, HCatRecord>> hcatData =
p.apply("read",
HadoopInputFormatIO.<Long, String>read()
.withConfiguration(hcatConf)
.withValueTranslation(hcatOutputValueType);
HadoopInputFormatIO.<Long, HCatRecord>read()
.withConfiguration(hcatConf);
```
```py
# The Beam SDK for Python does not support Hadoop InputFormat IO.
```
The `HCatInputFormat` key class is `java.lang.Long` `Long`, which has a Beam `Coder`. The `HCatInputFormat` value class is `org.apache.hive.hcatalog.data.HCatRecord` `HCatRecord`, which does not have a Beam `Coder`. Rather than write a new coder, you can provide your own translation method, as follows:
```java
SimpleFunction<HCatRecord, String> hcatOutputValueType = SimpleFunction<HCatRecord, String>()
{
public String apply(HCatRecord record) {
return record.toString();
}
};
```
```py
# The Beam SDK for Python does not support Hadoop InputFormat IO.
```
```

0 comments on commit 192878c

Please sign in to comment.