[BEAM-2411] Make the write transform of HBaseIO simpler#3391
[BEAM-2411] Make the write transform of HBaseIO simpler#3391asfgit merged 2 commits intoapache:masterfrom
Conversation
|
R: @jkff A simplification based on our discussion on the Bigtable-HBase duality. |
a82f283 to
eee4350
Compare
|
There is an extra thing, notice that I made elementCounters for debug per bundle, this is a fix on the previous version, and aligns it with BigtableIO. |
jkff
left a comment
There was a problem hiding this comment.
Thanks! LGTM but also feel free to implement the suggested coder improvement.
|
|
||
| public static final Coder<KV<byte[], Iterable<Mutation>>> WRITE_CODER = | ||
| KvCoder.of(ByteArrayCoder.of(), IterableCoder.of(HBaseMutationCoder.of())); | ||
| public static final Coder<Mutation> WRITE_CODER = HBaseMutationCoder.of(); |
There was a problem hiding this comment.
While we're at simplifying things.. I believe you can remove this and all the setCoder calls by using something like https://github.com/apache/beam/blob/master/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryCoderProviderRegistrar.java
|
Thanks for the hint on the CoderProviderRegistrar, I am going to take a look and add it to the PR |
|
Retest this please |
eee4350 to
d42f633
Compare
|
Thanks for the review/ideas Eugene. |
Be sure to do all of the following to help us incorporate your contribution
quickly and easily:
[BEAM-<Jira issue #>] Description of pull requestmvn clean verify.<Jira issue #>in the title with the actual Jira issuenumber, if there is one.
Individual Contributor License Agreement.