Shaded jar version with hbase client shaded and included

[srwilson]

We are currently using the CDH 5.11 for HBase client in our service code, specifically hbase-client-1.2.0-cdh5.11.1. We are also including com.google.cloud.bigtable:bigtable-hbase-1.x-shaded:1.10.0 as a dependency. It appears the bigtable hbase library is using a more recent version of the hbase client 1.4.9. This causes some issues because the newer hbase client has some changes from our 1.2.0-based CDH client version.

Specifically we see on an Admin.listTables() call we get this stack trace

Exception in thread "main" java.io.IOException: Failed to getTableDescriptor() on action_dispatches
	at org.apache.hadoop.hbase.client.AbstractBigtableAdmin.getTableDescriptor(AbstractBigtableAdmin.java:265)
	at org.apache.hadoop.hbase.client.AbstractBigtableAdmin.getTableDescriptors(AbstractBigtableAdmin.java:170)
	at org.apache.hadoop.hbase.client.AbstractBigtableAdmin.listTables(AbstractBigtableAdmin.java:164)
	at com.sift.ScottTest.main(ScottTest.java:44)
Caused by: java.lang.NoSuchMethodError: org.apache.hadoop.hbase.HTableDescriptor.addFamily(Lorg/apache/hadoop/hbase/HColumnDescriptor;)Lorg/apache/hadoop/hbase/HTableDescriptor;
	at com.google.cloud.bigtable.hbase.adapters.admin.TableAdapter.adapt(TableAdapter.java:97)
	at org.apache.hadoop.hbase.client.AbstractBigtableAdmin.getTableDescriptor(AbstractBigtableAdmin.java:260)
	... 3 more

This is due to the fact that hbase mainline client added returning the HTableDescriptor to the addFamily() method. https://github.com/apache/hbase/blob/763f27f583cf8fd7ecf79fb6f3ef57f1615dbf9b/hbase-client/src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java#L879

Upgrading our service code to use mainline hbase client version 1.4.9 would be a major project for us so it's not really feasible.

Is there way we could a get a version of the bigtable hbase client jar that also shades the hbase client it is using? Current that shaded jar that's produced simply excludes the hbase-client dependency.

[sduskis]

Interesting. First, I would suggest using bigtable-hbase-1.x-hadoop instead of bigtable-hbase-1.x-shaded.

In terms of a shaded client, I don't think that us shading in hbase is actually going to help you. We'd need to rename all of the org.apache.hadoop.hbase.* packages to something else in order for this to potentially be useful to you.

We can add some fixes plus testing that ensures that we'll always be compatible with 1.2.0, which might solve your problem.

@igorbernstein2, FYI.

[igorbernstein2]

I don't think 1.2.0 will help with this.

I see that addFamily returned the descriptor as far back as mainline 1.0:
https://github.com/apache/hbase/blob/rel/1.0.0/hbase-client/src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java#L786

And CDH still returns void as of today:
https://github.com/cloudera/hbase/blob/trunk/src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java#L457

This seems like a fundamental binary incompatibility between CDH's api and apache/hbase's api. We compile our client against apache/hbase, so the binaries that we publish expect a method signature that has a return type

[igorbernstein2]

My apologies, I misunderstood CDH's git repo. It seems like master is not up to date. It looks like starting with CDH 6.0.0, addFamily has a return type:
https://github.com/cloudera/hbase/blob/cdh6.0.0-release/hbase-client/src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java#L436

[srwilson]

I don't quite understand why relocating org.apache.hadoop.hbase.* under com.google.bigtable.repackaged. won't fix this problem. The code inside the bigtable client Admin will use the shaded verison which is 1.4.9. Anything in our service code will use the CDH version.

My current workaround is copying HTableDescriptor.java from CDH5.11 into my service's source tree and put it under the correct package, then manually modify the addFamily method in this copy to return the value. Then this implementation will override the one from the dependencies. Not great.

[igorbernstein2]

Hi Scott,

The expected usage of the client (using fully qualified types) is:

org.apache.hadoop.hbase.client.Connection connection = BigtableConfiguration.connect("my-project", "my-instance");
org.apache.hadoop.hbase.client.Table table = connection.getTable(
   org.apache.hadoop.hbase.TableName.valueOf("my-table"));

org.apache.hadoop.hbase.client.Result result = table.get(
  new org.apache.hadoop.hbase.client.Get("my-key".getBytes()));

If we shaded org.apache.hadoop.hbase.* then all of those types would change and the client surface would be incompatible with both mainline HBase and CDH's fork. Shading can only help with internal types, if a type is on the surface it can no longer be shaded without changing the interface. I think the only way to make bigtable-hbase-1.x-* compatible with CDH's fork is to fork the bigtable client and compile it against hbase-shaded-client-1.2.0-cdh5.11.0.

If you don't care about having compatibility with HBase or CDH's fork and are just trying to deal with dependency hell where you are required to have CDH's jars on your classpath. You can create your own intermediate artifact to do the repackaging (similar to how bigtable-hbase-1.x-hadoop works). If go down this road, then I would recommend for you use the bigtable client directly without the hbase compatibility layer:
https://github.com/googleapis/google-cloud-java/tree/master/google-cloud-clients/google-cloud-bigtable

[srwilson]

Thanks for the response, sounds like there's no easy solution here. For now I'll stick with my workaround of adding hacked version of CDH versions of classes as necessary. If that turns into a bigger problem, I'll probably have to fork the bigtable client and get it to compile against the CDH client. We have a ton of existing code that uses the HBase client so rewriting that against the native bigtable client is not feasible.