Skip to content

Commit

Permalink
HBASE-23800 Add documentation about the CECPs changes (#1487)
Browse files Browse the repository at this point in the history
Signed-off-by: stack <stack@apache.org>
  • Loading branch information
Apache9 committed Apr 18, 2020
1 parent b02064f commit 2c8d104
Show file tree
Hide file tree
Showing 2 changed files with 99 additions and 4 deletions.
31 changes: 27 additions & 4 deletions src/main/asciidoc/_chapters/cp.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -178,11 +178,28 @@ average or summation for an entire table which spans hundreds of regions.
In contrast to observer coprocessors, where your code is run transparently, endpoint
coprocessors must be explicitly invoked using the
link:https://hbase.apache.org/devapidocs/org/apache/hadoop/hbase/client/Table.html#coprocessorService-java.lang.Class-byte:A-byte:A-org.apache.hadoop.hbase.client.coprocessor.Batch.Call-[CoprocessorService()]
link:https://hbase.apache.org/devapidocs/org/apache/hadoop/hbase/client/AsyncTable.html#coprocessorService-java.util.function.Function-org.apache.hadoop.hbase.client.ServiceCaller-byte:A-[CoprocessorService()]
method available in
link:https://hbase.apache.org/devapidocs/org/apache/hadoop/hbase/client/Table.html[Table]
or
link:https://hbase.apache.org/devapidocs/org/apache/hadoop/hbase/client/HTable.html[HTable].
link:https://hbase.apache.org/devapidocs/org/apache/hadoop/hbase/client/AsyncTable.html[AsyncTable].
[WARNING]
.On using coprocessorService method with sync client
====
The coprocessorService method in link:https://hbase.apache.org/devapidocs/org/apache/hadoop/hbase/client/Table.html[Table]
has been deprecated.
In link:https://issues.apache.org/jira/browse/HBASE-21512[HBASE-21512]
we reimplement the sync client based on the async client. The coprocessorService
method defined in `Table` interface directly references a method from protobuf's
`BlockingInterface`, which means we need to use a separate thread pool to execute
the method so we avoid blocking the async client(We want to avoid blocking calls in
our async implementation).
Since coprocessor is an advanced feature, we believe it is OK for coprocessor users to
instead switch over to use `AsyncTable`. There is a lightweight
link:https://hbase.apache.org/devapidocs/org/apache/hadoop/hbase/client/Connection.html#toAsyncConnection--[toAsyncConnection]
method to get an `AsyncConnection` from `Connection` if needed.
====
Starting with HBase 0.96, endpoint coprocessors are implemented using Google Protocol
Buffers (protobuf). For more details on protobuf, see Google's
Expand All @@ -193,6 +210,12 @@ link:https://issues.apache.org/jira/browse/HBASE-5448[HBASE-5448]). To upgrade y
HBase cluster from 0.94 or earlier to 0.96 or later, you need to reimplement your
coprocessor.
In HBase 2.x, we made use of a shaded version of protobuf 3.x, but kept the
protobuf for coprocessors on 2.5.0. In HBase 3.0.0, we removed all dependencies on
non-shaded protobuf so you need to reimplement your coprocessor to make use of the
shaded protobuf version provided in hbase-thirdparty. Please see
the <<protobuf,protobuf>> section for more details.
Coprocessor Endpoints should make no use of HBase internals and
only avail of public APIs; ideally a CPEP should depend on Interfaces
and data structures only. This is not always possible but beware
Expand Down
72 changes: 72 additions & 0 deletions src/main/asciidoc/_chapters/protobuf.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -148,3 +148,75 @@ consider extending it also in
Going forward, we will provide a new module of common types for use
by CPEPs that will have the same guarantees against change as does our
public API. TODO.

=== protobuf changes for hbase-3.0.0 (HBASE-23797)
Since hadoop(start from 3.3.x) also shades protobuf and bumps the version to
3.x, there is no reason for us to stay on protobuf 2.5.0 any more.

In HBase 3.0.0, the hbase-protocol module has been purged, the CPEP
implementation should use the protos in hbase-protocol-shaded module, and also
make use of the shaded protobuf in hbase-thirdparty. In general, we will keep
the protobuf version compatible for a whole major release, unless there are
critical problems, for example, a critical CVE on protobuf.

Add this dependency to your pom:
[source,xml]
----
<dependency>
<groupId>org.apache.hbase.thirdparty</groupId>
<artifactId>hbase-shaded-protobuf</artifactId>
<!-- use the version that your target hbase cluster uses -->
<version>${hbase-thirdparty.version}</version>
<scope>provided</scope>
</dependency>
----

And typically you also need to add this plugin to your pom to make your
generated protobuf code also use the shaded and relocated protobuf version
in hbase-thirdparty.
[source,xml]
----
<plugin>
<groupId>com.google.code.maven-replacer-plugin</groupId>
<artifactId>replacer</artifactId>
<version>1.5.3</version>
<executions>
<execution>
<phase>process-sources</phase>
<goals>
<goal>replace</goal>
</goals>
</execution>
</executions>
<configuration>
<basedir>${basedir}/target/generated-sources/</basedir>
<includes>
<include>**/*.java</include>
</includes>
<!-- Ignore errors when missing files, because it means this build
was run with -Dprotoc.skip and there is no -Dreplacer.skip -->
<ignoreErrors>true</ignoreErrors>
<replacements>
<replacement>
<token>([^\.])com.google.protobuf</token>
<value>$1org.apache.hbase.thirdparty.com.google.protobuf</value>
</replacement>
<replacement>
<token>(public)(\W+static)?(\W+final)?(\W+class)</token>
<value>@javax.annotation.Generated("proto") $1$2$3$4</value>
</replacement>
<!-- replacer doesn't support anchoring or negative lookbehind -->
<replacement>
<token>(@javax.annotation.Generated\("proto"\) ){2}</token>
<value>$1</value>
</replacement>
</replacements>
</configuration>
</plugin>
----

In hbase-examples module, we have some examples under the
`org.apache.hadoop.hbase.coprocessor.example` package. You can see
`BulkDeleteEndpoint` and `BulkDelete.proto` for more details, and you can also
check the `pom.xml` of hbase-examples module to see how to make use of the above
plugin.

0 comments on commit 2c8d104

Please sign in to comment.