HIVE-27651: Upgrade hbase version #4649
Conversation
asf-ci-hive
added
tests pending
tests failed
tests unstable
and removed
tests pending
tests failed
labels
zhangbutao
force-pushed
the
HIVE-27651
branch
from
510f9bb
to
39a1994
Compare
asf-ci-hive
added
tests pending
tests unstable
and removed
tests unstable
tests pending
labels
pom.xml
Outdated
| <hbase.version>2.0.0-alpha4</hbase.version> | ||
| <hbase.version>2.5.5</hbase.version> |
There was a problem hiding this comment.
may be you need 2.5.5-hadoop3 here
https://mvnrepository.com/artifact/org.apache.hbase/hbase-client/2.5.5-hadoop3
can try that, if the test passes post that
There was a problem hiding this comment.
Thanks for suggestion. I have used the 2.5.5 and 2.5.5-hadoop3 version, but both of the two failed.
I think Hbase2.5.5 has some compatibility issues with Hadoop3.3.1, and i have filed a Hbase ticket https://issues.apache.org/jira/browse/HBASE-28061. And i also find another ticket about Hbase2.5.5&Hadoop3.3.1 https://issues.apache.org/jira/browse/HBASE-28053. I hope that can get some valuable info from Hbase community.
There was a problem hiding this comment.
Any pointers, here I see you said on the ticket hbase built with 3.3.1 works fine
There was a problem hiding this comment.
This need hbase team releases a hbase version againt hadoop3.3.1, but the hbase-hadoop3 official maven binaries was built againt hadoop3.2.4.
I test hbase with hadoop3.3.1 binary in my local maven which rebuild hbase with hadoop3.3.1 and it indeed works fine.
I have not explore the root cause, maybe need some time or need hbase community to help to fix this. :(
There was a problem hiding this comment.
I think 2.5.6 is released, we can give that a try? HBASE-28061 was fixed in 2.5.6
There was a problem hiding this comment.
Sure, Changed hbase version to 2.5.6. Thanks for reminding!
zhangbutao
force-pushed
the
HIVE-27651
branch
from
b7192f8
to
2ad7d8e
Compare
|
Kudos, SonarCloud Quality Gate passed!
|
zhangbutao
force-pushed
the
HIVE-27651
branch
from
2ad7d8e
to
79b1134
Compare
asf-ci-hive
added
tests pending
tests unstable
and removed
tests unstable
tests pending
labels
|
Kudos, SonarCloud Quality Gate passed!
|
|
Luckily, got a green pass. :) |
| Found 3 items | ||
| drwxr-xr-x - ### USER ### ### GROUP ### 0 ### HDFS DATE ### hdfs://### HDFS PATH ### | ||
| Found 2 items |
There was a problem hiding this comment.
what is this change? One entry got missing?
There was a problem hiding this comment.
Here i did a quick local test using hbase2.2.4 and hbase2.5.6:
I create a hbase table using Hbase shell command cli:
hbase:001:0> create 'mytable', 'cf1', 'cf2'
Created table mytable
- I checked the hdfs data directory of the table in Hbase 2.2.4 and found it has three directory
Found 3 items
drwxr-xr-x - root supergroup 0 2023-10-29 16:16 hdfs://localhost:8028/hbase224/data/default/mytable/.tabledesc
drwxr-xr-x - root supergroup 0 2023-10-29 16:16 hdfs://localhost:8028/hbase224/data/default/mytable/.tmp
drwxr-xr-x - root supergroup 0 2023-10-29 16:16 hdfs://localhost:8028/hbase224/data/default/mytable/556dcd44aa1534942170a780383ad2d3
- I checked the hdfs data directory of the table in Hbase 2.5.6 and found it only has two directory
Found 2 items
drwxr-xr-x - root supergroup 0 2023-10-29 16:44 hdfs://localhost:8028/hbase256/data/default/mytable/.tabledesc
drwxr-xr-x - root supergroup 0 2023-10-29 16:44 hdfs://localhost:8028/hbase256/data/default/mytable/50cac0c44862cd5f9e8387677b0cc0ad
It is quite clear that new Hbase version(2.5.6) has changed its data directory structure, and this This change has nothing to do with Apache Hive.
Do you think my this test can prove that this change has nothing to do with Apache Hive?
There was a problem hiding this comment.
Looks fine, Will merge later today
…d by Ayush Saxena)
What changes were proposed in this pull request?
Upgrade hbase to latest stable version 2.5.6
Why are the changes needed?
HIVE-26124 has tried to upgrade hbase to 2.4.11, but it failed with hadoop3 compatibility problem.
From https://hbase.apache.org/downloads.html, Hbase2.5.6 seems to be compatible with hadoop3, so let's try to upgrade hbase to 2.5.6.
Does this PR introduce any user-facing change?
No
Is the change a dependency upgrade?
How was this patch tested?