Skip to content

HBASE-26864 SplitTableRegionProcedure calls openParentRegions() at a …#4261

Merged
apurtell merged 2 commits intoapache:masterfrom
huaxiangsun:master.HBASE-26864
Mar 26, 2022
Merged

HBASE-26864 SplitTableRegionProcedure calls openParentRegions() at a …#4261
apurtell merged 2 commits intoapache:masterfrom
huaxiangsun:master.HBASE-26864

Conversation

@huaxiangsun
Copy link
Contributor

…wrong state during rollback

@Apache-HBase
Copy link

🎊 +1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 1m 1s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+1 💚 hbaseanti 0m 0s Patch does not have any anti-patterns.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
_ master Compile Tests _
+1 💚 mvninstall 2m 22s master passed
+1 💚 compile 2m 15s master passed
+1 💚 checkstyle 0m 37s master passed
+1 💚 spotbugs 1m 19s master passed
_ Patch Compile Tests _
+1 💚 mvninstall 2m 11s the patch passed
+1 💚 compile 2m 12s the patch passed
+1 💚 javac 2m 12s the patch passed
+1 💚 checkstyle 0m 36s the patch passed
+1 💚 whitespace 0m 0s The patch has no whitespace issues.
+1 💚 hadoopcheck 11m 24s Patch does not cause any errors with Hadoop 3.1.2 3.2.2 3.3.1.
+1 💚 spotbugs 1m 21s the patch passed
_ Other Tests _
+1 💚 asflicense 0m 10s The patch does not generate ASF License warnings.
30m 36s
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4261/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR #4261
Optional Tests dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile
uname Linux db972f91c02c 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / b3f00d0
Default Java AdoptOpenJDK-1.8.0_282-b08
Max. process+thread count 60 (vs. ulimit of 30000)
modules C: hbase-server U: hbase-server
Console output https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4261/1/console
versions git=2.17.1 maven=3.6.3 spotbugs=4.2.2
Powered by Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase
Copy link

🎊 +1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 0m 35s Docker mode activated.
-0 ⚠️ yetus 0m 4s Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚 mvninstall 3m 31s master passed
+1 💚 compile 0m 59s master passed
+1 💚 shadedjars 5m 24s branch has no errors when building our shaded downstream artifacts.
+1 💚 javadoc 0m 34s master passed
_ Patch Compile Tests _
+1 💚 mvninstall 2m 57s the patch passed
+1 💚 compile 0m 50s the patch passed
+1 💚 javac 0m 50s the patch passed
+1 💚 shadedjars 4m 38s patch has no errors when building our shaded downstream artifacts.
+1 💚 javadoc 0m 26s the patch passed
_ Other Tests _
+1 💚 unit 207m 16s hbase-server in the patch passed.
229m 4s
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4261/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR #4261
Optional Tests javac javadoc unit shadedjars compile
uname Linux ef2bca3797e5 5.4.0-96-generic #109-Ubuntu SMP Wed Jan 12 16:49:16 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / b3f00d0
Default Java AdoptOpenJDK-1.8.0_282-b08
Test Results https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4261/1/testReport/
Max. process+thread count 2293 (vs. ulimit of 30000)
modules C: hbase-server U: hbase-server
Console output https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4261/1/console
versions git=2.17.1 maven=3.6.3
Powered by Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase
Copy link

🎊 +1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 1m 20s Docker mode activated.
-0 ⚠️ yetus 0m 4s Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚 mvninstall 4m 30s master passed
+1 💚 compile 1m 12s master passed
+1 💚 shadedjars 5m 28s branch has no errors when building our shaded downstream artifacts.
+1 💚 javadoc 0m 42s master passed
_ Patch Compile Tests _
+1 💚 mvninstall 3m 53s the patch passed
+1 💚 compile 1m 11s the patch passed
+1 💚 javac 1m 11s the patch passed
+1 💚 shadedjars 5m 22s patch has no errors when building our shaded downstream artifacts.
+1 💚 javadoc 0m 39s the patch passed
_ Other Tests _
+1 💚 unit 302m 30s hbase-server in the patch passed.
330m 1s
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4261/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR #4261
Optional Tests javac javadoc unit shadedjars compile
uname Linux 4e4fb3a93b36 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / b3f00d0
Default Java AdoptOpenJDK-11.0.10+9
Test Results https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4261/1/testReport/
Max. process+thread count 2481 (vs. ulimit of 30000)
modules C: hbase-server U: hbase-server
Console output https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4261/1/console
versions git=2.17.1 maven=3.6.3
Powered by Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@joshelser
Copy link
Member

Nice find! Able to write a unit-test to cover this?

Copy link
Contributor

@Apache9 Apache9 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

And how could we get IOException when creating procedures for unassign regions?

@huaxiangsun
Copy link
Contributor Author

huaxiangsun commented Mar 23, 2022

And how could we get IOException when creating procedures for unassign regions?

In our specific case, there is region replica configured for the table. When the primary region is opened at one region server and it needs to split, it sends a SplitRegionRequest to master. At the master side, it checks all replica regions. If one replica region is in RIT (being opened), it will throw out IOException in creating procedures for unassign regions.

@huaxiangsun
Copy link
Contributor Author

Nice find! Able to write a unit-test to cover this?

I spent quite some time trying to come up with a unit-test. It is hard to trigger some exception in SplitTableRegionProcedure's state machine. Let me try with regionReplica and a coprocessor to delay open of a replica region.

@huaxiangsun
Copy link
Contributor Author

Created a unitest case. During the test, I found that there is an issue where ignoreRIT flag does not apply to replicaRegions. However, from our production log, I do not find any issue I run into with the unitest. I made sure that with the fix, the new unittest case passes and without the fix, it will fail.

@Apache9 and @joshelser, can you review the new patch? Thanks.

@Apache-HBase
Copy link

🎊 +1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 0m 58s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+1 💚 hbaseanti 0m 0s Patch does not have any anti-patterns.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
_ master Compile Tests _
+1 💚 mvninstall 2m 29s master passed
+1 💚 compile 2m 14s master passed
+1 💚 checkstyle 0m 35s master passed
+1 💚 spotbugs 1m 14s master passed
_ Patch Compile Tests _
+1 💚 mvninstall 2m 20s the patch passed
+1 💚 compile 2m 11s the patch passed
+1 💚 javac 2m 11s the patch passed
-0 ⚠️ checkstyle 0m 32s hbase-server: The patch generated 4 new + 3 unchanged - 0 fixed = 7 total (was 3)
+1 💚 whitespace 0m 0s The patch has no whitespace issues.
+1 💚 hadoopcheck 11m 33s Patch does not cause any errors with Hadoop 3.1.2 3.2.2 3.3.1.
+1 💚 spotbugs 1m 17s the patch passed
_ Other Tests _
+1 💚 asflicense 0m 8s The patch does not generate ASF License warnings.
30m 32s
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4261/2/artifact/yetus-general-check/output/Dockerfile
GITHUB PR #4261
Optional Tests dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile
uname Linux 3fc88f035c2b 5.4.0-1025-aws #25~18.04.1-Ubuntu SMP Fri Sep 11 12:03:04 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / ab4351a
Default Java AdoptOpenJDK-1.8.0_282-b08
checkstyle https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4261/2/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt
Max. process+thread count 60 (vs. ulimit of 30000)
modules C: hbase-server U: hbase-server
Console output https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4261/2/console
versions git=2.17.1 maven=3.6.3 spotbugs=4.2.2
Powered by Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase
Copy link

💔 -1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 0m 28s Docker mode activated.
-0 ⚠️ yetus 0m 2s Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚 mvninstall 2m 14s master passed
+1 💚 compile 0m 38s master passed
+1 💚 shadedjars 3m 40s branch has no errors when building our shaded downstream artifacts.
+1 💚 javadoc 0m 23s master passed
_ Patch Compile Tests _
+1 💚 mvninstall 2m 12s the patch passed
+1 💚 compile 0m 38s the patch passed
+1 💚 javac 0m 38s the patch passed
+1 💚 shadedjars 3m 36s patch has no errors when building our shaded downstream artifacts.
+1 💚 javadoc 0m 24s the patch passed
_ Other Tests _
-1 ❌ unit 170m 0s hbase-server in the patch failed.
185m 28s
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4261/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR #4261
Optional Tests javac javadoc unit shadedjars compile
uname Linux aa3c2867022f 5.4.0-96-generic #109-Ubuntu SMP Wed Jan 12 16:49:16 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / ab4351a
Default Java AdoptOpenJDK-1.8.0_282-b08
unit https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4261/2/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt
Test Results https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4261/2/testReport/
Max. process+thread count 2890 (vs. ulimit of 30000)
modules C: hbase-server U: hbase-server
Console output https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4261/2/console
versions git=2.17.1 maven=3.6.3
Powered by Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase
Copy link

💔 -1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 1m 29s Docker mode activated.
-0 ⚠️ yetus 0m 3s Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚 mvninstall 4m 12s master passed
+1 💚 compile 1m 17s master passed
+1 💚 shadedjars 5m 28s branch has no errors when building our shaded downstream artifacts.
+1 💚 javadoc 0m 41s master passed
_ Patch Compile Tests _
+1 💚 mvninstall 4m 0s the patch passed
+1 💚 compile 1m 12s the patch passed
+1 💚 javac 1m 12s the patch passed
+1 💚 shadedjars 5m 24s patch has no errors when building our shaded downstream artifacts.
+1 💚 javadoc 0m 39s the patch passed
_ Other Tests _
-1 ❌ unit 263m 37s hbase-server in the patch failed.
290m 49s
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4261/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR #4261
Optional Tests javac javadoc unit shadedjars compile
uname Linux 19423a2f2702 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / ab4351a
Default Java AdoptOpenJDK-11.0.10+9
unit https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4261/2/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt
Test Results https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4261/2/testReport/
Max. process+thread count 2439 (vs. ulimit of 30000)
modules C: hbase-server U: hbase-server
Console output https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4261/2/console
versions git=2.17.1 maven=3.6.3
Powered by Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase
Copy link

🎊 +1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 1m 4s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+1 💚 hbaseanti 0m 0s Patch does not have any anti-patterns.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
_ master Compile Tests _
+1 💚 mvninstall 2m 9s master passed
+1 💚 compile 2m 12s master passed
+1 💚 checkstyle 0m 35s master passed
+1 💚 spotbugs 1m 14s master passed
_ Patch Compile Tests _
+1 💚 mvninstall 2m 11s the patch passed
+1 💚 compile 2m 14s the patch passed
+1 💚 javac 2m 14s the patch passed
-0 ⚠️ checkstyle 0m 35s hbase-server: The patch generated 4 new + 3 unchanged - 0 fixed = 7 total (was 3)
+1 💚 whitespace 0m 0s The patch has no whitespace issues.
+1 💚 hadoopcheck 11m 18s Patch does not cause any errors with Hadoop 3.1.2 3.2.2 3.3.1.
+1 💚 spotbugs 1m 23s the patch passed
_ Other Tests _
+1 💚 asflicense 0m 9s The patch does not generate ASF License warnings.
29m 46s
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4261/3/artifact/yetus-general-check/output/Dockerfile
GITHUB PR #4261
Optional Tests dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile
uname Linux 53bf03f9d79b 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / ec71417
Default Java AdoptOpenJDK-1.8.0_282-b08
checkstyle https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4261/3/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt
Max. process+thread count 60 (vs. ulimit of 30000)
modules C: hbase-server U: hbase-server
Console output https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4261/3/console
versions git=2.17.1 maven=3.6.3 spotbugs=4.2.2
Powered by Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase
Copy link

🎊 +1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 0m 25s Docker mode activated.
-0 ⚠️ yetus 0m 2s Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚 mvninstall 3m 50s master passed
+1 💚 compile 0m 53s master passed
+1 💚 shadedjars 4m 10s branch has no errors when building our shaded downstream artifacts.
+1 💚 javadoc 0m 25s master passed
_ Patch Compile Tests _
+1 💚 mvninstall 2m 9s the patch passed
+1 💚 compile 0m 38s the patch passed
+1 💚 javac 0m 38s the patch passed
+1 💚 shadedjars 3m 38s patch has no errors when building our shaded downstream artifacts.
+1 💚 javadoc 0m 24s the patch passed
_ Other Tests _
+1 💚 unit 170m 5s hbase-server in the patch passed.
188m 13s
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4261/3/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR #4261
Optional Tests javac javadoc unit shadedjars compile
uname Linux a424639c6ec5 5.4.0-96-generic #109-Ubuntu SMP Wed Jan 12 16:49:16 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / ec71417
Default Java AdoptOpenJDK-1.8.0_282-b08
Test Results https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4261/3/testReport/
Max. process+thread count 3102 (vs. ulimit of 30000)
modules C: hbase-server U: hbase-server
Console output https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4261/3/console
versions git=2.17.1 maven=3.6.3
Powered by Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@huaxiangsun
Copy link
Contributor Author

I run locally the failed test TestQuotaAdmin, it passed.

@huaxiangsun
Copy link
Contributor Author

Ping @joshelser and @Apache9, any comments about the new diff? Thanks.

@apurtell apurtell merged commit 698937d into apache:master Mar 26, 2022
asfgit pushed a commit that referenced this pull request Mar 26, 2022
#4261)

Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
asfgit pushed a commit that referenced this pull request Mar 26, 2022
#4261)

Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
asfgit pushed a commit that referenced this pull request Mar 26, 2022
#4261)

Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
vinayakphegde pushed a commit to vinayakphegde/hbase that referenced this pull request Apr 4, 2024
apache#4261)

Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
(cherry picked from commit 34cb2ee)
Change-Id: I03c1df63352f07b679b10edb892cfa749d8b6708
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants

Comments