Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HBASE-24379 CatalogJanitor misreports region holes when there are act… #1741

Merged
merged 1 commit into from
May 22, 2020

Conversation

huaxiangsun
Copy link
Contributor

…ually over laps.

@Apache-HBase
Copy link

🎊 +1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 1m 41s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+1 💚 hbaseanti 0m 0s Patch does not have any anti-patterns.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
_ master Compile Tests _
+1 💚 mvninstall 4m 17s master passed
+1 💚 checkstyle 1m 17s master passed
+1 💚 spotbugs 2m 16s master passed
_ Patch Compile Tests _
+1 💚 mvninstall 3m 50s the patch passed
+1 💚 checkstyle 1m 13s the patch passed
+1 💚 whitespace 0m 0s The patch has no whitespace issues.
+1 💚 hadoopcheck 12m 32s Patch does not cause any errors with Hadoop 3.1.2 3.2.1.
+1 💚 spotbugs 2m 42s the patch passed
_ Other Tests _
+1 💚 asflicense 0m 14s The patch does not generate ASF License warnings.
38m 44s
Subsystem Report/Notes
Docker Client=19.03.9 Server=19.03.9 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1741/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR #1741
Optional Tests dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle
uname Linux 2ae1d519d620 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / 5fb9e51
Max. process+thread count 84 (vs. ulimit of 12500)
modules C: hbase-server U: hbase-server
Console output https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1741/1/console
versions git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12
Powered by Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase
Copy link

🎊 +1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 1m 40s Docker mode activated.
-0 ⚠️ yetus 0m 3s Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚 mvninstall 4m 18s master passed
+1 💚 compile 1m 1s master passed
+1 💚 shadedjars 6m 16s branch has no errors when building our shaded downstream artifacts.
+1 💚 javadoc 0m 39s master passed
_ Patch Compile Tests _
+1 💚 mvninstall 3m 56s the patch passed
+1 💚 compile 0m 58s the patch passed
+1 💚 javac 0m 58s the patch passed
+1 💚 shadedjars 6m 5s patch has no errors when building our shaded downstream artifacts.
+1 💚 javadoc 0m 36s the patch passed
_ Other Tests _
+1 💚 unit 199m 29s hbase-server in the patch passed.
226m 44s
Subsystem Report/Notes
Docker Client=19.03.9 Server=19.03.9 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1741/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR #1741
Optional Tests javac javadoc unit shadedjars compile
uname Linux feabdc161a4a 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / 5fb9e51
Default Java 1.8.0_232
Test Results https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1741/1/testReport/
Max. process+thread count 3165 (vs. ulimit of 12500)
modules C: hbase-server U: hbase-server
Console output https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1741/1/console
versions git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f)
Powered by Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase
Copy link

💔 -1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 1m 12s Docker mode activated.
-0 ⚠️ yetus 0m 3s Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚 mvninstall 4m 45s master passed
+1 💚 compile 1m 8s master passed
+1 💚 shadedjars 6m 19s branch has no errors when building our shaded downstream artifacts.
-0 ⚠️ javadoc 0m 43s hbase-server in master failed.
_ Patch Compile Tests _
+1 💚 mvninstall 4m 38s the patch passed
+1 💚 compile 1m 8s the patch passed
+1 💚 javac 1m 8s the patch passed
+1 💚 shadedjars 6m 25s patch has no errors when building our shaded downstream artifacts.
-0 ⚠️ javadoc 0m 40s hbase-server in the patch failed.
_ Other Tests _
-1 ❌ unit 218m 11s hbase-server in the patch failed.
247m 4s
Subsystem Report/Notes
Docker Client=19.03.9 Server=19.03.9 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1741/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR #1741
Optional Tests javac javadoc unit shadedjars compile
uname Linux c9075d69d90e 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / 5fb9e51
Default Java 2020-01-14
javadoc https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1741/1/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt
javadoc https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1741/1/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt
unit https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1741/1/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt
Test Results https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1741/1/testReport/
Max. process+thread count 3281 (vs. ulimit of 12500)
modules C: hbase-server U: hbase-server
Console output https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1741/1/console
versions git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f)
Powered by Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

@huaxiangsun
Copy link
Contributor Author

I run the failed test locally and it passed. The test case timed out, I am kicking off another run.

@Apache-HBase
Copy link

🎊 +1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 1m 37s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+1 💚 hbaseanti 0m 0s Patch does not have any anti-patterns.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
_ master Compile Tests _
+1 💚 mvninstall 4m 16s master passed
+1 💚 checkstyle 1m 14s master passed
+1 💚 spotbugs 2m 19s master passed
_ Patch Compile Tests _
+1 💚 mvninstall 3m 51s the patch passed
+1 💚 checkstyle 1m 13s the patch passed
+1 💚 whitespace 0m 0s The patch has no whitespace issues.
+1 💚 hadoopcheck 12m 42s Patch does not cause any errors with Hadoop 3.1.2 3.2.1.
+1 💚 spotbugs 2m 53s the patch passed
_ Other Tests _
+1 💚 asflicense 0m 17s The patch does not generate ASF License warnings.
39m 9s
Subsystem Report/Notes
Docker Client=19.03.9 Server=19.03.9 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1741/2/artifact/yetus-general-check/output/Dockerfile
GITHUB PR #1741
Optional Tests dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle
uname Linux b912cb5d8a5e 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / 5fb9e51
Max. process+thread count 84 (vs. ulimit of 12500)
modules C: hbase-server U: hbase-server
Console output https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1741/2/console
versions git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12
Powered by Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase
Copy link

🎊 +1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 0m 31s Docker mode activated.
-0 ⚠️ yetus 0m 3s Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚 mvninstall 4m 16s master passed
+1 💚 compile 1m 6s master passed
+1 💚 shadedjars 5m 42s branch has no errors when building our shaded downstream artifacts.
-0 ⚠️ javadoc 0m 41s hbase-server in master failed.
_ Patch Compile Tests _
+1 💚 mvninstall 4m 0s the patch passed
+1 💚 compile 1m 2s the patch passed
+1 💚 javac 1m 2s the patch passed
+1 💚 shadedjars 5m 42s patch has no errors when building our shaded downstream artifacts.
-0 ⚠️ javadoc 0m 40s hbase-server in the patch failed.
_ Other Tests _
+1 💚 unit 126m 35s hbase-server in the patch passed.
152m 26s
Subsystem Report/Notes
Docker Client=19.03.9 Server=19.03.9 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1741/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR #1741
Optional Tests javac javadoc unit shadedjars compile
uname Linux 3de75476ab1f 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / 5fb9e51
Default Java 2020-01-14
javadoc https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1741/2/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt
javadoc https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1741/2/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt
Test Results https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1741/2/testReport/
Max. process+thread count 4256 (vs. ulimit of 12500)
modules C: hbase-server U: hbase-server
Console output https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1741/2/console
versions git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f)
Powered by Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

Copy link
Contributor

@saintstack saintstack left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we try more combos here Huaxiang? I think this will get one case for sure but what happens if:

[a, g), [bb, cc), [dd, ee)

... will we report hole between a-bb or cc-dd or ee-g ?

Or will it just show as overlap?

Otherwise +1.

@saintstack
Copy link
Contributor

Or rather, while we are in here, maybe add some more test cases since they cheap to do.

@huaxiangsun
Copy link
Contributor Author

Should we try more combos here Huaxiang? I think this will get one case for sure but what happens if:

[a, g), [bb, cc), [dd, ee)

... will we report hole between a-bb or cc-dd or ee-g ?

Or will it just show as overlap?

Thanks Stack.

Yeah, they will be reported as overlaps. In this case, g is the region with highest key and it overlaps with the other two regions. The algorithm reporting holes only has one corner case which is fixed in this patch. I will add more test cases.

@huaxiangsun
Copy link
Contributor Author

Add more test case, updated the patch.

@Apache-HBase
Copy link

💔 -1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 1m 5s Docker mode activated.
-0 ⚠️ yetus 0m 3s Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚 mvninstall 4m 44s master passed
+1 💚 compile 1m 8s master passed
+1 💚 shadedjars 6m 18s branch has no errors when building our shaded downstream artifacts.
-0 ⚠️ javadoc 0m 42s hbase-server in master failed.
_ Patch Compile Tests _
+1 💚 mvninstall 4m 29s the patch passed
+1 💚 compile 1m 7s the patch passed
+1 💚 javac 1m 7s the patch passed
+1 💚 shadedjars 6m 25s patch has no errors when building our shaded downstream artifacts.
-0 ⚠️ javadoc 0m 42s hbase-server in the patch failed.
_ Other Tests _
-1 ❌ unit 9m 28s hbase-server in the patch failed.
37m 11s
Subsystem Report/Notes
Docker Client=19.03.9 Server=19.03.9 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1741/3/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR #1741
Optional Tests javac javadoc unit shadedjars compile
uname Linux a9d03d7d25f3 4.15.0-101-generic #102-Ubuntu SMP Mon May 11 10:07:26 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / e1d7dc8
Default Java 2020-01-14
javadoc https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1741/3/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt
javadoc https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1741/3/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt
unit https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1741/3/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt
Test Results https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1741/3/testReport/
Max. process+thread count 609 (vs. ulimit of 12500)
modules C: hbase-server U: hbase-server
Console output https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1741/3/console
versions git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f)
Powered by Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase
Copy link

🎊 +1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 1m 49s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+1 💚 hbaseanti 0m 0s Patch does not have any anti-patterns.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
_ master Compile Tests _
+1 💚 mvninstall 4m 51s master passed
+1 💚 checkstyle 1m 30s master passed
+1 💚 spotbugs 2m 30s master passed
_ Patch Compile Tests _
+1 💚 mvninstall 4m 21s the patch passed
+1 💚 checkstyle 1m 25s the patch passed
+1 💚 whitespace 0m 0s The patch has no whitespace issues.
+1 💚 hadoopcheck 13m 57s Patch does not cause any errors with Hadoop 3.1.2 3.2.1.
+1 💚 spotbugs 2m 45s the patch passed
_ Other Tests _
+1 💚 asflicense 0m 14s The patch does not generate ASF License warnings.
41m 31s
Subsystem Report/Notes
Docker Client=19.03.9 Server=19.03.9 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1741/3/artifact/yetus-general-check/output/Dockerfile
GITHUB PR #1741
Optional Tests dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle
uname Linux baa27d8dcc12 4.15.0-101-generic #102-Ubuntu SMP Mon May 11 10:07:26 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / e1d7dc8
Max. process+thread count 84 (vs. ulimit of 12500)
modules C: hbase-server U: hbase-server
Console output https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1741/3/console
versions git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12
Powered by Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase
Copy link

🎊 +1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 0m 31s Docker mode activated.
-0 ⚠️ yetus 0m 3s Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚 mvninstall 3m 48s master passed
+1 💚 compile 0m 54s master passed
+1 💚 shadedjars 5m 34s branch has no errors when building our shaded downstream artifacts.
+1 💚 javadoc 0m 39s master passed
_ Patch Compile Tests _
+1 💚 mvninstall 3m 36s the patch passed
+1 💚 compile 0m 56s the patch passed
+1 💚 javac 0m 56s the patch passed
+1 💚 shadedjars 5m 38s patch has no errors when building our shaded downstream artifacts.
+1 💚 javadoc 0m 38s the patch passed
_ Other Tests _
+1 💚 unit 136m 5s hbase-server in the patch passed.
160m 8s
Subsystem Report/Notes
Docker Client=19.03.9 Server=19.03.9 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1741/3/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR #1741
Optional Tests javac javadoc unit shadedjars compile
uname Linux 7ca8d45ac0d2 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / e1d7dc8
Default Java 1.8.0_232
Test Results https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1741/3/testReport/
Max. process+thread count 4525 (vs. ulimit of 12500)
modules C: hbase-server U: hbase-server
Console output https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1741/3/console
versions git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f)
Powered by Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

@huaxiangsun
Copy link
Contributor Author

The test failure is HBASE-23976 (seems that it still happens).

@huaxiangsun huaxiangsun merged commit 2b894dc into apache:master May 22, 2020
huaxiangsun added a commit to huaxiangsun/hbase that referenced this pull request May 25, 2020
…ually over laps. (apache#1741)

Signed-off-by: stack <stack@apache.org>
huaxiangsun added a commit to huaxiangsun/hbase that referenced this pull request May 25, 2020
…ually over laps. (apache#1741)

Signed-off-by: stack <stack@apache.org>
huaxiangsun added a commit that referenced this pull request May 26, 2020
…ually over laps. (#1741) (#1777)

Signed-off-by: stack <stack@apache.org>
huaxiangsun added a commit that referenced this pull request May 26, 2020
…ually over laps. (#1741) (#1776)

Signed-off-by: stack <stack@apache.org>
huaxiangsun added a commit to huaxiangsun/hbase that referenced this pull request May 26, 2020
…ually over laps. (apache#1741)

Signed-off-by: stack <stack@apache.org>
huaxiangsun added a commit that referenced this pull request May 27, 2020
…ually over laps. (#1741) (#1785)

Signed-off-by: stack <stack@apache.org>
HomminLee pushed a commit to HomminLee/hbase that referenced this pull request Jun 12, 2020
…ually over laps. (apache#1741) (apache#1785)

Signed-off-by: stack <stack@apache.org>
infraio pushed a commit to infraio/hbase that referenced this pull request Aug 17, 2020
…ually over laps. (apache#1741) (apache#1776)

Signed-off-by: stack <stack@apache.org>
clarax pushed a commit to clarax/hbase that referenced this pull request Nov 15, 2020
…ually over laps. (apache#1741) (apache#1777)

Signed-off-by: stack <stack@apache.org>
symat pushed a commit to symat/hbase that referenced this pull request Feb 17, 2021
…ually over laps. (apache#1741) (apache#1785)

Author: huaxiangsun
Reason: Bug
Ref: CDPD-15964

Signed-off-by: stack <stack@apache.org>

Change-Id: Ic77366213b962dc28c428ac57dcdc19d5c2287f1
(cherry picked from commit b850603)
wchevreuil pushed a commit to wchevreuil/hbase that referenced this pull request May 24, 2021
…ually over laps. (apache#1741) (apache#1785)

Signed-off-by: stack <stack@apache.org>
(cherry picked from commit 482956e)

Change-Id: Ic77366213b962dc28c428ac57dcdc19d5c2287f1
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
3 participants