Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HBASE-25206 Data loss can happen if a cloned table loses original spl… #2569

Merged
merged 1 commit into from
Oct 24, 2020

Conversation

brfrn169
Copy link
Member

…it region(delete table)

@brfrn169 brfrn169 self-assigned this Oct 20, 2020
@Apache-HBase

This comment has been minimized.

@Apache9
Copy link
Contributor

Apache9 commented Oct 20, 2020

They are added by this issue.

https://issues.apache.org/jira/browse/HBASE-23187

Let me take a look on the PR about why we add these checks.

@Apache9
Copy link
Contributor

Apache9 commented Oct 20, 2020

Oh the logic is already there before HBASE-23187. In HBASE-23187 we just add checks for RegionInfo. Let me check for earlier changes.

@Apache9
Copy link
Contributor

Apache9 commented Oct 20, 2020

Oh it is in HBASE-14614, where the patch is several MB, which means we can not know the actual reason on why the code is like this...

If you do not mind, I plan to open an issue first to address the problem of the include method, and then we could come back to fix the bug here.

Thanks.

@Apache-HBase

This comment has been minimized.

@Apache-HBase

This comment has been minimized.

@brfrn169
Copy link
Member Author

Thank you for reviewing this! @Apache9

If you do not mind, I plan to open an issue first to address the problem of the include method, and then we could come back to fix the bug here.

Sure. Please go ahead. Thanks.

@@ -100,6 +101,12 @@ protected Flow executeFromState(final MasterProcedureEnv env, DeleteTableState s
// TODO: Move out... in the acquireLock()
LOG.debug("Waiting for RIT for {}", this);
regions = env.getAssignmentManager().getRegionStates().getRegionsOfTable(getTableName());
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think here, we could add a method in RegionStates to get regions of table for disabling, so we do not need to interate on the region state nodes of this table again.

@brfrn169
Copy link
Member Author

@Apache9 I just modified the patch for your review. Can you please review it when you get a chance? Thanks.

@Apache-HBase
Copy link

🎊 +1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 1m 10s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+1 💚 hbaseanti 0m 0s Patch does not have any anti-patterns.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
_ master Compile Tests _
+1 💚 mvninstall 3m 24s master passed
+1 💚 checkstyle 1m 2s master passed
+1 💚 spotbugs 2m 1s master passed
_ Patch Compile Tests _
+1 💚 mvninstall 3m 22s the patch passed
+1 💚 checkstyle 1m 3s the patch passed
+1 💚 whitespace 0m 0s The patch has no whitespace issues.
+1 💚 hadoopcheck 16m 55s Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0.
+1 💚 spotbugs 2m 11s the patch passed
_ Other Tests _
+1 💚 asflicense 0m 16s The patch does not generate ASF License warnings.
38m 37s
Subsystem Report/Notes
Docker Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2569/3/artifact/yetus-general-check/output/Dockerfile
GITHUB PR #2569
Optional Tests dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle
uname Linux 354b4c290bbd 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17 17:06:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / d8de24c
Max. process+thread count 94 (vs. ulimit of 30000)
modules C: hbase-server U: hbase-server
Console output https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2569/3/console
versions git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12
Powered by Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase
Copy link

💔 -1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 1m 4s Docker mode activated.
-0 ⚠️ yetus 0m 3s Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚 mvninstall 3m 57s master passed
+1 💚 compile 1m 7s master passed
+1 💚 shadedjars 6m 38s branch has no errors when building our shaded downstream artifacts.
+1 💚 javadoc 0m 39s master passed
_ Patch Compile Tests _
+1 💚 mvninstall 3m 58s the patch passed
+1 💚 compile 1m 5s the patch passed
+1 💚 javac 1m 5s the patch passed
+1 💚 shadedjars 6m 35s patch has no errors when building our shaded downstream artifacts.
+1 💚 javadoc 0m 40s the patch passed
_ Other Tests _
-1 ❌ unit 143m 45s hbase-server in the patch failed.
171m 33s
Subsystem Report/Notes
Docker Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2569/3/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR #2569
Optional Tests javac javadoc unit shadedjars compile
uname Linux 7e3a58a3529c 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / d8de24c
Default Java 2020-01-14
unit https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2569/3/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt
Test Results https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2569/3/testReport/
Max. process+thread count 3959 (vs. ulimit of 30000)
modules C: hbase-server U: hbase-server
Console output https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2569/3/console
versions git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f)
Powered by Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase
Copy link

💔 -1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 0m 32s Docker mode activated.
-0 ⚠️ yetus 0m 3s Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚 mvninstall 3m 24s master passed
+1 💚 compile 0m 56s master passed
+1 💚 shadedjars 6m 51s branch has no errors when building our shaded downstream artifacts.
+1 💚 javadoc 0m 37s master passed
_ Patch Compile Tests _
+1 💚 mvninstall 3m 41s the patch passed
+1 💚 compile 1m 0s the patch passed
+1 💚 javac 1m 0s the patch passed
+1 💚 shadedjars 6m 57s patch has no errors when building our shaded downstream artifacts.
+1 💚 javadoc 0m 35s the patch passed
_ Other Tests _
-1 ❌ unit 146m 52s hbase-server in the patch failed.
172m 40s
Subsystem Report/Notes
Docker Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2569/3/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR #2569
Optional Tests javac javadoc unit shadedjars compile
uname Linux 293b21117b6f 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / d8de24c
Default Java 1.8.0_232
unit https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2569/3/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt
Test Results https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2569/3/testReport/
Max. process+thread count 4751 (vs. ulimit of 30000)
modules C: hbase-server U: hbase-server
Console output https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2569/3/console
versions git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f)
Powered by Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

@brfrn169
Copy link
Member Author

Let me check the failed UTs.

@Apache-HBase
Copy link

🎊 +1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 1m 8s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+1 💚 hbaseanti 0m 0s Patch does not have any anti-patterns.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
_ master Compile Tests _
+1 💚 mvninstall 4m 22s master passed
+1 💚 checkstyle 1m 16s master passed
+1 💚 spotbugs 2m 12s master passed
_ Patch Compile Tests _
+1 💚 mvninstall 3m 53s the patch passed
+1 💚 checkstyle 1m 10s the patch passed
+1 💚 whitespace 0m 0s The patch has no whitespace issues.
+1 💚 hadoopcheck 18m 47s Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0.
+1 💚 spotbugs 2m 16s the patch passed
_ Other Tests _
+1 💚 asflicense 0m 12s The patch does not generate ASF License warnings.
42m 59s
Subsystem Report/Notes
Docker Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2569/4/artifact/yetus-general-check/output/Dockerfile
GITHUB PR #2569
Optional Tests dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle
uname Linux c9c8c170d1ac 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / 5e3ffb1
Max. process+thread count 84 (vs. ulimit of 30000)
modules C: hbase-server U: hbase-server
Console output https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2569/4/console
versions git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12
Powered by Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase
Copy link

💔 -1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 2m 18s Docker mode activated.
-0 ⚠️ yetus 0m 3s Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚 mvninstall 4m 21s master passed
+1 💚 compile 1m 5s master passed
+1 💚 shadedjars 6m 47s branch has no errors when building our shaded downstream artifacts.
+1 💚 javadoc 0m 44s master passed
_ Patch Compile Tests _
+1 💚 mvninstall 4m 3s the patch passed
+1 💚 compile 1m 5s the patch passed
+1 💚 javac 1m 5s the patch passed
+1 💚 shadedjars 6m 40s patch has no errors when building our shaded downstream artifacts.
+1 💚 javadoc 0m 40s the patch passed
_ Other Tests _
-1 ❌ unit 138m 24s hbase-server in the patch failed.
168m 26s
Subsystem Report/Notes
Docker Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2569/4/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR #2569
Optional Tests javac javadoc unit shadedjars compile
uname Linux dca92352478c 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / 5e3ffb1
Default Java 2020-01-14
unit https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2569/4/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt
Test Results https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2569/4/testReport/
Max. process+thread count 3901 (vs. ulimit of 30000)
modules C: hbase-server U: hbase-server
Console output https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2569/4/console
versions git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f)
Powered by Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase
Copy link

🎊 +1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 0m 32s Docker mode activated.
-0 ⚠️ yetus 0m 3s Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚 mvninstall 3m 34s master passed
+1 💚 compile 1m 0s master passed
+1 💚 shadedjars 7m 49s branch has no errors when building our shaded downstream artifacts.
+1 💚 javadoc 0m 39s master passed
_ Patch Compile Tests _
+1 💚 mvninstall 3m 39s the patch passed
+1 💚 compile 0m 57s the patch passed
+1 💚 javac 0m 57s the patch passed
+1 💚 shadedjars 6m 36s patch has no errors when building our shaded downstream artifacts.
+1 💚 javadoc 0m 38s the patch passed
_ Other Tests _
+1 💚 unit 141m 18s hbase-server in the patch passed.
168m 54s
Subsystem Report/Notes
Docker Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2569/4/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR #2569
Optional Tests javac javadoc unit shadedjars compile
uname Linux aa7135c46b49 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / 5e3ffb1
Default Java 1.8.0_232
Test Results https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2569/4/testReport/
Max. process+thread count 5369 (vs. ulimit of 30000)
modules C: hbase-server U: hbase-server
Console output https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2569/4/console
versions git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f)
Powered by Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

@brfrn169
Copy link
Member Author

brfrn169 commented Oct 23, 2020

I don't think the failed UT in the last QA is related to the patch. The UT was successful locally.

Copy link
Contributor

@Apache9 Apache9 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

@Apache9 Apache9 merged commit 881c92b into apache:master Oct 24, 2020
Apache9 pushed a commit that referenced this pull request Oct 24, 2020
…it region(delete table) (#2569)

Signed-off-by: Duo Zhang <zhangduo@apache.org>
Apache9 pushed a commit that referenced this pull request Oct 24, 2020
…it region(delete table) (#2569)

Signed-off-by: Duo Zhang <zhangduo@apache.org>
Apache9 pushed a commit that referenced this pull request Oct 24, 2020
…it region(delete table) (#2569)

Signed-off-by: Duo Zhang <zhangduo@apache.org>
@@ -348,6 +348,7 @@ protected Flow executeFromState(MasterProcedureEnv env, RegionStateTransitionSta
LOG.error(
"Cannot assign replica region {} because its primary region {} does not exist.",
regionNode.getRegionInfo(), defaultRI);
regionNode.unsetProcedure(this);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a good catch, moving the region out of RIT.

clarax pushed a commit to clarax/hbase that referenced this pull request Nov 15, 2020
…it region(delete table) (apache#2569)

Signed-off-by: Duo Zhang <zhangduo@apache.org>
wchevreuil pushed a commit to wchevreuil/hbase that referenced this pull request May 24, 2021
…it region(delete table) (apache#2569)

Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit da4d639)

Change-Id: Ia75c7678d2273b42bfb8b2ed5ca33a612b5ad74c
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants