Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HBASE-28687 BackupSystemTable#checkSystemTable should ensure system tables are enabled #6018

Merged
merged 2 commits into from
Jun 26, 2024

Conversation

rmdmattingly
Copy link
Contributor

See https://issues.apache.org/jira/browse/HBASE-28687

If the backup system tables become disabled, then we enter a state which the backup client will not recover from. Without manual intervention, every subsequent backup attempt will fail on BackupSystemTable's calls to waitForSystemTable.

This checkSystemTable method already ensures that the tables exist — it should also ensure that the tables are enabled before we await that condition.

Alternatively, we could fast-fail if the tables are disabled rather than awaiting an enabled state that will never occur.

cc @ndimiduk @charlesconnell @hgromer

@Apache-HBase
Copy link

🎊 +1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 0m 46s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+1 💚 hbaseanti 0m 0s Patch does not have any anti-patterns.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
_ master Compile Tests _
+1 💚 mvninstall 4m 11s master passed
+1 💚 compile 0m 39s master passed
+1 💚 checkstyle 0m 13s master passed
+1 💚 spotless 0m 54s branch has no errors when running spotless:check.
+1 💚 spotbugs 0m 37s master passed
_ Patch Compile Tests _
+1 💚 mvninstall 3m 19s the patch passed
+1 💚 compile 0m 29s the patch passed
+1 💚 javac 0m 29s the patch passed
+1 💚 checkstyle 0m 10s the patch passed
+1 💚 whitespace 0m 0s The patch has no whitespace issues.
+1 💚 hadoopcheck 6m 30s Patch does not cause any errors with Hadoop 3.3.6.
+1 💚 spotless 1m 7s patch has no errors when running spotless:check.
+1 💚 spotbugs 0m 52s the patch passed
_ Other Tests _
+1 💚 asflicense 0m 13s The patch does not generate ASF License warnings.
27m 29s
Subsystem Report/Notes
Docker ClientAPI=1.45 ServerAPI=1.45 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6018/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR #6018
Optional Tests dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile
uname Linux 735f00c0c42d 5.4.0-182-generic #202-Ubuntu SMP Fri Apr 26 12:29:36 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / dd694e4
Default Java Eclipse Adoptium-11.0.17+8
Max. process+thread count 80 (vs. ulimit of 30000)
modules C: hbase-backup U: hbase-backup
Console output https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6018/1/console
versions git=2.34.1 maven=3.8.6 spotbugs=4.7.3
Powered by Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase
Copy link

💔 -1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 0m 41s Docker mode activated.
-0 ⚠️ yetus 0m 3s Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚 mvninstall 2m 55s master passed
+1 💚 compile 0m 19s master passed
+1 💚 shadedjars 5m 17s branch has no errors when building our shaded downstream artifacts.
+1 💚 javadoc 0m 16s master passed
_ Patch Compile Tests _
+1 💚 mvninstall 2m 50s the patch passed
+1 💚 compile 0m 19s the patch passed
+1 💚 javac 0m 19s the patch passed
+1 💚 shadedjars 5m 18s patch has no errors when building our shaded downstream artifacts.
+1 💚 javadoc 0m 15s the patch passed
_ Other Tests _
-1 ❌ unit 12m 26s hbase-backup in the patch failed.
31m 36s
Subsystem Report/Notes
Docker ClientAPI=1.45 ServerAPI=1.45 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6018/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR #6018
Optional Tests javac javadoc unit shadedjars compile
uname Linux 6196b7d74ef3 5.4.0-174-generic #193-Ubuntu SMP Thu Mar 7 14:29:28 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / dd694e4
Default Java Eclipse Adoptium-11.0.17+8
unit https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6018/1/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-backup.txt
Test Results https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6018/1/testReport/
Max. process+thread count 3887 (vs. ulimit of 30000)
modules C: hbase-backup U: hbase-backup
Console output https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6018/1/console
versions git=2.34.1 maven=3.8.6
Powered by Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase
Copy link

💔 -1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 0m 31s Docker mode activated.
-0 ⚠️ yetus 0m 2s Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚 mvninstall 3m 24s master passed
+1 💚 compile 0m 17s master passed
+1 💚 shadedjars 5m 45s branch has no errors when building our shaded downstream artifacts.
+1 💚 javadoc 0m 14s master passed
_ Patch Compile Tests _
+1 💚 mvninstall 2m 57s the patch passed
+1 💚 compile 0m 17s the patch passed
+1 💚 javac 0m 17s the patch passed
+1 💚 shadedjars 5m 44s patch has no errors when building our shaded downstream artifacts.
+1 💚 javadoc 0m 12s the patch passed
_ Other Tests _
-1 ❌ unit 11m 48s hbase-backup in the patch failed.
32m 19s
Subsystem Report/Notes
Docker ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6018/1/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile
GITHUB PR #6018
Optional Tests javac javadoc unit shadedjars compile
uname Linux 0511fd632856 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / dd694e4
Default Java Eclipse Adoptium-17.0.10+7
unit https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6018/1/artifact/yetus-jdk17-hadoop3-check/output/patch-unit-hbase-backup.txt
Test Results https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6018/1/testReport/
Max. process+thread count 3897 (vs. ulimit of 30000)
modules C: hbase-backup U: hbase-backup
Console output https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6018/1/console
versions git=2.34.1 maven=3.8.6
Powered by Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase
Copy link

💔 -1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 0m 45s Docker mode activated.
-0 ⚠️ yetus 0m 3s Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚 mvninstall 3m 56s master passed
+1 💚 compile 0m 17s master passed
+1 💚 shadedjars 6m 49s branch has no errors when building our shaded downstream artifacts.
+1 💚 javadoc 0m 16s master passed
_ Patch Compile Tests _
+1 💚 mvninstall 3m 15s the patch passed
+1 💚 compile 0m 21s the patch passed
+1 💚 javac 0m 21s the patch passed
+1 💚 shadedjars 5m 55s patch has no errors when building our shaded downstream artifacts.
+1 💚 javadoc 0m 12s the patch passed
_ Other Tests _
-1 ❌ unit 13m 15s hbase-backup in the patch failed.
36m 8s
Subsystem Report/Notes
Docker ClientAPI=1.45 ServerAPI=1.45 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6018/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR #6018
Optional Tests javac javadoc unit shadedjars compile
uname Linux 61289fbcd5ed 5.4.0-182-generic #202-Ubuntu SMP Fri Apr 26 12:29:36 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / dd694e4
Default Java Temurin-1.8.0_352-b08
unit https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6018/1/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-backup.txt
Test Results https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6018/1/testReport/
Max. process+thread count 3390 (vs. ulimit of 30000)
modules C: hbase-backup U: hbase-backup
Console output https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6018/1/console
versions git=2.34.1 maven=3.8.6
Powered by Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@rmdmattingly
Copy link
Contributor Author

rmdmattingly commented Jun 21, 2024

Hmm. Test failures could be related. Looking into it

edit: they pass on my machine 😓

@Apache-HBase
Copy link

🎊 +1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 0m 46s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+1 💚 hbaseanti 0m 0s Patch does not have any anti-patterns.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
_ master Compile Tests _
+1 💚 mvninstall 4m 15s master passed
+1 💚 compile 0m 39s master passed
+1 💚 checkstyle 0m 13s master passed
+1 💚 spotless 0m 57s branch has no errors when running spotless:check.
+1 💚 spotbugs 0m 37s master passed
_ Patch Compile Tests _
+1 💚 mvninstall 3m 17s the patch passed
+1 💚 compile 0m 26s the patch passed
+1 💚 javac 0m 26s the patch passed
+1 💚 checkstyle 0m 10s the patch passed
+1 💚 whitespace 0m 0s The patch has no whitespace issues.
+1 💚 hadoopcheck 6m 39s Patch does not cause any errors with Hadoop 3.3.6.
+1 💚 spotless 1m 9s patch has no errors when running spotless:check.
+1 💚 spotbugs 0m 53s the patch passed
_ Other Tests _
+1 💚 asflicense 0m 12s The patch does not generate ASF License warnings.
27m 44s
Subsystem Report/Notes
Docker ClientAPI=1.45 ServerAPI=1.45 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6018/2/artifact/yetus-general-check/output/Dockerfile
GITHUB PR #6018
Optional Tests dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile
uname Linux 156b0629edcd 5.4.0-182-generic #202-Ubuntu SMP Fri Apr 26 12:29:36 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / dd694e4
Default Java Eclipse Adoptium-11.0.17+8
Max. process+thread count 76 (vs. ulimit of 30000)
modules C: hbase-backup U: hbase-backup
Console output https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6018/2/console
versions git=2.34.1 maven=3.8.6 spotbugs=4.7.3
Powered by Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase
Copy link

💔 -1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 0m 25s Docker mode activated.
-0 ⚠️ yetus 0m 2s Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚 mvninstall 3m 18s master passed
+1 💚 compile 0m 17s master passed
+1 💚 shadedjars 5m 56s branch has no errors when building our shaded downstream artifacts.
+1 💚 javadoc 0m 14s master passed
_ Patch Compile Tests _
+1 💚 mvninstall 2m 58s the patch passed
+1 💚 compile 0m 17s the patch passed
+1 💚 javac 0m 17s the patch passed
+1 💚 shadedjars 5m 54s patch has no errors when building our shaded downstream artifacts.
+1 💚 javadoc 0m 12s the patch passed
_ Other Tests _
-1 ❌ unit 10m 57s hbase-backup in the patch failed.
31m 26s
Subsystem Report/Notes
Docker ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6018/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR #6018
Optional Tests javac javadoc unit shadedjars compile
uname Linux 459a8a436a0c 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / dd694e4
Default Java Eclipse Adoptium-11.0.17+8
unit https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6018/2/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-backup.txt
Test Results https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6018/2/testReport/
Max. process+thread count 3663 (vs. ulimit of 30000)
modules C: hbase-backup U: hbase-backup
Console output https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6018/2/console
versions git=2.34.1 maven=3.8.6
Powered by Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase
Copy link

💔 -1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 0m 46s Docker mode activated.
-0 ⚠️ yetus 0m 2s Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚 mvninstall 3m 58s master passed
+1 💚 compile 0m 18s master passed
+1 💚 shadedjars 6m 47s branch has no errors when building our shaded downstream artifacts.
+1 💚 javadoc 0m 15s master passed
_ Patch Compile Tests _
+1 💚 mvninstall 3m 20s the patch passed
+1 💚 compile 0m 19s the patch passed
+1 💚 javac 0m 19s the patch passed
+1 💚 shadedjars 5m 55s patch has no errors when building our shaded downstream artifacts.
+1 💚 javadoc 0m 12s the patch passed
_ Other Tests _
-1 ❌ unit 12m 57s hbase-backup in the patch failed.
35m 53s
Subsystem Report/Notes
Docker ClientAPI=1.45 ServerAPI=1.45 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6018/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR #6018
Optional Tests javac javadoc unit shadedjars compile
uname Linux 416d5cf8691b 5.4.0-182-generic #202-Ubuntu SMP Fri Apr 26 12:29:36 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / dd694e4
Default Java Temurin-1.8.0_352-b08
unit https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6018/2/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-backup.txt
Test Results https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6018/2/testReport/
Max. process+thread count 3688 (vs. ulimit of 30000)
modules C: hbase-backup U: hbase-backup
Console output https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6018/2/console
versions git=2.34.1 maven=3.8.6
Powered by Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@rmdmattingly
Copy link
Contributor Author

rmdmattingly commented Jun 21, 2024

This same test is failing across multiple PRs: #6017
Must be flaky, because it doesn't make sense for either changeset to cause this. Looking into a fix

@Apache-HBase
Copy link

💔 -1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 0m 30s Docker mode activated.
-0 ⚠️ yetus 0m 3s Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚 mvninstall 4m 10s master passed
+1 💚 compile 0m 23s master passed
+1 💚 shadedjars 6m 12s branch has no errors when building our shaded downstream artifacts.
+1 💚 javadoc 0m 17s master passed
_ Patch Compile Tests _
+1 💚 mvninstall 4m 9s the patch passed
+1 💚 compile 0m 22s the patch passed
+1 💚 javac 0m 22s the patch passed
+1 💚 shadedjars 6m 16s patch has no errors when building our shaded downstream artifacts.
+1 💚 javadoc 0m 14s the patch passed
_ Other Tests _
-1 ❌ unit 14m 25s hbase-backup in the patch failed.
38m 12s
Subsystem Report/Notes
Docker ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6018/2/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile
GITHUB PR #6018
Optional Tests javac javadoc unit shadedjars compile
uname Linux 4def8dd33e84 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / dd694e4
Default Java Eclipse Adoptium-17.0.10+7
unit https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6018/2/artifact/yetus-jdk17-hadoop3-check/output/patch-unit-hbase-backup.txt
Test Results https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6018/2/testReport/
Max. process+thread count 3714 (vs. ulimit of 30000)
modules C: hbase-backup U: hbase-backup
Console output https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6018/2/console
versions git=2.34.1 maven=3.8.6
Powered by Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@rmdmattingly
Copy link
Contributor Author

rmdmattingly commented Jun 21, 2024

Okay, I can reproduce the failures if I run all backups tests locally rather than just the one. I bet this has something to do with all of the inheritance going on in these unit tests

Copy link
Contributor

@bbeaudreault bbeaudreault left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO enabling them makes sense since kicking off a backup is an implicitly request to use the backup state. It's also consistent with the other states handled here.

The only other thing I'd say is, do we need to handle exceptions? For example if we check that the table is disabled and it is, so we go to enable and it's already enabled. I've seen this problem in other areas where we handle table lifecycle automatically. Basically if multiple callers conflict, it throws an exception that could easily have been handled.

If this is already protected by the backup lock, then may not be necessary.

Comment on lines +121 to +151
@Before
public void ensurePreviousBackupTestsAreCleanedUp() throws Exception {
// Every operation here may not be necessary for any given test,
// some often being no-ops. the goal is to help ensure atomicity
// of that tests that implement TestBackupBase
try (BackupAdmin backupAdmin = getBackupAdmin()) {
backupManager.finishBackupSession();
backupAdmin.listBackupSets().forEach(backupSet -> {
try {
backupAdmin.deleteBackupSet(backupSet.getName());
} catch (IOException ignored) {
}
});
} catch (Exception ignored) {
}
Arrays.stream(TEST_UTIL.getAdmin().listTableNames())
.filter(tableName -> !tableName.isSystemTable()).forEach(tableName -> {
try {
TEST_UTIL.truncateTable(tableName);
} catch (IOException ignored) {
}
});
TEST_UTIL.getMiniHBaseCluster().getRegionServerThreads().forEach(rst -> {
try {
LogRoller walRoller = rst.getRegionServer().getWalRoller();
walRoller.requestRollAll();
walRoller.waitUntilWalRollFinished();
} catch (Exception ignored) {
}
});
}
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After some playing around, this seemed like a pretty good way to get TestBackupMerge, and others, to consistently pass even when ran in bulk. Another option would be to get rid of the TestBackupMerge inheritance, but that would be a really annoying change to implement, and would probably be significantly slower in the build pipeline

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, let's try it.

For what it's worth, I've had success with refactoring inheritance out of these junit4 tests by moving shared logic into an ExternalResource implementation. It doesn't let you reuse @Test implementations, but it does make implementing before/after logic more composable.

@rmdmattingly
Copy link
Contributor Author

Agreed with the feedback, I added handling of TableNotDisabled exceptions

@Apache-HBase
Copy link

🎊 +1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 45s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 0s codespell was not available.
+0 🆗 detsecrets 0m 0s detect-secrets was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 hbaseanti 0m 0s Patch does not have any anti-patterns.
_ master Compile Tests _
+1 💚 mvninstall 3m 43s master passed
+1 💚 compile 0m 33s master passed
+1 💚 checkstyle 0m 11s master passed
+1 💚 spotbugs 0m 36s master passed
+1 💚 spotless 0m 59s branch has no errors when running spotless:check.
_ Patch Compile Tests _
+1 💚 mvninstall 3m 11s the patch passed
+1 💚 compile 0m 32s the patch passed
+1 💚 javac 0m 32s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
+1 💚 checkstyle 0m 13s the patch passed
+1 💚 spotbugs 0m 49s the patch passed
+1 💚 hadoopcheck 6m 0s Patch does not cause any errors with Hadoop 3.3.6.
+1 💚 spotless 0m 55s patch has no errors when running spotless:check.
_ Other Tests _
+1 💚 asflicense 0m 11s The patch does not generate ASF License warnings.
26m 36s
Subsystem Report/Notes
Docker ClientAPI=1.45 ServerAPI=1.45 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6018/3/artifact/yetus-general-check/output/Dockerfile
GITHUB PR #6018
Optional Tests dupname asflicense javac spotbugs checkstyle codespell detsecrets compile hadoopcheck hbaseanti spotless
uname Linux 4303e2075707 5.4.0-182-generic #202-Ubuntu SMP Fri Apr 26 12:29:36 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / c684a59
Default Java Eclipse Adoptium-11.0.17+8
Max. process+thread count 79 (vs. ulimit of 30000)
modules C: hbase-backup U: hbase-backup
Console output https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6018/3/console
versions git=2.34.1 maven=3.8.6 spotbugs=4.7.3
Powered by Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase
Copy link

🎊 +1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 17s Docker mode activated.
-0 ⚠️ yetus 0m 2s Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚 mvninstall 2m 38s master passed
+1 💚 compile 0m 17s master passed
+1 💚 javadoc 0m 15s master passed
+1 💚 shadedjars 5m 9s branch has no errors when building our shaded downstream artifacts.
_ Patch Compile Tests _
+1 💚 mvninstall 2m 28s the patch passed
+1 💚 compile 0m 17s the patch passed
+1 💚 javac 0m 17s the patch passed
+1 💚 javadoc 0m 13s the patch passed
+1 💚 shadedjars 5m 8s patch has no errors when building our shaded downstream artifacts.
_ Other Tests _
+1 💚 unit 11m 30s hbase-backup in the patch passed.
29m 8s
Subsystem Report/Notes
Docker ClientAPI=1.45 ServerAPI=1.45 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6018/3/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR #6018
Optional Tests javac javadoc unit compile shadedjars
uname Linux e40eba7f3ff3 5.4.0-182-generic #202-Ubuntu SMP Fri Apr 26 12:29:36 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / c684a59
Default Java Temurin-1.8.0_352-b08
Test Results https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6018/3/testReport/
Max. process+thread count 3089 (vs. ulimit of 30000)
modules C: hbase-backup U: hbase-backup
Console output https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6018/3/console
versions git=2.34.1 maven=3.8.6
Powered by Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase
Copy link

🎊 +1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 25s Docker mode activated.
-0 ⚠️ yetus 0m 3s Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚 mvninstall 3m 18s master passed
+1 💚 compile 0m 17s master passed
+1 💚 javadoc 0m 15s master passed
+1 💚 shadedjars 5m 54s branch has no errors when building our shaded downstream artifacts.
_ Patch Compile Tests _
+1 💚 mvninstall 3m 0s the patch passed
+1 💚 compile 0m 18s the patch passed
+1 💚 javac 0m 18s the patch passed
+1 💚 javadoc 0m 13s the patch passed
+1 💚 shadedjars 6m 27s patch has no errors when building our shaded downstream artifacts.
_ Other Tests _
+1 💚 unit 12m 2s hbase-backup in the patch passed.
33m 9s
Subsystem Report/Notes
Docker ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6018/3/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR #6018
Optional Tests javac javadoc unit compile shadedjars
uname Linux a85fbb1ed8e9 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / c684a59
Default Java Eclipse Adoptium-11.0.17+8
Test Results https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6018/3/testReport/
Max. process+thread count 3700 (vs. ulimit of 30000)
modules C: hbase-backup U: hbase-backup
Console output https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6018/3/console
versions git=2.34.1 maven=3.8.6
Powered by Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase
Copy link

🎊 +1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 40s Docker mode activated.
-0 ⚠️ yetus 0m 4s Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚 mvninstall 3m 22s master passed
+1 💚 compile 0m 22s master passed
+1 💚 javadoc 0m 20s master passed
+1 💚 shadedjars 6m 29s branch has no errors when building our shaded downstream artifacts.
_ Patch Compile Tests _
+1 💚 mvninstall 3m 43s the patch passed
+1 💚 compile 0m 22s the patch passed
+1 💚 javac 0m 22s the patch passed
+1 💚 javadoc 0m 14s the patch passed
+1 💚 shadedjars 6m 16s patch has no errors when building our shaded downstream artifacts.
_ Other Tests _
+1 💚 unit 11m 33s hbase-backup in the patch passed.
34m 27s
Subsystem Report/Notes
Docker ClientAPI=1.45 ServerAPI=1.45 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6018/3/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile
GITHUB PR #6018
Optional Tests javac javadoc unit compile shadedjars
uname Linux 0064c05571c5 5.4.0-186-generic #206-Ubuntu SMP Fri Apr 26 12:31:10 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / c684a59
Default Java Eclipse Adoptium-17.0.10+7
Test Results https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6018/3/testReport/
Max. process+thread count 4014 (vs. ulimit of 30000)
modules C: hbase-backup U: hbase-backup
Console output https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6018/3/console
versions git=2.34.1 maven=3.8.6
Powered by Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

Comment on lines +121 to +151
@Before
public void ensurePreviousBackupTestsAreCleanedUp() throws Exception {
// Every operation here may not be necessary for any given test,
// some often being no-ops. the goal is to help ensure atomicity
// of that tests that implement TestBackupBase
try (BackupAdmin backupAdmin = getBackupAdmin()) {
backupManager.finishBackupSession();
backupAdmin.listBackupSets().forEach(backupSet -> {
try {
backupAdmin.deleteBackupSet(backupSet.getName());
} catch (IOException ignored) {
}
});
} catch (Exception ignored) {
}
Arrays.stream(TEST_UTIL.getAdmin().listTableNames())
.filter(tableName -> !tableName.isSystemTable()).forEach(tableName -> {
try {
TEST_UTIL.truncateTable(tableName);
} catch (IOException ignored) {
}
});
TEST_UTIL.getMiniHBaseCluster().getRegionServerThreads().forEach(rst -> {
try {
LogRoller walRoller = rst.getRegionServer().getWalRoller();
walRoller.requestRollAll();
walRoller.waitUntilWalRollFinished();
} catch (Exception ignored) {
}
});
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, let's try it.

For what it's worth, I've had success with refactoring inheritance out of these junit4 tests by moving shared logic into an ExternalResource implementation. It doesn't let you reuse @Test implementations, but it does make implementing before/after logic more composable.

@ndimiduk ndimiduk merged commit 7e8fd07 into apache:master Jun 26, 2024
1 check passed
@ndimiduk ndimiduk deleted the HBASE-28687 branch June 26, 2024 12:40
ndimiduk pushed a commit to ndimiduk/hbase that referenced this pull request Jun 26, 2024
…ables are enabled (apache#6018)

Co-authored-by: Ray Mattingly <rmattingly@hubspot.com>
Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org>
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
ndimiduk pushed a commit to ndimiduk/hbase that referenced this pull request Jun 26, 2024
…ables are enabled (apache#6018)

Co-authored-by: Ray Mattingly <rmattingly@hubspot.com>
Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org>
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
ndimiduk pushed a commit to ndimiduk/hbase that referenced this pull request Jun 26, 2024
…ables are enabled (apache#6018)

Co-authored-by: Ray Mattingly <rmattingly@hubspot.com>
Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org>
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
ndimiduk pushed a commit that referenced this pull request Jun 27, 2024
…ables are enabled (#6018)

Co-authored-by: Ray Mattingly <rmattingly@hubspot.com>
Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org>
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
ndimiduk pushed a commit that referenced this pull request Jun 27, 2024
…ables are enabled (#6018)

Co-authored-by: Ray Mattingly <rmattingly@hubspot.com>
Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org>
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
ndimiduk pushed a commit that referenced this pull request Jun 27, 2024
…ables are enabled (#6018)

Co-authored-by: Ray Mattingly <rmattingly@hubspot.com>
Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org>
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
rmdmattingly added a commit to HubSpot/hbase that referenced this pull request Jun 28, 2024
…ables are enabled (apache#6018)

Co-authored-by: Ray Mattingly <rmattingly@hubspot.com>
Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org>
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
rmdmattingly added a commit to HubSpot/hbase that referenced this pull request Jun 28, 2024
…ables are enabled (apache#6018)

Co-authored-by: Ray Mattingly <rmattingly@hubspot.com>
Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org>
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
rmdmattingly added a commit to HubSpot/hbase that referenced this pull request Jul 8, 2024
…ld ensure system tables are enabled (apache#6018) (#102)

Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org>
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Co-authored-by: Ray Mattingly <rmattingly@hubspot.com>
rmdmattingly added a commit to HubSpot/hbase that referenced this pull request Jul 8, 2024
…ld ensure system tables are enabled (apache#6018) (#103)

Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org>
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Co-authored-by: Ray Mattingly <rmattingly@hubspot.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants