HBASE-28625 ExportSnapshot should verify checksums for the source file and the target file #5950

2005hithlj · 2024-05-29T11:44:32Z

https://issues.apache.org/jira/browse/HBASE-28625

Apache-HBase · 2024-05-29T12:19:14Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 17s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ master Compile Tests _
+1 💚	mvninstall	3m 46s	master passed
+1 💚	compile	0m 44s	master passed
+1 💚	checkstyle	0m 13s	master passed
+1 💚	spotless	0m 54s	branch has no errors when running spotless:check.
+1 💚	spotbugs	0m 43s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	3m 36s	the patch passed
+1 💚	compile	0m 44s	the patch passed
+1 💚	javac	0m 44s	the patch passed
+1 💚	checkstyle	0m 12s	hbase-mapreduce: The patch generated 0 new + 6 unchanged - 3 fixed = 6 total (was 9)
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	hadoopcheck	6m 37s	Patch does not cause any errors with Hadoop 3.3.6.
+1 💚	spotless	0m 56s	patch has no errors when running spotless:check.
+1 💚	spotbugs	0m 52s	the patch passed
		_ Other Tests _
+1 💚	asflicense	0m 11s	The patch does not generate ASF License warnings.
		28m 37s

Subsystem	Report/Notes
Docker	ClientAPI=1.45 ServerAPI=1.45 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5950/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#5950
Optional Tests	dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile
uname	Linux 50e38f9a8815 5.4.0-182-generic #202-Ubuntu SMP Fri Apr 26 12:29:36 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `771e1d9`
Default Java	Eclipse Adoptium-11.0.17+8
Max. process+thread count	79 (vs. ulimit of 30000)
modules	C: hbase-mapreduce U: hbase-mapreduce
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5950/1/console
versions	git=2.34.1 maven=3.8.6 spotbugs=4.7.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2024-05-29T12:26:42Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 27s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	2m 48s	master passed
+1 💚	compile	0m 16s	master passed
+1 💚	shadedjars	5m 40s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 13s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	2m 26s	the patch passed
+1 💚	compile	0m 16s	the patch passed
+1 💚	javac	0m 16s	the patch passed
+1 💚	shadedjars	5m 42s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 11s	the patch passed
		_ Other Tests _
-1 ❌	unit	17m 11s	hbase-mapreduce in the patch failed.
		36m 7s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5950/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR	#5950
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux a0e03c78534c 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `771e1d9`
Default Java	Temurin-1.8.0_352-b08
unit	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5950/1/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-mapreduce.txt
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5950/1/testReport/
Max. process+thread count	2901 (vs. ulimit of 30000)
modules	C: hbase-mapreduce U: hbase-mapreduce
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5950/1/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2024-05-29T12:33:48Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 38s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	4m 1s	master passed
+1 💚	compile	0m 25s	master passed
+1 💚	shadedjars	6m 16s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 18s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	3m 41s	the patch passed
+1 💚	compile	0m 23s	the patch passed
+1 💚	javac	0m 23s	the patch passed
+1 💚	shadedjars	6m 14s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 14s	the patch passed
		_ Other Tests _
-1 ❌	unit	19m 47s	hbase-mapreduce in the patch failed.
		43m 7s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5950/1/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile
GITHUB PR	#5950
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 00eea139e56c 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `771e1d9`
Default Java	Eclipse Adoptium-17.0.10+7
unit	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5950/1/artifact/yetus-jdk17-hadoop3-check/output/patch-unit-hbase-mapreduce.txt
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5950/1/testReport/
Max. process+thread count	2956 (vs. ulimit of 30000)
modules	C: hbase-mapreduce U: hbase-mapreduce
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5950/1/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2024-05-29T12:41:43Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 36s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	3m 45s	master passed
+1 💚	compile	0m 26s	master passed
+1 💚	shadedjars	6m 23s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 18s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	3m 39s	the patch passed
+1 💚	compile	0m 24s	the patch passed
+1 💚	javac	0m 24s	the patch passed
+1 💚	shadedjars	6m 59s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 16s	the patch passed
		_ Other Tests _
-1 ❌	unit	27m 20s	hbase-mapreduce in the patch failed.
		51m 8s

Subsystem	Report/Notes
Docker	ClientAPI=1.45 ServerAPI=1.45 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5950/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#5950
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 683563b80106 5.4.0-182-generic #202-Ubuntu SMP Fri Apr 26 12:29:36 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `771e1d9`
Default Java	Eclipse Adoptium-11.0.17+8
unit	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5950/1/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-mapreduce.txt
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5950/1/testReport/
Max. process+thread count	2569 (vs. ulimit of 30000)
modules	C: hbase-mapreduce U: hbase-mapreduce
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5950/1/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2024-05-30T04:42:35Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 35s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ master Compile Tests _
+1 💚	mvninstall	3m 2s	master passed
+1 💚	compile	0m 36s	master passed
+1 💚	checkstyle	0m 14s	master passed
+1 💚	spotless	0m 45s	branch has no errors when running spotless:check.
+1 💚	spotbugs	0m 34s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	2m 44s	the patch passed
+1 💚	compile	0m 33s	the patch passed
+1 💚	javac	0m 33s	the patch passed
+1 💚	checkstyle	0m 12s	hbase-mapreduce: The patch generated 0 new + 6 unchanged - 3 fixed = 6 total (was 9)
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	hadoopcheck	4m 58s	Patch does not cause any errors with Hadoop 3.3.6.
+1 💚	spotless	0m 43s	patch has no errors when running spotless:check.
+1 💚	spotbugs	0m 39s	the patch passed
		_ Other Tests _
+1 💚	asflicense	0m 11s	The patch does not generate ASF License warnings.
		22m 3s

Subsystem	Report/Notes
Docker	ClientAPI=1.45 ServerAPI=1.45 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5950/2/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#5950
Optional Tests	dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile
uname	Linux cb277341ddb0 5.4.0-182-generic #202-Ubuntu SMP Fri Apr 26 12:29:36 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `c4a7606`
Default Java	Eclipse Adoptium-11.0.17+8
Max. process+thread count	79 (vs. ulimit of 30000)
modules	C: hbase-mapreduce U: hbase-mapreduce
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5950/2/console
versions	git=2.34.1 maven=3.8.6 spotbugs=4.7.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2024-05-30T04:54:49Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 28s	Docker mode activated.
-0 ⚠️	yetus	0m 2s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	2m 24s	master passed
+1 💚	compile	0m 17s	master passed
+1 💚	shadedjars	5m 40s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 11s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	2m 25s	the patch passed
+1 💚	compile	0m 16s	the patch passed
+1 💚	javac	0m 16s	the patch passed
+1 💚	shadedjars	5m 44s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 11s	the patch passed
		_ Other Tests _
-1 ❌	unit	15m 33s	hbase-mapreduce in the patch failed.
		34m 16s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5950/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR	#5950
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux c2008fabca80 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `c4a7606`
Default Java	Temurin-1.8.0_352-b08
unit	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5950/2/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-mapreduce.txt
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5950/2/testReport/
Max. process+thread count	2892 (vs. ulimit of 30000)
modules	C: hbase-mapreduce U: hbase-mapreduce
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5950/2/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2024-05-30T04:56:11Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 31s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	3m 19s	master passed
+1 💚	compile	0m 21s	master passed
+1 💚	shadedjars	6m 6s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 14s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	3m 18s	the patch passed
+1 💚	compile	0m 19s	the patch passed
+1 💚	javac	0m 19s	the patch passed
+1 💚	shadedjars	5m 40s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 12s	the patch passed
		_ Other Tests _
-1 ❌	unit	14m 28s	hbase-mapreduce in the patch failed.
		35m 36s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5950/2/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile
GITHUB PR	#5950
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 1694add21e70 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `c4a7606`
Default Java	Eclipse Adoptium-17.0.10+7
unit	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5950/2/artifact/yetus-jdk17-hadoop3-check/output/patch-unit-hbase-mapreduce.txt
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5950/2/testReport/
Max. process+thread count	3136 (vs. ulimit of 30000)
modules	C: hbase-mapreduce U: hbase-mapreduce
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5950/2/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2024-05-30T04:56:33Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 45s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	3m 12s	master passed
+1 💚	compile	0m 21s	master passed
+1 💚	shadedjars	5m 14s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 16s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	2m 47s	the patch passed
+1 💚	compile	0m 20s	the patch passed
+1 💚	javac	0m 20s	the patch passed
+1 💚	shadedjars	5m 11s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 15s	the patch passed
		_ Other Tests _
-1 ❌	unit	16m 21s	hbase-mapreduce in the patch failed.
		35m 57s

Subsystem	Report/Notes
Docker	ClientAPI=1.45 ServerAPI=1.45 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5950/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#5950
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux f542a28ad1b7 5.4.0-174-generic #193-Ubuntu SMP Thu Mar 7 14:29:28 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `c4a7606`
Default Java	Eclipse Adoptium-11.0.17+8
unit	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5950/2/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-mapreduce.txt
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5950/2/testReport/
Max. process+thread count	3173 (vs. ulimit of 30000)
modules	C: hbase-mapreduce U: hbase-mapreduce
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5950/2/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache9 · 2024-05-30T06:32:35Z

hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/snapshot/ExportSnapshot.java

@@ -538,21 +537,47 @@ private FileChecksum getFileChecksum(final FileSystem fs, final Path path) {
     * Check if the two files are equal by looking at the file length, and at the checksum (if user
     * has specified the verifyChecksum flag).
     */
-    private boolean sameFile(final FileStatus inputStat, final FileStatus outputStat) {
+    private boolean sameFile(final FileStatus inputStat, final FileStatus outputStat,
+      final boolean force) throws IOException {


The parameter name 'force' seems a bit confusing here...

Maybe a better choice is to introduce another method called verifyCopyResult?

OK sir, I will have a new commit soon.

…e and the target file

Apache-HBase · 2024-05-30T12:17:12Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 40s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ master Compile Tests _
+1 💚	mvninstall	3m 24s	master passed
+1 💚	compile	0m 42s	master passed
+1 💚	checkstyle	0m 14s	master passed
+1 💚	spotless	0m 58s	branch has no errors when running spotless:check.
+1 💚	spotbugs	0m 44s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	3m 32s	the patch passed
+1 💚	compile	0m 41s	the patch passed
+1 💚	javac	0m 41s	the patch passed
+1 💚	checkstyle	0m 13s	the patch passed
+1 💚	whitespace	0m 1s	The patch has no whitespace issues.
+1 💚	hadoopcheck	6m 6s	Patch does not cause any errors with Hadoop 3.3.6.
+1 💚	spotless	0m 52s	patch has no errors when running spotless:check.
+1 💚	spotbugs	0m 53s	the patch passed
		_ Other Tests _
+1 💚	asflicense	0m 9s	The patch does not generate ASF License warnings.
		26m 35s

Subsystem	Report/Notes
Docker	ClientAPI=1.45 ServerAPI=1.45 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5950/3/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#5950
Optional Tests	dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile
uname	Linux e9bb2a78269b 5.4.0-182-generic #202-Ubuntu SMP Fri Apr 26 12:29:36 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `cd0b29c`
Default Java	Eclipse Adoptium-11.0.17+8
Max. process+thread count	78 (vs. ulimit of 30000)
modules	C: hbase-mapreduce U: hbase-mapreduce
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5950/3/console
versions	git=2.34.1 maven=3.8.6 spotbugs=4.7.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2024-05-30T12:27:37Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 15s	Docker mode activated.
-0 ⚠️	yetus	0m 2s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	3m 57s	master passed
+1 💚	compile	0m 25s	master passed
+1 💚	shadedjars	6m 8s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 16s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	2m 48s	the patch passed
+1 💚	compile	0m 21s	the patch passed
+1 💚	javac	0m 21s	the patch passed
+1 💚	shadedjars	6m 21s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 15s	the patch passed
		_ Other Tests _
+1 💚	unit	15m 2s	hbase-mapreduce in the patch passed.
		37m 1s

Subsystem	Report/Notes
Docker	ClientAPI=1.45 ServerAPI=1.45 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5950/3/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile
GITHUB PR	#5950
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux d131314fc068 5.4.0-182-generic #202-Ubuntu SMP Fri Apr 26 12:29:36 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `cd0b29c`
Default Java	Eclipse Adoptium-17.0.10+7
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5950/3/testReport/
Max. process+thread count	3370 (vs. ulimit of 30000)
modules	C: hbase-mapreduce U: hbase-mapreduce
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5950/3/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2024-05-30T12:28:00Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 10s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	2m 43s	master passed
+1 💚	compile	0m 19s	master passed
+1 💚	shadedjars	5m 11s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 15s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	2m 27s	the patch passed
+1 💚	compile	0m 18s	the patch passed
+1 💚	javac	0m 18s	the patch passed
+1 💚	shadedjars	5m 7s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 14s	the patch passed
		_ Other Tests _
+1 💚	unit	19m 37s	hbase-mapreduce in the patch passed.
		37m 29s

Subsystem	Report/Notes
Docker	ClientAPI=1.45 ServerAPI=1.45 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5950/3/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR	#5950
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux afd80e79d2a8 5.4.0-182-generic #202-Ubuntu SMP Fri Apr 26 12:29:36 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `cd0b29c`
Default Java	Temurin-1.8.0_352-b08
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5950/3/testReport/
Max. process+thread count	2848 (vs. ulimit of 30000)
modules	C: hbase-mapreduce U: hbase-mapreduce
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5950/3/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2024-05-30T12:44:44Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 40s	Docker mode activated.
-0 ⚠️	yetus	0m 2s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	4m 49s	master passed
+1 💚	compile	0m 38s	master passed
+1 💚	shadedjars	7m 56s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 20s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	4m 12s	the patch passed
+1 💚	compile	0m 24s	the patch passed
+1 💚	javac	0m 24s	the patch passed
+1 💚	shadedjars	9m 0s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 20s	the patch passed
		_ Other Tests _
+1 💚	unit	24m 11s	hbase-mapreduce in the patch passed.
		54m 3s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5950/3/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#5950
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 5d6a82efd887 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `cd0b29c`
Default Java	Eclipse Adoptium-11.0.17+8
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5950/3/testReport/
Max. process+thread count	2903 (vs. ulimit of 30000)
modules	C: hbase-mapreduce U: hbase-mapreduce
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5950/3/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

…e and the target file (#5950)

jojochuang · 2024-06-07T21:03:59Z

I'm late to this, but from a file system perspective, we should be more graceful about it because checksums between file systems are not necessarily comparable.

For example, if you export snapshot from HDFS to S3, The file checksum are simply different regardless. It would be nice to have a flag to skip the file checksum check, similar to the hdfs distcp -skipcrc option, or skip if the source and destination are on different FS.

Apache9 · 2024-06-08T01:21:29Z

Ping @2005hithlj . Checked the code, at least if the FileChecksums are computed with different algorithm, we should not compare them...

2005hithlj · 2024-06-12T01:31:43Z

@Apache9 @jojochuang OK, sirs. I will open a new issue to solve this problem.

2005hithlj force-pushed the HBASE-28625 branch from b65ea3f to 78979d9 Compare May 30, 2024 04:18

Apache9 reviewed May 30, 2024

View reviewed changes

2005hithlj force-pushed the HBASE-28625 branch from 78979d9 to 38d3f6d Compare May 30, 2024 11:35

HBASE-28625 ExportSnapshot should verify checksums for the source fil…

1cacdeb

…e and the target file

2005hithlj force-pushed the HBASE-28625 branch from 38d3f6d to 1cacdeb Compare May 30, 2024 11:41

Apache9 approved these changes Jun 3, 2024

View reviewed changes

2005hithlj merged commit 070bca1 into apache:master Jun 6, 2024
1 check passed

2005hithlj added a commit that referenced this pull request Jun 6, 2024

HBASE-28625 ExportSnapshot should verify checksums for the source fil…

4974229

…e and the target file (#5950)

2005hithlj added a commit that referenced this pull request Jun 6, 2024

HBASE-28625 ExportSnapshot should verify checksums for the source fil…

e3aa767

…e and the target file (#5950)

2005hithlj added a commit that referenced this pull request Jun 6, 2024

HBASE-28625 ExportSnapshot should verify checksums for the source fil…

2eacee3

…e and the target file (#5950)

2005hithlj added a commit that referenced this pull request Jun 6, 2024

HBASE-28625 ExportSnapshot should verify checksums for the source fil…

00b1d07

…e and the target file (#5950)

2005hithlj added a commit that referenced this pull request Jun 6, 2024

HBASE-28625 ExportSnapshot should verify checksums for the source fil…

97a3723

…e and the target file (#5950)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

HBASE-28625 ExportSnapshot should verify checksums for the source file and the target file #5950

HBASE-28625 ExportSnapshot should verify checksums for the source file and the target file #5950

2005hithlj commented May 29, 2024

Apache-HBase commented May 29, 2024

Apache-HBase commented May 29, 2024

Apache-HBase commented May 29, 2024

Apache-HBase commented May 29, 2024

Apache-HBase commented May 30, 2024

Apache-HBase commented May 30, 2024

Apache-HBase commented May 30, 2024

Apache-HBase commented May 30, 2024

Apache9 May 30, 2024

2005hithlj May 30, 2024

Apache-HBase commented May 30, 2024

Apache-HBase commented May 30, 2024

Apache-HBase commented May 30, 2024

Apache-HBase commented May 30, 2024

jojochuang commented Jun 7, 2024

Apache9 commented Jun 8, 2024

2005hithlj commented Jun 12, 2024

HBASE-28625 ExportSnapshot should verify checksums for the source file and the target file #5950

HBASE-28625 ExportSnapshot should verify checksums for the source file and the target file #5950

Conversation

2005hithlj commented May 29, 2024

Apache-HBase commented May 29, 2024

Apache-HBase commented May 29, 2024

Apache-HBase commented May 29, 2024

Apache-HBase commented May 29, 2024

Apache-HBase commented May 30, 2024

Apache-HBase commented May 30, 2024

Apache-HBase commented May 30, 2024

Apache-HBase commented May 30, 2024

Apache9 May 30, 2024

Choose a reason for hiding this comment

2005hithlj May 30, 2024

Choose a reason for hiding this comment

Apache-HBase commented May 30, 2024

Apache-HBase commented May 30, 2024

Apache-HBase commented May 30, 2024

Apache-HBase commented May 30, 2024

jojochuang commented Jun 7, 2024

Apache9 commented Jun 8, 2024

2005hithlj commented Jun 12, 2024