HBASE-23202 ExportSnapshot (import) will fail if copying files to roo… #1791

huaxiangsun · 2020-05-27T17:35:02Z

…t directory takes longer than cleaner TTL

huaxiangsun · 2020-05-27T17:41:56Z

This is a rebase of old PR, minor changes, no code logic change.

@busbey @z-york Please take a look, thanks.

Apache-HBase · 2020-05-27T18:18:39Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 23s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ master Compile Tests _
+1 💚	mvninstall	4m 38s	master passed
+1 💚	checkstyle	1m 32s	master passed
+1 💚	spotbugs	2m 30s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	3m 56s	the patch passed
+1 💚	checkstyle	1m 9s	the patch passed
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	hadoopcheck	12m 23s	Patch does not cause any errors with Hadoop 3.1.2 3.2.1.
+1 💚	spotbugs	2m 18s	the patch passed
		_ Other Tests _
+1 💚	asflicense	0m 13s	The patch does not generate ASF License warnings.
		38m 11s

Subsystem	Report/Notes
Docker	Client=19.03.9 Server=19.03.9 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1791/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#1791
Optional Tests	dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle
uname	Linux ec69bbecbe95 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `48e9835`
Max. process+thread count	84 (vs. ulimit of 12500)
modules	C: hbase-server U: hbase-server
Console output	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1791/1/console
versions	git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12
Powered by	Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/SnapshotFileCache.java

Apache-HBase · 2020-05-27T21:23:39Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 4s	Docker mode activated.
-0 ⚠️	yetus	0m 2s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	4m 3s	master passed
+1 💚	compile	0m 58s	master passed
+1 💚	shadedjars	6m 6s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 37s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	3m 45s	the patch passed
+1 💚	compile	0m 55s	the patch passed
+1 💚	javac	0m 55s	the patch passed
+1 💚	shadedjars	6m 2s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 36s	the patch passed
		_ Other Tests _
+1 💚	unit	197m 11s	hbase-server in the patch passed.
		223m 10s

Subsystem	Report/Notes
Docker	Client=19.03.9 Server=19.03.9 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1791/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR	#1791
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 1ab2864134df 4.15.0-101-generic #102-Ubuntu SMP Mon May 11 10:07:26 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `48e9835`
Default Java	1.8.0_232
Test Results	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1791/1/testReport/
Max. process+thread count	3593 (vs. ulimit of 12500)
modules	C: hbase-server U: hbase-server
Console output	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1791/1/console
versions	git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f)
Powered by	Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2020-05-27T22:07:46Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 6s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	4m 47s	master passed
+1 💚	compile	1m 9s	master passed
+1 💚	shadedjars	6m 25s	branch has no errors when building our shaded downstream artifacts.
-0 ⚠️	javadoc	0m 48s	hbase-server in master failed.
		_ Patch Compile Tests _
+1 💚	mvninstall	4m 52s	the patch passed
+1 💚	compile	1m 14s	the patch passed
+1 💚	javac	1m 14s	the patch passed
+1 💚	shadedjars	6m 26s	patch has no errors when building our shaded downstream artifacts.
-0 ⚠️	javadoc	0m 40s	hbase-server in the patch failed.
		_ Other Tests _
-1 ❌	unit	236m 34s	hbase-server in the patch failed.
		267m 11s

Subsystem	Report/Notes
Docker	Client=19.03.9 Server=19.03.9 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1791/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#1791
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 6f84e372f262 4.15.0-101-generic #102-Ubuntu SMP Mon May 11 10:07:26 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `48e9835`
Default Java	2020-01-14
javadoc	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1791/1/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt
javadoc	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1791/1/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt
unit	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1791/1/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt
Test Results	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1791/1/testReport/
Max. process+thread count	3084 (vs. ulimit of 12500)
modules	C: hbase-server U: hbase-server
Console output	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1791/1/console
versions	git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f)
Powered by	Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

huaxiangsun · 2020-05-28T04:40:17Z

Test failure is a flaky, HBASE-24429

hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/SnapshotFileCache.java

...e-server/src/test/java/org/apache/hadoop/hbase/master/snapshot/TestSnapshotHFileCleaner.java

hbase-server/src/test/java/org/apache/hadoop/hbase/master/snapshot/TestSnapshotFileCache.java

huaxiangsun · 2020-06-02T23:51:13Z

@z-york @ndimiduk I updated the patch based on your comments. Here is a summary of the changes:

The major changes are addressing SNAPSHOT_WORKING_DIR config, added a new Test class to test against SNAPSHOT_WORKING_DIR to be a different filesystem from the rootdir's.
Other changes are for fixing testing issues.

Can you review the changes? Thanks.

p.s., The current state is that hfile cleaner and snapshot wont step on each other. The only missing part is ExportSnapshot. However, there is an issue in ExportSnapshot which does not honor the case that working dir/root dir are from different filesystem.

Apache-HBase · 2020-06-03T00:09:35Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 34s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ master Compile Tests _
+1 💚	mvninstall	3m 47s	master passed
+1 💚	checkstyle	1m 8s	master passed
+1 💚	spotbugs	2m 3s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	3m 23s	the patch passed
-0 ⚠️	checkstyle	1m 4s	hbase-server: The patch generated 5 new + 38 unchanged - 4 fixed = 43 total (was 42)
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	hadoopcheck	11m 38s	Patch does not cause any errors with Hadoop 3.1.2 3.2.1.
+1 💚	spotbugs	2m 14s	the patch passed
		_ Other Tests _
+1 💚	asflicense	0m 14s	The patch does not generate ASF License warnings.
		33m 59s

Subsystem	Report/Notes
Docker	Client=19.03.11 Server=19.03.11 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1791/2/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#1791
Optional Tests	dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle
uname	Linux ab2f654c03ff 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `cb5c0d2`
checkstyle	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1791/2/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt
Max. process+thread count	94 (vs. ulimit of 12500)
modules	C: hbase-server U: hbase-server
Console output	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1791/2/console
versions	git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12
Powered by	Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

z-york

Thanks for making those changes, just a few lingering things.

hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/SnapshotHFileCleaner.java

hbase-server/src/main/java/org/apache/hadoop/hbase/snapshot/SnapshotDescriptionUtils.java

...e-server/src/test/java/org/apache/hadoop/hbase/master/snapshot/TestSnapshotHFileCleaner.java

Apache-HBase · 2020-06-03T02:14:43Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 24s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	3m 44s	master passed
+1 💚	compile	0m 53s	master passed
+1 💚	shadedjars	5m 36s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 38s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	3m 25s	the patch passed
+1 💚	compile	0m 56s	the patch passed
+1 💚	javac	0m 56s	the patch passed
+1 💚	shadedjars	5m 31s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 35s	the patch passed
		_ Other Tests _
+1 💚	unit	134m 14s	hbase-server in the patch passed.
		159m 7s

Subsystem	Report/Notes
Docker	Client=19.03.11 Server=19.03.11 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1791/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR	#1791
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux d85d69081ba7 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `cb5c0d2`
Default Java	1.8.0_232
Test Results	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1791/2/testReport/
Max. process+thread count	4376 (vs. ulimit of 12500)
modules	C: hbase-server U: hbase-server
Console output	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1791/2/console
versions	git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f)
Powered by	Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2020-06-03T03:34:40Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	3m 32s	Docker mode activated.
-0 ⚠️	yetus	0m 4s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	5m 15s	master passed
+1 💚	compile	1m 24s	master passed
+1 💚	shadedjars	7m 6s	branch has no errors when building our shaded downstream artifacts.
-0 ⚠️	javadoc	0m 49s	hbase-server in master failed.
		_ Patch Compile Tests _
+1 💚	mvninstall	5m 23s	the patch passed
+1 💚	compile	1m 32s	the patch passed
+1 💚	javac	1m 32s	the patch passed
+1 💚	shadedjars	6m 54s	patch has no errors when building our shaded downstream artifacts.
-0 ⚠️	javadoc	0m 47s	hbase-server in the patch failed.
		_ Other Tests _
-1 ❌	unit	204m 24s	hbase-server in the patch failed.
		239m 3s

Subsystem	Report/Notes
Docker	Client=19.03.11 Server=19.03.11 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1791/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#1791
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 20b741f17d0e 4.15.0-101-generic #102-Ubuntu SMP Mon May 11 10:07:26 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `cb5c0d2`
Default Java	2020-01-14
javadoc	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1791/2/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt
javadoc	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1791/2/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt
unit	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1791/2/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt
Test Results	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1791/2/testReport/
Max. process+thread count	2938 (vs. ulimit of 12500)
modules	C: hbase-server U: hbase-server
Console output	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1791/2/console
versions	git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f)
Powered by	Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2020-06-03T04:11:55Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 34s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ master Compile Tests _
+1 💚	mvninstall	3m 48s	master passed
+1 💚	checkstyle	1m 4s	master passed
+1 💚	spotbugs	2m 4s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	3m 21s	the patch passed
-0 ⚠️	checkstyle	1m 5s	hbase-server: The patch generated 5 new + 38 unchanged - 4 fixed = 43 total (was 42)
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	hadoopcheck	11m 4s	Patch does not cause any errors with Hadoop 3.1.2 3.2.1.
+1 💚	spotbugs	3m 0s	the patch passed
		_ Other Tests _
+1 💚	asflicense	0m 17s	The patch does not generate ASF License warnings.
		35m 3s

Subsystem	Report/Notes
Docker	Client=19.03.11 Server=19.03.11 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1791/3/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#1791
Optional Tests	dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle
uname	Linux bb13a89cc7bc 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `f71f1cd`
checkstyle	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1791/3/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt
Max. process+thread count	94 (vs. ulimit of 12500)
modules	C: hbase-server U: hbase-server
Console output	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1791/3/console
versions	git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12
Powered by	Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2020-06-03T06:09:31Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 33s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	4m 27s	master passed
+1 💚	compile	1m 6s	master passed
+1 💚	shadedjars	5m 54s	branch has no errors when building our shaded downstream artifacts.
-0 ⚠️	javadoc	0m 45s	hbase-server in master failed.
		_ Patch Compile Tests _
+1 💚	mvninstall	4m 6s	the patch passed
+1 💚	compile	1m 4s	the patch passed
+1 💚	javac	1m 4s	the patch passed
+1 💚	shadedjars	5m 53s	patch has no errors when building our shaded downstream artifacts.
-0 ⚠️	javadoc	0m 39s	hbase-server in the patch failed.
		_ Other Tests _
+1 💚	unit	126m 1s	hbase-server in the patch passed.
		152m 37s

Subsystem	Report/Notes
Docker	Client=19.03.11 Server=19.03.11 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1791/3/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#1791
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 999780d57651 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `f71f1cd`
Default Java	2020-01-14
javadoc	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1791/3/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt
javadoc	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1791/3/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt
Test Results	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1791/3/testReport/
Max. process+thread count	4426 (vs. ulimit of 12500)
modules	C: hbase-server U: hbase-server
Console output	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1791/3/console
versions	git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f)
Powered by	Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2020-06-03T06:16:12Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 31s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	3m 41s	master passed
+1 💚	compile	0m 56s	master passed
+1 💚	shadedjars	5m 35s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 39s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	3m 23s	the patch passed
+1 💚	compile	0m 53s	the patch passed
+1 💚	javac	0m 53s	the patch passed
+1 💚	shadedjars	5m 29s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 34s	the patch passed
		_ Other Tests _
-1 ❌	unit	135m 28s	hbase-server in the patch failed.
		159m 22s

Subsystem	Report/Notes
Docker	Client=19.03.11 Server=19.03.11 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1791/3/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR	#1791
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 2f03ebdea3bc 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `f71f1cd`
Default Java	1.8.0_232
unit	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1791/3/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt
Test Results	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1791/3/testReport/
Max. process+thread count	4669 (vs. ulimit of 12500)
modules	C: hbase-server U: hbase-server
Console output	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1791/3/console
versions	git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f)
Powered by	Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

huaxiangsun · 2020-06-03T19:14:26Z

@z-york I uploaded a new patch which addresses some of your comments, put up explain for rest of your comments. Please let me know if there is anything missing, thanks.

Apache-HBase · 2020-06-03T19:26:08Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 36s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ master Compile Tests _
+1 💚	mvninstall	3m 47s	master passed
+1 💚	checkstyle	1m 9s	master passed
+1 💚	spotbugs	2m 1s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	3m 25s	the patch passed
-0 ⚠️	checkstyle	1m 6s	hbase-server: The patch generated 5 new + 38 unchanged - 4 fixed = 43 total (was 42)
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	hadoopcheck	11m 13s	Patch does not cause any errors with Hadoop 3.1.2 3.2.1.
+1 💚	spotbugs	2m 37s	the patch passed
		_ Other Tests _
+1 💚	asflicense	0m 15s	The patch does not generate ASF License warnings.
		35m 27s

Subsystem	Report/Notes
Docker	Client=19.03.11 Server=19.03.11 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1791/4/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#1791
Optional Tests	dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle
uname	Linux ad3ca0482ebd 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `0078268`
checkstyle	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1791/4/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt
Max. process+thread count	94 (vs. ulimit of 12500)
modules	C: hbase-server U: hbase-server
Console output	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1791/4/console
versions	git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12
Powered by	Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2020-06-03T21:34:19Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 36s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	3m 48s	master passed
+1 💚	compile	0m 57s	master passed
+1 💚	shadedjars	5m 36s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 40s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	3m 29s	the patch passed
+1 💚	compile	0m 54s	the patch passed
+1 💚	javac	0m 54s	the patch passed
+1 💚	shadedjars	5m 31s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 37s	the patch passed
		_ Other Tests _
+1 💚	unit	139m 28s	hbase-server in the patch passed.
		163m 36s

Subsystem	Report/Notes
Docker	Client=19.03.11 Server=19.03.11 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1791/4/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR	#1791
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 5120ea93e01a 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `0078268`
Default Java	1.8.0_232
Test Results	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1791/4/testReport/
Max. process+thread count	3771 (vs. ulimit of 12500)
modules	C: hbase-server U: hbase-server
Console output	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1791/4/console
versions	git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f)
Powered by	Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2020-06-03T22:50:05Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 31s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	5m 18s	master passed
+1 💚	compile	1m 19s	master passed
+1 💚	shadedjars	6m 56s	branch has no errors when building our shaded downstream artifacts.
-0 ⚠️	javadoc	0m 47s	hbase-server in master failed.
		_ Patch Compile Tests _
+1 💚	mvninstall	5m 11s	the patch passed
+1 💚	compile	1m 19s	the patch passed
+1 💚	javac	1m 19s	the patch passed
+1 💚	shadedjars	7m 0s	patch has no errors when building our shaded downstream artifacts.
-0 ⚠️	javadoc	0m 47s	hbase-server in the patch failed.
		_ Other Tests _
+1 💚	unit	207m 13s	hbase-server in the patch passed.
		239m 24s

Subsystem	Report/Notes
Docker	Client=19.03.11 Server=19.03.11 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1791/4/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#1791
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux afee4a7af76b 4.15.0-101-generic #102-Ubuntu SMP Mon May 11 10:07:26 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `0078268`
Default Java	2020-01-14
javadoc	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1791/4/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt
javadoc	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1791/4/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt
Test Results	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1791/4/testReport/
Max. process+thread count	3037 (vs. ulimit of 12500)
modules	C: hbase-server U: hbase-server
Console output	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1791/4/console
versions	git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f)
Powered by	Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

z-york · 2020-06-03T23:39:42Z

Looks good to me. Thanks for the changes. Let's wait to see if @ndimiduk has any further comments.

huaxiangsun · 2020-06-04T16:16:03Z

Hi @ndimiduk, can you review the latest patch to see if there is anything missing? Thanks!

ndimiduk

Looks good. A couple comments about tests.

hbase-server/src/test/java/org/apache/hadoop/hbase/master/snapshot/TestSnapshotFileCache.java

...va/org/apache/hadoop/hbase/master/snapshot/TestSnapshotFileCacheWithDifferentWorkingDir.java

…t directory takes longer than cleaner TTL

Apache-HBase · 2020-06-08T18:36:04Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 34s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ master Compile Tests _
+1 💚	mvninstall	3m 51s	master passed
+1 💚	checkstyle	1m 9s	master passed
+1 💚	spotbugs	2m 7s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	3m 20s	the patch passed
-0 ⚠️	checkstyle	1m 5s	hbase-server: The patch generated 6 new + 38 unchanged - 4 fixed = 44 total (was 42)
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	hadoopcheck	11m 14s	Patch does not cause any errors with Hadoop 3.1.2 3.2.1.
+1 💚	spotbugs	3m 1s	the patch passed
		_ Other Tests _
+1 💚	asflicense	0m 17s	The patch does not generate ASF License warnings.
		35m 32s

Subsystem	Report/Notes
Docker	Client=19.03.11 Server=19.03.11 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1791/5/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#1791
Optional Tests	dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle
uname	Linux aca7d6bbd803 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `920d60f`
checkstyle	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1791/5/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt
Max. process+thread count	94 (vs. ulimit of 12500)
modules	C: hbase-server U: hbase-server
Console output	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1791/5/console
versions	git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12
Powered by	Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2020-06-08T20:33:56Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 33s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	4m 31s	master passed
+1 💚	compile	1m 5s	master passed
+1 💚	shadedjars	5m 50s	branch has no errors when building our shaded downstream artifacts.
-0 ⚠️	javadoc	0m 41s	hbase-server in master failed.
		_ Patch Compile Tests _
+1 💚	mvninstall	4m 7s	the patch passed
+1 💚	compile	1m 4s	the patch passed
+1 💚	javac	1m 4s	the patch passed
+1 💚	shadedjars	5m 47s	patch has no errors when building our shaded downstream artifacts.
-0 ⚠️	javadoc	0m 41s	hbase-server in the patch failed.
		_ Other Tests _
+1 💚	unit	127m 7s	hbase-server in the patch passed.
		153m 23s

Subsystem	Report/Notes
Docker	Client=19.03.11 Server=19.03.11 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1791/5/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#1791
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 036b9fe03771 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `920d60f`
Default Java	2020-01-14
javadoc	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1791/5/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt
javadoc	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1791/5/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt
Test Results	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1791/5/testReport/
Max. process+thread count	4171 (vs. ulimit of 12500)
modules	C: hbase-server U: hbase-server
Console output	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1791/5/console
versions	git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f)
Powered by	Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2020-06-08T20:47:12Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 31s	Docker mode activated.
-0 ⚠️	yetus	0m 2s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	3m 29s	master passed
+1 💚	compile	0m 55s	master passed
+1 💚	shadedjars	5m 42s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 39s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	3m 36s	the patch passed
+1 💚	compile	0m 54s	the patch passed
+1 💚	javac	0m 54s	the patch passed
+1 💚	shadedjars	5m 36s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 36s	the patch passed
		_ Other Tests _
+1 💚	unit	142m 52s	hbase-server in the patch passed.
		166m 42s

Subsystem	Report/Notes
Docker	Client=19.03.11 Server=19.03.11 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1791/5/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR	#1791
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux e0ef731b3868 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `920d60f`
Default Java	1.8.0_232
Test Results	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1791/5/testReport/
Max. process+thread count	4290 (vs. ulimit of 12500)
modules	C: hbase-server U: hbase-server
Console output	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1791/5/console
versions	git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f)
Powered by	Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

…s to root directory takes longer than cleaner TTL (#1791)" This reverts commit f862f3d.

…s to root directory takes longer than cleaner TTL (#1791)" (#1873) This reverts commit f862f3d.

huaxiangsun mentioned this pull request May 27, 2020

HBASE-23202 ExportSnapshot (import) will fail if copying files to root directory takes longer than cleaner TTL #769

Closed

ndimiduk reviewed May 27, 2020

View reviewed changes

hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/SnapshotFileCache.java Outdated Show resolved Hide resolved

z-york requested changes May 28, 2020

View reviewed changes

huaxiangsun force-pushed the master.HBASE-23202 branch 2 times, most recently from 6f1a191 to d42f462 Compare June 2, 2020 23:38

z-york requested changes Jun 3, 2020

View reviewed changes

huaxiangsun force-pushed the master.HBASE-23202 branch from d42f462 to 4375fec Compare June 3, 2020 18:40

z-york approved these changes Jun 3, 2020

View reviewed changes

ndimiduk approved these changes Jun 4, 2020

View reviewed changes

HBASE-23202 ExportSnapshot (import) will fail if copying files to roo…

ede033b

…t directory takes longer than cleaner TTL

huaxiangsun force-pushed the master.HBASE-23202 branch from 4375fec to ede033b Compare June 8, 2020 17:50

huaxiangsun merged commit f862f3d into apache:master Jun 8, 2020

huaxiangsun added a commit that referenced this pull request Jun 8, 2020

Revert "HBASE-23202 ExportSnapshot (import) will fail if copying file…

3df034a

…s to root directory takes longer than cleaner TTL (#1791)" This reverts commit f862f3d.

huaxiangsun mentioned this pull request Jun 8, 2020

Revert "HBASE-23202 ExportSnapshot (import) will fail if copying files to roo…" #1873

Merged

huaxiangsun added a commit that referenced this pull request Jun 8, 2020

Revert "HBASE-23202 ExportSnapshot (import) will fail if copying file…

5042b59

…s to root directory takes longer than cleaner TTL (#1791)" (#1873) This reverts commit f862f3d.

huaxiangsun mentioned this pull request Jun 8, 2020

HBASE-23202 ExportSnapshot (import) will fail if copying files to roo… #1874

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

HBASE-23202 ExportSnapshot (import) will fail if copying files to roo… #1791

HBASE-23202 ExportSnapshot (import) will fail if copying files to roo… #1791

huaxiangsun commented May 27, 2020

huaxiangsun commented May 27, 2020

Apache-HBase commented May 27, 2020

Apache-HBase commented May 27, 2020

Apache-HBase commented May 27, 2020

huaxiangsun commented May 28, 2020

huaxiangsun commented Jun 2, 2020

Apache-HBase commented Jun 3, 2020

z-york left a comment

Apache-HBase commented Jun 3, 2020

Apache-HBase commented Jun 3, 2020

Apache-HBase commented Jun 3, 2020

Apache-HBase commented Jun 3, 2020

Apache-HBase commented Jun 3, 2020

huaxiangsun commented Jun 3, 2020

Apache-HBase commented Jun 3, 2020

Apache-HBase commented Jun 3, 2020

Apache-HBase commented Jun 3, 2020

z-york commented Jun 3, 2020

huaxiangsun commented Jun 4, 2020

ndimiduk left a comment

Apache-HBase commented Jun 8, 2020

Apache-HBase commented Jun 8, 2020

Apache-HBase commented Jun 8, 2020

HBASE-23202 ExportSnapshot (import) will fail if copying files to roo… #1791

HBASE-23202 ExportSnapshot (import) will fail if copying files to roo… #1791

Conversation

huaxiangsun commented May 27, 2020

huaxiangsun commented May 27, 2020

Apache-HBase commented May 27, 2020

Apache-HBase commented May 27, 2020

Apache-HBase commented May 27, 2020

huaxiangsun commented May 28, 2020

huaxiangsun commented Jun 2, 2020

Apache-HBase commented Jun 3, 2020

z-york left a comment

Choose a reason for hiding this comment

Apache-HBase commented Jun 3, 2020

Apache-HBase commented Jun 3, 2020

Apache-HBase commented Jun 3, 2020

Apache-HBase commented Jun 3, 2020

Apache-HBase commented Jun 3, 2020

huaxiangsun commented Jun 3, 2020

Apache-HBase commented Jun 3, 2020

Apache-HBase commented Jun 3, 2020

Apache-HBase commented Jun 3, 2020

z-york commented Jun 3, 2020

huaxiangsun commented Jun 4, 2020

ndimiduk left a comment

Choose a reason for hiding this comment

Apache-HBase commented Jun 8, 2020

Apache-HBase commented Jun 8, 2020

Apache-HBase commented Jun 8, 2020