HBASE-24114 [Flakey Tests] TestSnapshotScannerHDFSAclController #1433

huaxiangsun · 2020-04-04T01:16:34Z

There is a gap between snapshot finished and hdfs acl setup, scanSnapshot kicks in between and gets permission error.

huaxiangsun · 2020-04-04T01:20:46Z

Hi @mymeiyi, this is a race condition which causes the error as described in the jira. The case is like this:
snapshot -> markStatusComplete -> user does sanpshotScan -> coprocessor sets up HDFS acl.

I added the code to sleep for one second after snapshot, which is not ideal. Is there a way to check HDFS acl is setup after snapshot? That will be an ideal solution. Thanks.

Apache-HBase · 2020-04-04T02:02:09Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 35s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ master Compile Tests _
+1 💚	mvninstall	6m 11s	master passed
+1 💚	checkstyle	1m 18s	master passed
+1 💚	spotbugs	2m 22s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	5m 38s	the patch passed
+1 💚	checkstyle	1m 13s	the patch passed
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	hadoopcheck	12m 8s	Patch does not cause any errors with Hadoop 2.10.0 or 3.1.2.
+1 💚	spotbugs	2m 53s	the patch passed
		_ Other Tests _
+1 💚	asflicense	0m 15s	The patch does not generate ASF License warnings.
		41m 55s

Subsystem	Report/Notes
Docker	Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1433/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#1433
Optional Tests	dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle
uname	Linux f41c5deeac60 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `ea69b87`
Max. process+thread count	83 (vs. ulimit of 10000)
modules	C: hbase-server U: hbase-server
Console output	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1433/1/console
versions	git=2.17.1 maven=2018-06-17T18:33:14Z) spotbugs=3.1.12
Powered by	Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2020-04-04T04:34:05Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 35s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	6m 12s	master passed
+1 💚	compile	1m 5s	master passed
+1 💚	shadedjars	5m 22s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 38s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	5m 40s	the patch passed
+1 💚	compile	1m 1s	the patch passed
+1 💚	javac	1m 1s	the patch passed
+1 💚	shadedjars	5m 14s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 37s	the patch passed
		_ Other Tests _
+1 💚	unit	164m 36s	hbase-server in the patch passed.
		193m 50s

Subsystem	Report/Notes
Docker	Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1433/1/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile
GITHUB PR	#1433
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux fcb4c77074ad 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `ea69b87`
Default Java	1.8.0_232
Test Results	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1433/1/testReport/
Max. process+thread count	3426 (vs. ulimit of 10000)
modules	C: hbase-server U: hbase-server
Console output	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1433/1/console
versions	git=2.17.1 maven=2018-06-17T18:33:14Z)
Powered by	Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2020-04-04T04:42:51Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 13s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	7m 10s	master passed
+1 💚	compile	1m 12s	master passed
+1 💚	shadedjars	6m 13s	branch has no errors when building our shaded downstream artifacts.
-0 ⚠️	javadoc	0m 42s	hbase-server in master failed.
		_ Patch Compile Tests _
+1 💚	mvninstall	6m 41s	the patch passed
+1 💚	compile	1m 12s	the patch passed
+1 💚	javac	1m 12s	the patch passed
+1 💚	shadedjars	6m 13s	patch has no errors when building our shaded downstream artifacts.
-0 ⚠️	javadoc	0m 42s	hbase-server in the patch failed.
		_ Other Tests _
+1 💚	unit	169m 30s	hbase-server in the patch passed.
		202m 35s

Subsystem	Report/Notes
Docker	Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1433/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#1433
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 7092088a2d92 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `ea69b87`
Default Java	2020-01-14
javadoc	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1433/1/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt
javadoc	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1433/1/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt
Test Results	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1433/1/testReport/
Max. process+thread count	2757 (vs. ulimit of 10000)
modules	C: hbase-server U: hbase-server
Console output	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1433/1/console
versions	git=2.17.1 maven=2018-06-17T18:33:14Z)
Powered by	Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

saintstack

+1

Lets try it.

I actually just pushed this on branch-2 since I'm studying it... lets see if helps.

saintstack · 2020-04-04T20:15:57Z

.../test/java/org/apache/hadoop/hbase/security/access/TestSnapshotScannerHDFSAclController.java

+    throws Exception{
+    admin.snapshot(snapShotName, tableName);
+    LOG.info("Sleep for one second, waiting for HDFS Acl setup");
+    Threads.sleep(1000);


Up this to 3 seconds? Yeah, this ugly (smile).

I updated it to be 3 seconds, will wait for the test result and commit. The branch-2 results with patch looks good so far.

saintstack · 2020-04-04T20:16:46Z

Oh, I pushed w/ 3 second wait instead of 1 second.

huaxiangsun · 2020-04-05T05:32:04Z

Thanks @saintstack , 3 seconds is good. Will wait for @mymeiyi 's comments to see if there is a better way to avoid this sleep and then merge to the master.

Apache-HBase · 2020-04-07T19:38:42Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 21s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ master Compile Tests _
+1 💚	mvninstall	5m 59s	master passed
+1 💚	checkstyle	1m 15s	master passed
+1 💚	spotbugs	2m 8s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	5m 29s	the patch passed
+1 💚	checkstyle	1m 12s	the patch passed
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	hadoopcheck	11m 28s	Patch does not cause any errors with Hadoop 2.10.0 or 3.1.2.
+1 💚	spotbugs	2m 16s	the patch passed
		_ Other Tests _
+1 💚	asflicense	0m 13s	The patch does not generate ASF License warnings.
		38m 23s

Subsystem	Report/Notes
Docker	Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1433/2/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#1433
Optional Tests	dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle
uname	Linux c717d8100ef1 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `80d4a09`
Max. process+thread count	83 (vs. ulimit of 10000)
modules	C: hbase-server U: hbase-server
Console output	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1433/2/console
versions	git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12
Powered by	Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2020-04-07T21:23:39Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 29s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	6m 24s	master passed
+1 💚	compile	1m 2s	master passed
+1 💚	shadedjars	5m 12s	branch has no errors when building our shaded downstream artifacts.
-0 ⚠️	javadoc	0m 41s	hbase-server in master failed.
		_ Patch Compile Tests _
+1 💚	mvninstall	5m 54s	the patch passed
+1 💚	compile	1m 2s	the patch passed
+1 💚	javac	1m 2s	the patch passed
+1 💚	shadedjars	5m 11s	patch has no errors when building our shaded downstream artifacts.
-0 ⚠️	javadoc	0m 39s	hbase-server in the patch failed.
		_ Other Tests _
+1 💚	unit	114m 36s	hbase-server in the patch passed.
		143m 22s

Subsystem	Report/Notes
Docker	Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1433/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#1433
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 4bd4168f048a 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `80d4a09`
Default Java	2020-01-14
javadoc	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1433/2/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt
javadoc	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1433/2/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt
Test Results	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1433/2/testReport/
Max. process+thread count	4151 (vs. ulimit of 10000)
modules	C: hbase-server U: hbase-server
Console output	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1433/2/console
versions	git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f)
Powered by	Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2020-04-07T22:25:24Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 10s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	6m 0s	master passed
+1 💚	compile	1m 4s	master passed
+1 💚	shadedjars	5m 24s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 40s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	5m 51s	the patch passed
+1 💚	compile	0m 57s	the patch passed
+1 💚	javac	0m 57s	the patch passed
+1 💚	shadedjars	4m 53s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 38s	the patch passed
		_ Other Tests _
+1 💚	unit	176m 42s	hbase-server in the patch passed.
		205m 6s

Subsystem	Report/Notes
Docker	Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1433/2/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile
GITHUB PR	#1433
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 93fcfc41cd92 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `80d4a09`
Default Java	1.8.0_232
Test Results	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1433/2/testReport/
Max. process+thread count	3216 (vs. ulimit of 10000)
modules	C: hbase-server U: hbase-server
Console output	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1433/2/console
versions	git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f)
Powered by	Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

…he#1433) Signed-off-by: stack <stack@apache.org>

… (#1459) Signed-off-by: stack <stack@apache.org>

…he#1433) Signed-off-by: stack <stack@apache.org>

huaxiangsun requested a review from mymeiyi April 4, 2020 01:17

saintstack approved these changes Apr 4, 2020

View reviewed changes

HBASE-24114 [Flakey Tests] TestSnapshotScannerHDFSAclController

92e8cb4

huaxiangsun force-pushed the master.HBASE-24114 branch from 5bdf3ca to 92e8cb4 Compare April 7, 2020 18:58

huaxiangsun merged commit d6451a8 into apache:master Apr 8, 2020

huaxiangsun added a commit to huaxiangsun/hbase that referenced this pull request Apr 8, 2020

HBASE-24114 [Flakey Tests] TestSnapshotScannerHDFSAclController (apac…

bb8d9df

…he#1433) Signed-off-by: stack <stack@apache.org>

huaxiangsun added a commit that referenced this pull request Apr 8, 2020

HBASE-24114 [Flakey Tests] TestSnapshotScannerHDFSAclController (#1433)…

5b1f908

… (#1459) Signed-off-by: stack <stack@apache.org>

thangTang pushed a commit to thangTang/hbase that referenced this pull request Apr 16, 2020

HBASE-24114 [Flakey Tests] TestSnapshotScannerHDFSAclController (apac…

c8f978a

…he#1433) Signed-off-by: stack <stack@apache.org>

thangTang pushed a commit to thangTang/hbase that referenced this pull request Apr 16, 2020

HBASE-24114 [Flakey Tests] TestSnapshotScannerHDFSAclController (apac…

7264892

…he#1433) Signed-off-by: stack <stack@apache.org>

thangTang pushed a commit to thangTang/hbase that referenced this pull request Apr 16, 2020

HBASE-24114 [Flakey Tests] TestSnapshotScannerHDFSAclController (apac…

3a88bd4

…he#1433) Signed-off-by: stack <stack@apache.org>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

HBASE-24114 [Flakey Tests] TestSnapshotScannerHDFSAclController #1433

HBASE-24114 [Flakey Tests] TestSnapshotScannerHDFSAclController #1433

huaxiangsun commented Apr 4, 2020

huaxiangsun commented Apr 4, 2020

Apache-HBase commented Apr 4, 2020

Apache-HBase commented Apr 4, 2020

Apache-HBase commented Apr 4, 2020

saintstack left a comment

saintstack Apr 4, 2020

huaxiangsun Apr 7, 2020

saintstack commented Apr 4, 2020

huaxiangsun commented Apr 5, 2020

Apache-HBase commented Apr 7, 2020

Apache-HBase commented Apr 7, 2020

Apache-HBase commented Apr 7, 2020

HBASE-24114 [Flakey Tests] TestSnapshotScannerHDFSAclController #1433

HBASE-24114 [Flakey Tests] TestSnapshotScannerHDFSAclController #1433

Conversation

huaxiangsun commented Apr 4, 2020

huaxiangsun commented Apr 4, 2020

Apache-HBase commented Apr 4, 2020

Apache-HBase commented Apr 4, 2020

Apache-HBase commented Apr 4, 2020

saintstack left a comment

Choose a reason for hiding this comment

saintstack Apr 4, 2020

Choose a reason for hiding this comment

huaxiangsun Apr 7, 2020

Choose a reason for hiding this comment

saintstack commented Apr 4, 2020

huaxiangsun commented Apr 5, 2020

Apache-HBase commented Apr 7, 2020

Apache-HBase commented Apr 7, 2020

Apache-HBase commented Apr 7, 2020