HADOOP-18388. Allow dynamic groupSearchFilter in LdapGroupsMapping. #4798

ayushtkn · 2022-08-24T12:07:24Z

Description of PR

Allow Dynamic Group Search Filter in LdapGroupsMapping

How was this patch tested?

Unit Test. (WIP: In actual env)

For code changes:

Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files?

hadoop-yetus · 2022-08-26T04:18:59Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 35s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 1s		codespell was not available.
+0 🆗	detsecrets	0m 1s		detect-secrets was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 1 new or modified test files.
			_ trunk Compile Tests _
+1 💚	mvninstall	38m 21s		trunk passed
+1 💚	compile	22m 58s		trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 💚	compile	20m 22s		trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚	checkstyle	1m 33s		trunk passed
+1 💚	mvnsite	1m 54s		trunk passed
+1 💚	javadoc	1m 44s		trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 💚	javadoc	1m 16s		trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚	spotbugs	2m 56s		trunk passed
+1 💚	shadedclient	23m 0s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	1m 2s		the patch passed
+1 💚	compile	22m 20s		the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 💚	javac	22m 20s		the patch passed
+1 💚	compile	20m 20s		the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚	javac	20m 20s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	1m 24s	/results-checkstyle-hadoop-common-project_hadoop-common.txt	hadoop-common-project/hadoop-common: The patch generated 1 new + 11 unchanged - 0 fixed = 12 total (was 11)
+1 💚	mvnsite	1m 49s		the patch passed
+1 💚	javadoc	1m 18s		the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 💚	javadoc	1m 3s		the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚	spotbugs	2m 52s		the patch passed
+1 💚	shadedclient	22m 41s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	18m 22s		hadoop-common in the patch passed.
+1 💚	asflicense	1m 13s		The patch does not generate ASF License warnings.
		210m 13s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4798/3/artifact/out/Dockerfile
GITHUB PR	#4798
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname	Linux 781e6ada6045 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4 01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `4e32347`
Default Java	Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4798/3/testReport/
Max. process+thread count	1686 (vs. ulimit of 5500)
modules	C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4798/3/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2022-08-31T18:04:11Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 53s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 1 new or modified test files.
			_ trunk Compile Tests _
+1 💚	mvninstall	38m 40s		trunk passed
+1 💚	compile	23m 12s		trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 💚	compile	20m 38s		trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚	checkstyle	1m 26s		trunk passed
+1 💚	mvnsite	2m 8s		trunk passed
+1 💚	javadoc	1m 29s		trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 💚	javadoc	1m 16s		trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚	spotbugs	3m 13s		trunk passed
+1 💚	shadedclient	23m 14s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	1m 1s		the patch passed
+1 💚	compile	22m 27s		the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 💚	javac	22m 27s		the patch passed
+1 💚	compile	20m 29s		the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚	javac	20m 29s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	1m 22s	/results-checkstyle-hadoop-common-project_hadoop-common.txt	hadoop-common-project/hadoop-common: The patch generated 1 new + 11 unchanged - 0 fixed = 12 total (was 11)
+1 💚	mvnsite	1m 57s		the patch passed
+1 💚	javadoc	1m 19s		the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 💚	javadoc	1m 4s		the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚	spotbugs	2m 53s		the patch passed
+1 💚	shadedclient	22m 47s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	18m 45s		hadoop-common in the patch passed.
+1 💚	asflicense	1m 20s		The patch does not generate ASF License warnings.
		212m 35s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4798/4/artifact/out/Dockerfile
GITHUB PR	#4798
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname	Linux 986a2500b6c4 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4 01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `22f33c3`
Default Java	Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4798/4/testReport/
Max. process+thread count	3159 (vs. ulimit of 5500)
modules	C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4798/4/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2022-08-31T18:24:07Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 58s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 1s		codespell was not available.
+0 🆗	detsecrets	0m 1s		detect-secrets was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 1 new or modified test files.
			_ trunk Compile Tests _
+1 💚	mvninstall	41m 3s		trunk passed
+1 💚	compile	25m 33s		trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 💚	compile	21m 56s		trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚	checkstyle	1m 24s		trunk passed
+1 💚	mvnsite	1m 53s		trunk passed
+1 💚	javadoc	1m 27s		trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 💚	javadoc	0m 57s		trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚	spotbugs	2m 57s		trunk passed
+1 💚	shadedclient	25m 48s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	1m 4s		the patch passed
+1 💚	compile	24m 22s		the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 💚	javac	24m 22s		the patch passed
+1 💚	compile	21m 49s		the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚	javac	21m 49s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	1m 17s	/results-checkstyle-hadoop-common-project_hadoop-common.txt	hadoop-common-project/hadoop-common: The patch generated 1 new + 11 unchanged - 0 fixed = 12 total (was 11)
+1 💚	mvnsite	1m 50s		the patch passed
+1 💚	javadoc	1m 16s		the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 💚	javadoc	0m 57s		the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚	spotbugs	2m 56s		the patch passed
+1 💚	shadedclient	26m 1s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	18m 25s		hadoop-common in the patch passed.
+1 💚	asflicense	1m 11s		The patch does not generate ASF License warnings.
		225m 35s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4798/5/artifact/out/Dockerfile
GITHUB PR	#4798
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname	Linux 0d2c0b1f6ecb 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4 01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `5547b80`
Default Java	Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4798/5/testReport/
Max. process+thread count	1253 (vs. ulimit of 5500)
modules	C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4798/5/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

lmccay

One question that needs to be resolved as answered or changed and one nit. :)

...on-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestLdapGroupsMapping.java

...common-project/hadoop-common/src/main/java/org/apache/hadoop/security/LdapGroupsMapping.java

hadoop-yetus · 2022-09-06T17:10:21Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 46s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+0 🆗	xmllint	0m 0s		xmllint was not available.
+0 🆗	markdownlint	0m 0s		markdownlint was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 1 new or modified test files.
			_ trunk Compile Tests _
+1 💚	mvninstall	40m 49s		trunk passed
+1 💚	compile	25m 31s		trunk passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04
+1 💚	compile	21m 24s		trunk passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07
+1 💚	checkstyle	1m 27s		trunk passed
+1 💚	mvnsite	2m 0s		trunk passed
+1 💚	javadoc	1m 24s		trunk passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04
+1 💚	javadoc	1m 1s		trunk passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07
+1 💚	spotbugs	3m 7s		trunk passed
+1 💚	shadedclient	24m 28s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	1m 5s		the patch passed
+1 💚	compile	26m 10s		the patch passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04
+1 💚	javac	26m 10s		the patch passed
+1 💚	compile	23m 36s		the patch passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07
+1 💚	javac	23m 36s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	1m 35s		the patch passed
+1 💚	mvnsite	2m 0s		the patch passed
+1 💚	javadoc	1m 30s		the patch passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04
+1 💚	javadoc	1m 7s		the patch passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07
+1 💚	spotbugs	3m 10s		the patch passed
+1 💚	shadedclient	26m 55s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	19m 15s		hadoop-common in the patch passed.
+1 💚	asflicense	1m 23s		The patch does not generate ASF License warnings.
		230m 13s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4798/8/artifact/out/Dockerfile
GITHUB PR	#4798
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint markdownlint
uname	Linux f624272391e1 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4 01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `5e56532`
Default Java	Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4798/8/testReport/
Max. process+thread count	1302 (vs. ulimit of 5500)
modules	C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4798/8/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

lmccay · 2022-09-06T22:39:19Z

I've merged this to trunk, @ayushtkn - thank you for your contribution!

…pache#4798) * HADOOP-18388. Allow dynamic groupSearchFilter in LdapGroupsMapping.

ayushtkn added 2 commits August 25, 2022 15:34

HADOOP-18388. Allow dynamic groupSearchFilter in LdapGroupsMapping.

5f40843

Changed.

4e32347

ayushtkn force-pushed the HADOOP-18388 branch from 8ae660d to 4e32347 Compare August 26, 2022 00:47

apache deleted a comment from hadoop-yetus Aug 26, 2022

ayushtkn added 2 commits August 31, 2022 19:59

Use configs rather than attribute for group filter params.

22f33c3

Fix log.

5547b80

lmccay requested changes Sep 1, 2022

View reviewed changes

...on-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestLdapGroupsMapping.java Outdated Show resolved Hide resolved

...common-project/hadoop-common/src/main/java/org/apache/hadoop/security/LdapGroupsMapping.java Show resolved Hide resolved

Add documentation & handle review comments.

0a11ee5

This comment was marked as outdated.

Sign in to view

Improve Doc & Fix Checkstyle.

5e56532

This comment was marked as outdated.

Sign in to view

lmccay approved these changes Sep 6, 2022

View reviewed changes

lmccay merged commit cc41ad6 into apache:trunk Sep 6, 2022

HarshitGupta11 pushed a commit to HarshitGupta11/hadoop that referenced this pull request Nov 28, 2022

HADOOP-18388. Allow dynamic groupSearchFilter in LdapGroupsMapping. (a…

7965c54

…pache#4798) * HADOOP-18388. Allow dynamic groupSearchFilter in LdapGroupsMapping.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

HADOOP-18388. Allow dynamic groupSearchFilter in LdapGroupsMapping. #4798

HADOOP-18388. Allow dynamic groupSearchFilter in LdapGroupsMapping. #4798

ayushtkn commented Aug 24, 2022 •

edited

hadoop-yetus commented Aug 26, 2022

hadoop-yetus commented Aug 31, 2022

hadoop-yetus commented Aug 31, 2022

lmccay left a comment

This comment was marked as outdated.

This comment was marked as outdated.

hadoop-yetus commented Sep 6, 2022

lmccay commented Sep 6, 2022

HADOOP-18388. Allow dynamic groupSearchFilter in LdapGroupsMapping. #4798

HADOOP-18388. Allow dynamic groupSearchFilter in LdapGroupsMapping. #4798

Conversation

ayushtkn commented Aug 24, 2022 • edited

Description of PR

How was this patch tested?

For code changes:

hadoop-yetus commented Aug 26, 2022

hadoop-yetus commented Aug 31, 2022

hadoop-yetus commented Aug 31, 2022

lmccay left a comment

Choose a reason for hiding this comment

This comment was marked as outdated.

This comment was marked as outdated.

hadoop-yetus commented Sep 6, 2022

lmccay commented Sep 6, 2022

ayushtkn commented Aug 24, 2022 •

edited