YARN-11255. Support loading alternative docker client config from system environment #4884

ashutoshcipher · 2022-09-13T17:33:23Z

Description of PR

Support loading alternative docker client config from system environment

When using YARN docker support, although the hadoop shell supported

-docker_client_config

to pass the client config file that contains security token to generate the docker config for each job as a temporary file.

For other applications that submit jobs to YARN, e.g. Spark, which loads the docker setting via system environment e.g.

spark.executorEnv.*

will not be able to add those authorization token because this system environment isn't considered in YARN.

Add genetic solution to handle these kind of cases without making changes in spark code or others

Eg

When using remote container registry, the YARN_CONTAINER_RUNTIME_DOCKER_CLIENT_CONFIG must reference the config.json
file containing the credentials used to authenticate.

DOCKER_IMAGE_NAME=hadoop-docker 
DOCKER_CLIENT_CONFIG=hdfs:///user/hadoop/config.json
spark-submit --master yarn \
--deploy-mode cluster \
--conf spark.executorEnv.YARN_CONTAINER_RUNTIME_TYPE=docker \
--conf spark.executorEnv.YARN_CONTAINER_RUNTIME_DOCKER_IMAGE=$DOCKER_IMAGE_NAME \
--conf spark.executorEnv.YARN_CONTAINER_RUNTIME_DOCKER_CLIENT_CONFIG=$DOCKER_CLIENT_CONFIG \
--conf spark.yarn.appMasterEnv.YARN_CONTAINER_RUNTIME_TYPE=docker \
--conf spark.yarn.appMasterEnv.YARN_CONTAINER_RUNTIME_DOCKER_IMAGE=$DOCKER_IMAGE_NAME \
--conf spark.yarn.appMasterEnv.YARN_CONTAINER_RUNTIME_DOCKER_CLIENT_CONFIG=$DOCKER_CLIENT_CONFIG \
sparkR.R

How was this patch tested?

Unit tests and manually verified by running on EMR.

For code changes:

Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files?

…tem environment

hadoop-yetus · 2022-09-13T19:39:46Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 43s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 1s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+1 💚	@author	0m 1s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 1 new or modified test files.
			_ trunk Compile Tests _
+1 💚	mvninstall	40m 25s		trunk passed
+1 💚	compile	1m 46s		trunk passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04
+1 💚	compile	1m 35s		trunk passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07
+1 💚	checkstyle	0m 48s		trunk passed
+1 💚	mvnsite	0m 58s		trunk passed
+1 💚	javadoc	1m 0s		trunk passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04
+1 💚	javadoc	0m 53s		trunk passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07
+1 💚	spotbugs	1m 51s		trunk passed
+1 💚	shadedclient	21m 35s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	0m 41s		the patch passed
+1 💚	compile	1m 25s		the patch passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04
+1 💚	javac	1m 25s		the patch passed
+1 💚	compile	1m 21s		the patch passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07
+1 💚	javac	1m 21s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	0m 29s	/results-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager: The patch generated 1 new + 15 unchanged - 0 fixed = 16 total (was 15)
+1 💚	mvnsite	0m 42s		the patch passed
+1 💚	javadoc	0m 38s		the patch passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04
+1 💚	javadoc	0m 33s		the patch passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07
+1 💚	spotbugs	1m 31s		the patch passed
+1 💚	shadedclient	20m 29s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	24m 4s	/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt	hadoop-yarn-server-nodemanager in the patch passed.
+1 💚	asflicense	0m 50s		The patch does not generate ASF License warnings.
		124m 55s

Reason	Tests
Failed junit tests	hadoop.yarn.server.nodemanager.containermanager.logaggregation.TestLogAggregationService

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4884/1/artifact/out/Dockerfile
GITHUB PR	#4884
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname	Linux 73697f8f86f0 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4 01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `79be2be`
Default Java	Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4884/1/testReport/
Max. process+thread count	562 (vs. ulimit of 5500)
modules	C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4884/1/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2022-09-13T22:12:50Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 50s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 1s		codespell was not available.
+0 🆗	detsecrets	0m 1s		detect-secrets was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 1 new or modified test files.
			_ trunk Compile Tests _
+1 💚	mvninstall	44m 7s		trunk passed
+1 💚	compile	1m 38s		trunk passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04
+1 💚	compile	1m 35s		trunk passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07
+1 💚	checkstyle	0m 53s		trunk passed
+1 💚	mvnsite	0m 57s		trunk passed
+1 💚	javadoc	0m 58s		trunk passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04
+1 💚	javadoc	0m 45s		trunk passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07
+1 💚	spotbugs	1m 48s		trunk passed
+1 💚	shadedclient	21m 15s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	0m 42s		the patch passed
+1 💚	compile	1m 28s		the patch passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04
+1 💚	javac	1m 28s		the patch passed
+1 💚	compile	1m 26s		the patch passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07
+1 💚	javac	1m 26s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	0m 30s		the patch passed
+1 💚	mvnsite	0m 46s		the patch passed
+1 💚	javadoc	0m 37s		the patch passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04
+1 💚	javadoc	0m 33s		the patch passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07
+1 💚	spotbugs	1m 29s		the patch passed
+1 💚	shadedclient	21m 7s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	24m 7s	/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt	hadoop-yarn-server-nodemanager in the patch passed.
+1 💚	asflicense	0m 48s		The patch does not generate ASF License warnings.
		129m 3s

Reason	Tests
Failed junit tests	hadoop.yarn.server.nodemanager.containermanager.logaggregation.TestLogAggregationService

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4884/2/artifact/out/Dockerfile
GITHUB PR	#4884
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname	Linux b96d121eb3ca 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4 01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `71159b4`
Default Java	Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4884/2/testReport/
Max. process+thread count	554 (vs. ulimit of 5500)
modules	C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4884/2/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

ashutoshcipher · 2022-09-14T01:51:50Z

UT Failure

hadoop.yarn.server.nodemanager.containermanager.logaggregation.TestLogAggregationService

is not relevant to this change. Tried running it in local and its passing.

PrabhuJoseph · 2022-09-15T04:13:41Z

...doop/yarn/server/nodemanager/containermanager/linux/runtime/DockerLinuxContainerRuntime.java

@@ -215,6 +217,12 @@ public class DockerLinuxContainerRuntime extends OCIContainerRuntime {
  public static final String ENV_DOCKER_CONTAINER_IMAGE =
      "YARN_CONTAINER_RUNTIME_DOCKER_IMAGE";
  @InterfaceAudience.Private
+  public static final String ENV_DOCKER_CONTAINER_IMAGE_FILE =


Thanks @ashutoshcipher for the patch. This variable is not used anywhere. Could you remove it if not needed.

Thanks @PrabhuJoseph. Addressed in my latest commit.

PrabhuJoseph · 2022-09-15T04:20:37Z

Thanks @ashutoshcipher for the patch. Overall looks good to me except a minor one.

Could you also document the steps to run an application like Spark using this new environment variable YARN_CONTAINER_RUNTIME_DOCKER_CLIENT_CONFIG in https://hadoop.apache.org/docs/r3.0.2/hadoop-yarn/hadoop-yarn-site/DockerContainers.html

For Example:

DOCKER_IMAGE_NAME=hadoop-docker 
DOCKER_CLIENT_CONFIG=hdfs:///user/hadoop/config.json
spark-submit --master yarn \
--deploy-mode cluster \
--conf spark.executorEnv.YARN_CONTAINER_RUNTIME_TYPE=docker \
--conf spark.executorEnv.YARN_CONTAINER_RUNTIME_DOCKER_IMAGE=$DOCKER_IMAGE_NAME \
--conf spark.executorEnv.YARN_CONTAINER_RUNTIME_DOCKER_CLIENT_CONFIG=$DOCKER_CLIENT_CONFIG \
--conf spark.yarn.appMasterEnv.YARN_CONTAINER_RUNTIME_TYPE=docker \
--conf spark.yarn.appMasterEnv.YARN_CONTAINER_RUNTIME_DOCKER_IMAGE=$DOCKER_IMAGE_NAME \
--conf spark.yarn.appMasterEnv.YARN_CONTAINER_RUNTIME_DOCKER_CLIENT_CONFIG=$DOCKER_CLIENT_CONFIG \
sparkR.R

ashutoshcipher · 2022-09-15T12:16:29Z

Thanks @ashutoshcipher for the patch. Overall looks good to me except a minor one.

Could you also document the steps to run an application like Spark using this new environment variable YARN_CONTAINER_RUNTIME_DOCKER_CLIENT_CONFIG in https://hadoop.apache.org/docs/r3.0.2/hadoop-yarn/hadoop-yarn-site/DockerContainers.html

For Example:
DOCKER_IMAGE_NAME=hadoop-docker 
DOCKER_CLIENT_CONFIG=hdfs:///user/hadoop/config.json
spark-submit --master yarn \
--deploy-mode cluster \
--conf spark.executorEnv.YARN_CONTAINER_RUNTIME_TYPE=docker \
--conf spark.executorEnv.YARN_CONTAINER_RUNTIME_DOCKER_IMAGE=$DOCKER_IMAGE_NAME \
--conf spark.executorEnv.YARN_CONTAINER_RUNTIME_DOCKER_CLIENT_CONFIG=$DOCKER_CLIENT_CONFIG \
--conf spark.yarn.appMasterEnv.YARN_CONTAINER_RUNTIME_TYPE=docker \
--conf spark.yarn.appMasterEnv.YARN_CONTAINER_RUNTIME_DOCKER_IMAGE=$DOCKER_IMAGE_NAME \
--conf spark.yarn.appMasterEnv.YARN_CONTAINER_RUNTIME_DOCKER_CLIENT_CONFIG=$DOCKER_CLIENT_CONFIG \
sparkR.R

Thanks @PrabhuJoseph . Added the documentation

hadoop-yetus · 2022-09-15T15:13:14Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 38s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+0 🆗	markdownlint	0m 0s		markdownlint was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 1 new or modified test files.
			_ trunk Compile Tests _
+0 🆗	mvndep	15m 30s		Maven dependency ordering for branch
+1 💚	mvninstall	25m 39s		trunk passed
+1 💚	compile	10m 0s		trunk passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04
+1 💚	compile	8m 51s		trunk passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07
+1 💚	checkstyle	2m 8s		trunk passed
+1 💚	mvnsite	2m 9s		trunk passed
+1 💚	javadoc	2m 17s		trunk passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04
+1 💚	javadoc	1m 55s		trunk passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07
+0 🆗	spotbugs	0m 55s		branch/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site no spotbugs output file (spotbugsXml.xml)
+1 💚	shadedclient	21m 1s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 25s		Maven dependency ordering for patch
+1 💚	mvninstall	1m 3s		the patch passed
+1 💚	compile	9m 11s		the patch passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04
+1 💚	javac	9m 11s		the patch passed
+1 💚	compile	8m 49s		the patch passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07
+1 💚	javac	8m 49s		the patch passed
-1 ❌	blanks	0m 0s	/blanks-eol.txt	The patch has 1 line(s) that end in blanks. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
+1 💚	checkstyle	2m 0s		the patch passed
+1 💚	mvnsite	2m 4s		the patch passed
+1 💚	javadoc	1m 56s		the patch passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04
+1 💚	javadoc	1m 44s		the patch passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07
+0 🆗	spotbugs	0m 43s		hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site has no data from spotbugs
+1 💚	shadedclient	23m 29s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	24m 31s		hadoop-yarn-server-nodemanager in the patch passed.
+1 💚	unit	0m 51s		hadoop-yarn-site in the patch passed.
+1 💚	asflicense	1m 9s		The patch does not generate ASF License warnings.
		177m 18s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4884/3/artifact/out/Dockerfile
GITHUB PR	#4884
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets markdownlint
uname	Linux 74a4f5d1d577 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4 01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `21aa9ab`
Default Java	Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4884/3/testReport/
Max. process+thread count	559 (vs. ulimit of 5500)
modules	C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site U: hadoop-yarn-project/hadoop-yarn
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4884/3/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2022-09-15T22:11:27Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 58s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+0 🆗	markdownlint	0m 0s		markdownlint was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 1 new or modified test files.
			_ trunk Compile Tests _
+0 🆗	mvndep	16m 14s		Maven dependency ordering for branch
+1 💚	mvninstall	28m 37s		trunk passed
+1 💚	compile	10m 37s		trunk passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04
+1 💚	compile	9m 6s		trunk passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07
+1 💚	checkstyle	2m 2s		trunk passed
+1 💚	mvnsite	1m 57s		trunk passed
+1 💚	javadoc	1m 51s		trunk passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04
+1 💚	javadoc	1m 40s		trunk passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07
+0 🆗	spotbugs	0m 42s		branch/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site no spotbugs output file (spotbugsXml.xml)
+1 💚	shadedclient	23m 31s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 25s		Maven dependency ordering for patch
+1 💚	mvninstall	0m 57s		the patch passed
+1 💚	compile	9m 56s		the patch passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04
+1 💚	javac	9m 56s		the patch passed
+1 💚	compile	9m 3s		the patch passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07
+1 💚	javac	9m 3s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	1m 51s		the patch passed
+1 💚	mvnsite	1m 43s		the patch passed
+1 💚	javadoc	1m 30s		the patch passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04
+1 💚	javadoc	1m 28s		the patch passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07
+0 🆗	spotbugs	0m 38s		hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site has no data from spotbugs
+1 💚	shadedclient	23m 43s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	23m 58s		hadoop-yarn-server-nodemanager in the patch passed.
+1 💚	unit	0m 38s		hadoop-yarn-site in the patch passed.
+1 💚	asflicense	1m 3s		The patch does not generate ASF License warnings.
		182m 8s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4884/4/artifact/out/Dockerfile
GITHUB PR	#4884
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets markdownlint
uname	Linux 57108abb0d6c 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4 01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `a106f38`
Default Java	Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4884/4/testReport/
Max. process+thread count	545 (vs. ulimit of 5500)
modules	C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site U: hadoop-yarn-project/hadoop-yarn
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4884/4/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

PrabhuJoseph · 2022-09-19T07:59:01Z

Thanks @ashutoshcipher for the patch. LGTM, +1. Will commit it if no other comments.

ashutoshcipher · 2022-09-20T03:15:36Z

Thanks @PrabhuJoseph for your reviewing and approving.

…tem environment (apache#4884) Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>

YARN-11255. Support loading alternative docker client config from sys…

79be2be

…tem environment

Fixing stylecheck

71159b4

PrabhuJoseph reviewed Sep 15, 2022

View reviewed changes

Adding documentation and removing unused property

21aa9ab

removing extra space at end of line

a106f38

ashutoshcipher requested a review from PrabhuJoseph September 16, 2022 04:15

PrabhuJoseph merged commit 917aef7 into apache:trunk Sep 21, 2022

HarshitGupta11 pushed a commit to HarshitGupta11/hadoop that referenced this pull request Nov 28, 2022

YARN-11255. Support loading alternative docker client config from sys…

d775a39

…tem environment (apache#4884) Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

YARN-11255. Support loading alternative docker client config from system environment #4884

YARN-11255. Support loading alternative docker client config from system environment #4884

ashutoshcipher commented Sep 13, 2022 •

edited

Loading

hadoop-yetus commented Sep 13, 2022

hadoop-yetus commented Sep 13, 2022

ashutoshcipher commented Sep 14, 2022

PrabhuJoseph Sep 15, 2022

ashutoshcipher Sep 15, 2022

PrabhuJoseph commented Sep 15, 2022

ashutoshcipher commented Sep 15, 2022

hadoop-yetus commented Sep 15, 2022

hadoop-yetus commented Sep 15, 2022

PrabhuJoseph commented Sep 19, 2022

ashutoshcipher commented Sep 20, 2022

YARN-11255. Support loading alternative docker client config from system environment #4884

YARN-11255. Support loading alternative docker client config from system environment #4884

Conversation

ashutoshcipher commented Sep 13, 2022 • edited Loading

Description of PR

How was this patch tested?

For code changes:

hadoop-yetus commented Sep 13, 2022

hadoop-yetus commented Sep 13, 2022

ashutoshcipher commented Sep 14, 2022

PrabhuJoseph Sep 15, 2022

Choose a reason for hiding this comment

ashutoshcipher Sep 15, 2022

Choose a reason for hiding this comment

PrabhuJoseph commented Sep 15, 2022

ashutoshcipher commented Sep 15, 2022

hadoop-yetus commented Sep 15, 2022

hadoop-yetus commented Sep 15, 2022

PrabhuJoseph commented Sep 19, 2022

ashutoshcipher commented Sep 20, 2022

ashutoshcipher commented Sep 13, 2022 •

edited

Loading