HADOOP-19081. Move some hadoop-common code to new hadoop-ftp module

[pjfanning]

Description of PR

HADOOP-19081 - try to reduce hadoop-common so that it doesn't have so many dependencies

WIP

• still need to work out if the new hadoop-ftp module should have its classes included in the bundle jars like minicluster and client or whether users who need FTP/SFTP support would need to add the new jar to their classpaths
• the core-default.xml needs to have its FTP configs updated to mention the new module or the configs need to move to a new file that possibly is part of hadoop-ftp module

How was this patch tested?

For code changes:

• Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
• Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
• If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
• If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files?

[hadoop-yetus]

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 23s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 1s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+0 🆗	xmllint	0m 0s		xmllint was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 28 new or modified test files.
			_ trunk Compile Tests _
+0 🆗	mvndep	14m 0s		Maven dependency ordering for branch
+1 💚	mvninstall	21m 6s		trunk passed
+1 💚	compile	9m 57s		trunk passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1
+1 💚	compile	9m 3s		trunk passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06
+1 💚	checkstyle	0m 42s		trunk passed
+1 💚	mvnsite	1m 53s		trunk passed
+1 💚	javadoc	1m 30s		trunk passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1
+1 💚	javadoc	1m 4s		trunk passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06
+1 💚	spotbugs	3m 40s		trunk passed
+1 💚	shadedclient	22m 33s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 20s		Maven dependency ordering for patch
-1 ❌	mvninstall	0m 26s	/patch-mvninstall-hadoop-common-project_hadoop-common.txt	hadoop-common in the patch failed.
-1 ❌	mvninstall	0m 31s	/patch-mvninstall-hadoop-common-project.txt	hadoop-common-project in the patch failed.
-1 ❌	compile	0m 44s	/patch-compile-root-jdkUbuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1.txt	root in the patch failed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1.
-1 ❌	javac	0m 44s	/patch-compile-root-jdkUbuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1.txt	root in the patch failed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1.
-1 ❌	compile	0m 37s	/patch-compile-root-jdkPrivateBuild-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06.txt	root in the patch failed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06.
-1 ❌	javac	0m 37s	/patch-compile-root-jdkPrivateBuild-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06.txt	root in the patch failed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06.
-1 ❌	blanks	0m 0s	/blanks-eol.txt	The patch has 18 line(s) that end in blanks. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
+1 💚	checkstyle	0m 33s		the patch passed
-1 ❌	mvnsite	0m 31s	/patch-mvnsite-hadoop-common-project_hadoop-common.txt	hadoop-common in the patch failed.
-1 ❌	mvnsite	0m 34s	/patch-mvnsite-hadoop-common-project.txt	hadoop-common-project in the patch failed.
+1 💚	javadoc	1m 25s		the patch passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1
+1 💚	javadoc	1m 3s		the patch passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06
-1 ❌	spotbugs	0m 31s	/patch-spotbugs-hadoop-common-project_hadoop-common.txt	hadoop-common in the patch failed.
-1 ❌	spotbugs	0m 12s	/patch-spotbugs-hadoop-common-project_hadoop-ftp.txt	hadoop-ftp in the patch failed.
-1 ❌	spotbugs	0m 50s	/patch-spotbugs-hadoop-common-project.txt	hadoop-common-project in the patch failed.
-1 ❌	shadedclient	8m 21s		patch has errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	0m 32s	/patch-unit-hadoop-common-project_hadoop-common.txt	hadoop-common in the patch failed.
+1 💚	unit	0m 32s		hadoop-ftp in the patch passed.
-1 ❌	unit	3m 13s	/patch-unit-hadoop-common-project.txt	hadoop-common-project in the patch failed.
+1 💚	asflicense	0m 21s		The patch does not generate ASF License warnings.
		102m 54s

Subsystem	Report/Notes
Docker	ClientAPI=1.44 ServerAPI=1.44 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6693/1/artifact/out/Dockerfile
GITHUB PR	#6693
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient codespell detsecrets xmllint spotbugs checkstyle
uname	Linux 63cc2787f246 5.15.0-94-generic #104-Ubuntu SMP Tue Jan 9 15:25:40 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / 796c9c8
Default Java	Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6693/1/testReport/
Max. process+thread count	553 (vs. ulimit of 5500)
modules	C: hadoop-common-project/hadoop-common hadoop-common-project/hadoop-ftp hadoop-common-project U: hadoop-common-project
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6693/1/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

[hadoop-yetus]

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 21s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 1s		codespell was not available.
+0 🆗	detsecrets	0m 1s		detect-secrets was not available.
+0 🆗	xmllint	0m 1s		xmllint was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 28 new or modified test files.
			_ trunk Compile Tests _
+0 🆗	mvndep	14m 2s		Maven dependency ordering for branch
+1 💚	mvninstall	20m 44s		trunk passed
+1 💚	compile	10m 4s		trunk passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1
+1 💚	compile	9m 6s		trunk passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06
+1 💚	checkstyle	0m 44s		trunk passed
+1 💚	mvnsite	1m 54s		trunk passed
+1 💚	javadoc	1m 28s		trunk passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1
+1 💚	javadoc	1m 13s		trunk passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06
+1 💚	spotbugs	3m 38s		trunk passed
+1 💚	shadedclient	22m 32s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 21s		Maven dependency ordering for patch
-1 ❌	mvninstall	0m 29s	/patch-mvninstall-hadoop-common-project_hadoop-common.txt	hadoop-common in the patch failed.
-1 ❌	mvninstall	0m 32s	/patch-mvninstall-hadoop-common-project.txt	hadoop-common-project in the patch failed.
-1 ❌	compile	0m 40s	/patch-compile-root-jdkUbuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1.txt	root in the patch failed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1.
-1 ❌	javac	0m 40s	/patch-compile-root-jdkUbuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1.txt	root in the patch failed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1.
-1 ❌	compile	0m 39s	/patch-compile-root-jdkPrivateBuild-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06.txt	root in the patch failed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06.
-1 ❌	javac	0m 39s	/patch-compile-root-jdkPrivateBuild-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06.txt	root in the patch failed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06.
-1 ❌	blanks	0m 0s	/blanks-eol.txt	The patch has 18 line(s) that end in blanks. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
+1 💚	checkstyle	0m 33s		the patch passed
-1 ❌	mvnsite	0m 27s	/patch-mvnsite-hadoop-common-project_hadoop-common.txt	hadoop-common in the patch failed.
-1 ❌	mvnsite	0m 34s	/patch-mvnsite-hadoop-common-project.txt	hadoop-common-project in the patch failed.
+1 💚	javadoc	1m 23s		the patch passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1
+1 💚	javadoc	1m 12s		the patch passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06
-1 ❌	spotbugs	0m 31s	/patch-spotbugs-hadoop-common-project_hadoop-common.txt	hadoop-common in the patch failed.
-1 ❌	spotbugs	0m 13s	/patch-spotbugs-hadoop-common-project_hadoop-ftp.txt	hadoop-ftp in the patch failed.
-1 ❌	spotbugs	0m 48s	/patch-spotbugs-hadoop-common-project.txt	hadoop-common-project in the patch failed.
-1 ❌	shadedclient	8m 25s		patch has errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	0m 30s	/patch-unit-hadoop-common-project_hadoop-common.txt	hadoop-common in the patch failed.
+1 💚	unit	0m 35s		hadoop-ftp in the patch passed.
-1 ❌	unit	3m 10s	/patch-unit-hadoop-common-project.txt	hadoop-common-project in the patch failed.
+1 💚	asflicense	0m 20s		The patch does not generate ASF License warnings.
		103m 1s

Subsystem	Report/Notes
Docker	ClientAPI=1.44 ServerAPI=1.44 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6693/2/artifact/out/Dockerfile
GITHUB PR	#6693
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient codespell detsecrets xmllint spotbugs checkstyle
uname	Linux 9fa2319e2e8e 5.15.0-94-generic #104-Ubuntu SMP Tue Jan 9 15:25:40 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / 796c9c8
Default Java	Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6693/2/testReport/
Max. process+thread count	563 (vs. ulimit of 5500)
modules	C: hadoop-common-project/hadoop-common hadoop-common-project/hadoop-ftp hadoop-common-project U: hadoop-common-project
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6693/2/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

[hadoop-yetus]

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 19s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 1s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+0 🆗	xmllint	0m 0s		xmllint was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 30 new or modified test files.
			_ trunk Compile Tests _
+0 🆗	mvndep	14m 16s		Maven dependency ordering for branch
+1 💚	mvninstall	19m 57s		trunk passed
+1 💚	compile	8m 52s		trunk passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1
+1 💚	compile	8m 5s		trunk passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06
+1 💚	checkstyle	0m 41s		trunk passed
+1 💚	mvnsite	1m 49s		trunk passed
+1 💚	javadoc	1m 30s		trunk passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1
+1 💚	javadoc	1m 13s		trunk passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06
+1 💚	spotbugs	3m 21s		trunk passed
+1 💚	shadedclient	21m 28s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 21s		Maven dependency ordering for patch
+1 💚	mvninstall	1m 14s		the patch passed
+1 💚	compile	8m 24s		the patch passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1
+1 💚	javac	8m 24s		the patch passed
+1 💚	compile	8m 3s		the patch passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06
+1 💚	javac	8m 3s		the patch passed
-1 ❌	blanks	0m 0s	/blanks-eol.txt	The patch has 18 line(s) that end in blanks. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
-0 ⚠️	checkstyle	0m 39s	/results-checkstyle-hadoop-common-project.txt	hadoop-common-project: The patch generated 1 new + 17 unchanged - 0 fixed = 18 total (was 17)
+1 💚	mvnsite	2m 13s		the patch passed
+1 💚	javadoc	1m 54s		the patch passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1
+1 💚	javadoc	1m 43s		the patch passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06
-1 ❌	spotbugs	0m 22s	/patch-spotbugs-hadoop-common-project_hadoop-ftp.txt	hadoop-ftp in the patch failed.
-1 ❌	spotbugs	1m 55s	/patch-spotbugs-hadoop-common-project.txt	hadoop-common-project in the patch failed.
+1 💚	shadedclient	20m 51s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	16m 27s	/patch-unit-hadoop-common-project_hadoop-common.txt	hadoop-common in the patch passed.
+1 💚	unit	0m 52s		hadoop-ftp in the patch passed.
-1 ❌	unit	22m 54s	/patch-unit-hadoop-common-project.txt	hadoop-common-project in the patch passed.
+1 💚	asflicense	0m 37s		The patch does not generate ASF License warnings.
		173m 50s

Reason	Tests
Failed junit tests	hadoop.fs.TestDelegateToFileSystem
	hadoop.conf.TestCommonConfigurationFields
	hadoop.fs.TestDelegateToFileSystem
	hadoop.conf.TestCommonConfigurationFields

Subsystem	Report/Notes
Docker	ClientAPI=1.44 ServerAPI=1.44 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6693/3/artifact/out/Dockerfile
GITHUB PR	#6693
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient codespell detsecrets xmllint spotbugs checkstyle
uname	Linux 5ac33755ae4e 5.15.0-94-generic #104-Ubuntu SMP Tue Jan 9 15:25:40 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / 6cdafb2
Default Java	Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6693/3/testReport/
Max. process+thread count	1274 (vs. ulimit of 5500)
modules	C: hadoop-common-project/hadoop-common hadoop-common-project/hadoop-ftp hadoop-common-project U: hadoop-common-project
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6693/3/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

[pjfanning]

@steveloughran I'm not sure that this PR is worth pursuing. The FTP classes are referenced in core-defaults.xml

https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-common/src/main/resources/core-default.xml

So it seems like values would need to be removed from core-defaults.xml. I am not an expert on Hadoop configuration and am unsure what should be done to the configs if we want to continue with this. Hadoop HA would have similar issues if we wanted to move the HA code out of hadoop-common.

[steveloughran]

no, we can stick stuff in core-default. there's s3a options there, etc.

[steveloughran]

I do think this is good, as it cuts down the dependencies.

I think in the past some of the hdfs failover code uses jsch, but think that zk replaced all of that a long time ago. still worth warning hdfs and yarn about though.

I'd propose moving to hadoop-tools

[pjfanning]

@steveloughran sure, the classes can move to an existing jar. Would you be able to suggest what to do about core-default.xml? The FTP related props will probably have to move too. The hadoop-common config tests break because I've moved the code.

[ayushtkn]

Just passing by-

The hadoop-common config tests break because I've moved the code.

AFAIK there are ways to exclude some configs in those tests, putting in some HashMap, like xmlPropsToSkipCompare or other, but there was a way I believe

[steveloughran]

• There's a dummy fs in org.apache.hadoop.fs.TestFileSystemCanonicalization.DummyFileSystem; that could be pulled up and used in {{TestDelegateToFileSystem}}
• TestCommonConfigurationFields: remove FtpConfigKeys.class, from configurationClasses.

[hadoop-yetus]

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 20s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+0 🆗	xmllint	0m 0s		xmllint was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 32 new or modified test files.
			_ trunk Compile Tests _
+0 🆗	mvndep	14m 10s		Maven dependency ordering for branch
+1 💚	mvninstall	21m 10s		trunk passed
+1 💚	compile	9m 28s		trunk passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1
+1 💚	compile	8m 47s		trunk passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06
+1 💚	checkstyle	0m 57s		trunk passed
+1 💚	mvnsite	1m 49s		trunk passed
+1 💚	javadoc	1m 32s		trunk passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1
+1 💚	javadoc	1m 11s		trunk passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06
+1 💚	spotbugs	3m 31s		trunk passed
+1 💚	shadedclient	23m 59s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 22s		Maven dependency ordering for patch
+1 💚	mvninstall	1m 15s		the patch passed
+1 💚	compile	9m 16s		the patch passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1
+1 💚	javac	9m 16s		the patch passed
+1 💚	compile	9m 21s		the patch passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06
+1 💚	javac	9m 21s		the patch passed
-1 ❌	blanks	0m 0s	/blanks-eol.txt	The patch has 18 line(s) that end in blanks. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
-0 ⚠️	checkstyle	0m 39s	/results-checkstyle-hadoop-common-project.txt	hadoop-common-project: The patch generated 1 new + 17 unchanged - 0 fixed = 18 total (was 17)
+1 💚	mvnsite	2m 18s		the patch passed
+1 💚	javadoc	1m 47s		the patch passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1
+1 💚	javadoc	1m 29s		the patch passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06
-1 ❌	spotbugs	0m 23s	/patch-spotbugs-hadoop-common-project_hadoop-ftp.txt	hadoop-ftp in the patch failed.
-1 ❌	spotbugs	1m 57s	/patch-spotbugs-hadoop-common-project.txt	hadoop-common-project in the patch failed.
+1 💚	shadedclient	23m 51s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	16m 14s		hadoop-common in the patch passed.
+1 💚	unit	0m 50s		hadoop-ftp in the patch passed.
+1 💚	unit	23m 4s		hadoop-common-project in the patch passed.
+1 💚	asflicense	0m 41s		The patch does not generate ASF License warnings.
		184m 7s

Subsystem	Report/Notes
Docker	ClientAPI=1.44 ServerAPI=1.44 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6693/4/artifact/out/Dockerfile
GITHUB PR	#6693
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient codespell detsecrets xmllint spotbugs checkstyle
uname	Linux 3e610f4aa5bb 5.15.0-94-generic #104-Ubuntu SMP Tue Jan 9 15:25:40 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / 8304d1c
Default Java	Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6693/4/testReport/
Max. process+thread count	3153 (vs. ulimit of 5500)
modules	C: hadoop-common-project/hadoop-common hadoop-common-project/hadoop-ftp hadoop-common-project U: hadoop-common-project
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6693/4/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

[hadoop-yetus]

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 21s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 1s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+0 🆗	xmllint	0m 0s		xmllint was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 32 new or modified test files.
			_ trunk Compile Tests _
+0 🆗	mvndep	14m 17s		Maven dependency ordering for branch
+1 💚	mvninstall	20m 42s		trunk passed
+1 💚	compile	9m 56s		trunk passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1
+1 💚	compile	9m 2s		trunk passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06
+1 💚	checkstyle	2m 20s		trunk passed
+1 💚	mvnsite	1m 43s		trunk passed
+1 💚	javadoc	1m 53s		trunk passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1
+1 💚	javadoc	1m 38s		trunk passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06
+1 💚	spotbugs	4m 6s		trunk passed
+1 💚	shadedclient	24m 20s		branch has no errors when building and testing our client artifacts.
-0 ⚠️	patch	24m 34s		Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 20s		Maven dependency ordering for patch
+1 💚	mvninstall	1m 30s		the patch passed
+1 💚	compile	10m 7s		the patch passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1
+1 💚	javac	10m 7s		the patch passed
+1 💚	compile	8m 55s		the patch passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06
+1 💚	javac	8m 55s		the patch passed
-1 ❌	blanks	0m 0s	/blanks-eol.txt	The patch has 18 line(s) that end in blanks. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
-0 ⚠️	checkstyle	2m 19s	/results-checkstyle-root.txt	root: The patch generated 1 new + 17 unchanged - 0 fixed = 18 total (was 17)
+1 💚	mvnsite	2m 4s		the patch passed
+1 💚	javadoc	2m 4s		the patch passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1
+1 💚	javadoc	2m 1s		the patch passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06
-1 ❌	spotbugs	0m 20s	/patch-spotbugs-hadoop-tools_hadoop-ftp.txt	hadoop-ftp in the patch failed.
-1 ❌	spotbugs	2m 39s	/patch-spotbugs-hadoop-tools.txt	hadoop-tools in the patch failed.
+1 💚	shadedclient	23m 40s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	16m 41s		hadoop-common in the patch passed.
+1 💚	unit	0m 41s		hadoop-ftp in the patch passed.
+1 💚	unit	67m 22s		hadoop-tools in the patch passed.
+1 💚	asflicense	0m 44s		The patch does not generate ASF License warnings.
		234m 55s

Subsystem	Report/Notes
Docker	ClientAPI=1.44 ServerAPI=1.44 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6693/5/artifact/out/Dockerfile
GITHUB PR	#6693
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient codespell detsecrets xmllint spotbugs checkstyle
uname	Linux 796802540ae4 5.15.0-94-generic #104-Ubuntu SMP Tue Jan 9 15:25:40 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / 18bc7e4
Default Java	Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6693/5/testReport/
Max. process+thread count	2153 (vs. ulimit of 5500)
modules	C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-ftp hadoop-tools U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6693/5/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

[hadoop-yetus]

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 23s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 1s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+0 🆗	xmllint	0m 0s		xmllint was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 32 new or modified test files.
			_ trunk Compile Tests _
+0 🆗	mvndep	14m 19s		Maven dependency ordering for branch
+1 💚	mvninstall	20m 11s		trunk passed
+1 💚	compile	8m 45s		trunk passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1
+1 💚	compile	7m 59s		trunk passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06
+1 💚	checkstyle	2m 4s		trunk passed
+1 💚	mvnsite	1m 44s		trunk passed
+1 💚	javadoc	2m 2s		trunk passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1
+1 💚	javadoc	1m 51s		trunk passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06
+1 💚	spotbugs	4m 24s		trunk passed
+1 💚	shadedclient	22m 40s		branch has no errors when building and testing our client artifacts.
-0 ⚠️	patch	22m 55s		Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 21s		Maven dependency ordering for patch
+1 💚	mvninstall	1m 34s		the patch passed
+1 💚	compile	8m 24s		the patch passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1
+1 💚	javac	8m 24s		the patch passed
+1 💚	compile	8m 5s		the patch passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06
+1 💚	javac	8m 5s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	2m 1s		the patch passed
+1 💚	mvnsite	2m 6s		the patch passed
+1 💚	javadoc	2m 15s		the patch passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1
+1 💚	javadoc	2m 15s		the patch passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06
-1 ❌	spotbugs	0m 22s	/patch-spotbugs-hadoop-tools_hadoop-ftp.txt	hadoop-ftp in the patch failed.
-1 ❌	spotbugs	2m 48s	/patch-spotbugs-hadoop-tools.txt	hadoop-tools in the patch failed.
+1 💚	shadedclient	20m 39s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	16m 28s		hadoop-common in the patch passed.
+1 💚	unit	0m 50s		hadoop-ftp in the patch passed.
+1 💚	unit	67m 24s		hadoop-tools in the patch passed.
+1 💚	asflicense	0m 41s		The patch does not generate ASF License warnings.
		225m 59s

Subsystem	Report/Notes
Docker	ClientAPI=1.44 ServerAPI=1.44 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6693/6/artifact/out/Dockerfile
GITHUB PR	#6693
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient codespell detsecrets xmllint spotbugs checkstyle
uname	Linux b3e084113086 5.15.0-94-generic #104-Ubuntu SMP Tue Jan 9 15:25:40 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / 73a9ecd
Default Java	Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6693/6/testReport/
Max. process+thread count	3153 (vs. ulimit of 5500)
modules	C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-ftp hadoop-tools U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6693/6/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

[hadoop-yetus]

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 31s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 1s		No case conflicting files found.
+0 🆗	codespell	0m 1s		codespell was not available.
+0 🆗	detsecrets	0m 1s		detect-secrets was not available.
+0 🆗	xmllint	0m 1s		xmllint was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 32 new or modified test files.
			_ trunk Compile Tests _
+0 🆗	mvndep	14m 53s		Maven dependency ordering for branch
+1 💚	mvninstall	33m 19s		trunk passed
+1 💚	compile	17m 28s		trunk passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1
+1 💚	compile	16m 11s		trunk passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06
+1 💚	checkstyle	4m 20s		trunk passed
+1 💚	mvnsite	3m 20s		trunk passed
+1 💚	javadoc	3m 34s		trunk passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1
+1 💚	javadoc	3m 0s		trunk passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06
+1 💚	spotbugs	7m 41s		trunk passed
+1 💚	shadedclient	37m 54s		branch has no errors when building and testing our client artifacts.
-0 ⚠️	patch	38m 20s		Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 34s		Maven dependency ordering for patch
+1 💚	mvninstall	2m 46s		the patch passed
+1 💚	compile	18m 7s		the patch passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1
+1 💚	javac	18m 7s		the patch passed
+1 💚	compile	16m 35s		the patch passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06
+1 💚	javac	16m 35s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	4m 52s		the patch passed
+1 💚	mvnsite	3m 54s		the patch passed
+1 💚	javadoc	3m 56s		the patch passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1
+1 💚	javadoc	3m 26s		the patch passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06
-1 ❌	spotbugs	0m 36s	/patch-spotbugs-hadoop-tools_hadoop-ftp.txt	hadoop-ftp in the patch failed.
-1 ❌	spotbugs	4m 40s	/patch-spotbugs-hadoop-tools.txt	hadoop-tools in the patch failed.
+1 💚	shadedclient	35m 53s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	18m 57s		hadoop-common in the patch passed.
+1 💚	unit	1m 3s		hadoop-ftp in the patch passed.
+1 💚	unit	74m 35s		hadoop-tools in the patch passed.
+1 💚	asflicense	1m 11s		The patch does not generate ASF License warnings.
		337m 58s

Subsystem	Report/Notes
Docker	ClientAPI=1.45 ServerAPI=1.45 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6693/7/artifact/out/Dockerfile
GITHUB PR	#6693
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient codespell detsecrets xmllint spotbugs checkstyle
uname	Linux 14277288ff8d 5.15.0-94-generic #104-Ubuntu SMP Tue Jan 9 15:25:40 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / fbe7009
Default Java	Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6693/7/testReport/
Max. process+thread count	3149 (vs. ulimit of 5500)
modules	C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-ftp hadoop-tools U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6693/7/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

[hadoop-yetus]

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 20s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 1s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+0 🆗	xmllint	0m 0s		xmllint was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 32 new or modified test files.
			_ trunk Compile Tests _
+0 🆗	mvndep	14m 12s		Maven dependency ordering for branch
+1 💚	mvninstall	20m 0s		trunk passed
+1 💚	compile	8m 46s		trunk passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1
+1 💚	compile	8m 10s		trunk passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06
+1 💚	checkstyle	2m 5s		trunk passed
+1 💚	mvnsite	1m 52s		trunk passed
+1 💚	javadoc	2m 1s		trunk passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1
+1 💚	javadoc	1m 52s		trunk passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06
+1 💚	spotbugs	4m 22s		trunk passed
+1 💚	shadedclient	22m 11s		branch has no errors when building and testing our client artifacts.
-0 ⚠️	patch	22m 27s		Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 21s		Maven dependency ordering for patch
+1 💚	mvninstall	1m 29s		the patch passed
+1 💚	compile	8m 35s		the patch passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1
+1 💚	javac	8m 35s		the patch passed
+1 💚	compile	8m 4s		the patch passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06
+1 💚	javac	8m 4s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	2m 1s		the patch passed
+1 💚	mvnsite	2m 9s		the patch passed
+1 💚	javadoc	2m 25s		the patch passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1
+1 💚	javadoc	2m 21s		the patch passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06
+1 💚	spotbugs	5m 22s		the patch passed
+1 💚	shadedclient	22m 47s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	16m 23s		hadoop-common in the patch passed.
+1 💚	unit	0m 45s		hadoop-ftp in the patch passed.
+1 💚	unit	67m 42s		hadoop-tools in the patch passed.
+1 💚	asflicense	0m 40s		The patch does not generate ASF License warnings.
		227m 15s

Subsystem	Report/Notes
Docker	ClientAPI=1.44 ServerAPI=1.44 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6693/8/artifact/out/Dockerfile
GITHUB PR	#6693
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient codespell detsecrets xmllint spotbugs checkstyle
uname	Linux 858e7eaa4059 5.15.0-94-generic #104-Ubuntu SMP Tue Jan 9 15:25:40 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / 7097202
Default Java	Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6693/8/testReport/
Max. process+thread count	3153 (vs. ulimit of 5500)
modules	C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-ftp hadoop-tools U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6693/8/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

[steveloughran]

core move looks good; we will have to tag as incompatible to
-anyone who wants the things on their CP
-anyone who wants it in common/lib and to use the ftp/sftp clients
without doing things there.

Do we still want to stitch it into the hadoop classpath?

[steveloughran]

does this class get into the JAR? if so it should go into test/resources

[github-actions]

We're closing this stale PR because it has been open for 100 days with no activity. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue manageable.
If you feel like this was a mistake, or you would like to continue working on it, please feel free to re-open it and ask for a committer to remove the stale tag and review again.
Thanks all for your contribution.