HDFS-16462. Make HDFS get tool cross platform #4003

GauthamBanasandra · 2022-02-20T07:07:52Z

Description of PR

The source files for hdfs_get uses getopt for parsing the command line arguments. getopt is available only on Linux and thus, isn't cross platform. We need to replace getopt with boost::program_options to make these tools cross platform.

hdfs_get has the same functionality as hdfs_copyToLocal. Thus, we're reusing the same to implement hdfs_get.

How was this patch tested?

Given that hdfs_get has the same functionality as hdfs_copyToLocal, we've implemented hdfs::tools::Get by deriving from hdfs::tools::CopyToLocal. Since we've already tested hdfs::tools::CopyToLocal for functional correctness, we don't need to do any additional testing to validate the same for hdfs::tools::Get.

However, we do ensure that the correct tool name shows up for hdfs_copyToLocal and hdfs_get.

hdfs_copyToLocal

$ ./hdfs_copyToLocal -h

hdfs_get

$ ./hdfs_get -h

For code changes:

Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files?

hadoop-yetus · 2022-02-23T13:40:53Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 57s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 1s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 4 new or modified test files.
			_ trunk Compile Tests _
+1 💚	mvninstall	35m 52s		trunk passed
+1 💚	compile	3m 28s		trunk passed
+1 💚	mvnsite	0m 23s		trunk passed
+1 💚	shadedclient	62m 46s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	0m 14s		the patch passed
+1 💚	compile	3m 24s		the patch passed
+1 💚	cc	3m 24s		the patch passed
+1 💚	golang	3m 24s		the patch passed
+1 💚	javac	3m 24s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	mvnsite	0m 15s		the patch passed
+1 💚	shadedclient	22m 11s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	37m 7s	/patch-unit-hadoop-hdfs-project_hadoop-hdfs-native-client.txt	hadoop-hdfs-native-client in the patch failed.
+1 💚	asflicense	0m 31s		The patch does not generate ASF License warnings.
		129m 36s

Reason	Tests
Failed CTEST tests	test_test_libhdfs_ops_hdfs_static
	test_test_libhdfs_threaded_hdfs_static
	test_test_libhdfs_zerocopy_hdfs_static
	test_libhdfs_threaded_hdfspp_test_shim_static
	libhdfs_mini_stress_valgrind_hdfspp_test_static
	memcheck_libhdfs_mini_stress_valgrind_hdfspp_test_static
	test_libhdfs_mini_stress_hdfspp_test_shim_static
	test_hdfs_ext_hdfspp_test_shim_static

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4003/6/artifact/out/Dockerfile
GITHUB PR	#4003
Optional Tests	dupname asflicense compile cc mvnsite javac unit codespell golang
uname	Linux b63fedc65a17 4.15.0-153-generic #160-Ubuntu SMP Thu Jul 29 06:54:29 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `c29a231`
Default Java	Red Hat, Inc.-1.8.0_322-b06
CTEST	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4003/6/artifact/out/patch-hadoop-hdfs-project_hadoop-hdfs-native-client-ctest.txt
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4003/6/testReport/
Max. process+thread count	615 (vs. ulimit of 5500)
modules	C: hadoop-hdfs-project/hadoop-hdfs-native-client U: hadoop-hdfs-project/hadoop-hdfs-native-client
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4003/6/console
versions	git=2.9.5 maven=3.6.3
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2022-02-24T13:40:49Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	38m 40s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 4 new or modified test files.
			_ trunk Compile Tests _
+1 💚	mvninstall	36m 29s		trunk passed
+1 💚	compile	4m 30s		trunk passed
+1 💚	mvnsite	0m 32s		trunk passed
+1 💚	shadedclient	68m 30s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	0m 24s		the patch passed
+1 💚	compile	4m 22s		the patch passed
+1 💚	cc	4m 22s		the patch passed
+1 💚	golang	4m 22s		the patch passed
+1 💚	javac	4m 22s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	mvnsite	0m 18s		the patch passed
+1 💚	shadedclient	24m 44s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	1m 45s	/patch-unit-hadoop-hdfs-project_hadoop-hdfs-native-client.txt	hadoop-hdfs-native-client in the patch failed.
+1 💚	asflicense	0m 41s		The patch does not generate ASF License warnings.
		141m 46s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4003/8/artifact/out/Dockerfile
GITHUB PR	#4003
Optional Tests	dupname asflicense compile cc mvnsite javac unit codespell golang
uname	Linux 457a702d7080 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `c29a231`
Default Java	Red Hat, Inc.-1.8.0_322-b06
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4003/8/testReport/
Max. process+thread count	548 (vs. ulimit of 5500)
modules	C: hadoop-hdfs-project/hadoop-hdfs-native-client U: hadoop-hdfs-project/hadoop-hdfs-native-client
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4003/8/console
versions	git=2.9.5 maven=3.6.3
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2022-02-24T14:10:05Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	79m 5s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 1s		codespell was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 4 new or modified test files.
			_ trunk Compile Tests _
+1 💚	mvninstall	43m 59s		trunk passed
+1 💚	compile	4m 33s		trunk passed
+1 💚	mvnsite	0m 32s		trunk passed
-1 ❌	shadedclient	76m 4s		branch has errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	0m 17s		the patch passed
+1 💚	compile	4m 34s		the patch passed
+1 💚	cc	4m 34s		the patch passed
+1 💚	golang	4m 34s		the patch passed
+1 💚	javac	4m 34s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	mvnsite	0m 21s		the patch passed
-1 ❌	shadedclient	22m 3s		patch has errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	20m 7s	/patch-unit-hadoop-hdfs-project_hadoop-hdfs-native-client.txt	hadoop-hdfs-native-client in the patch failed.
+1 💚	asflicense	0m 35s		The patch does not generate ASF License warnings.
		205m 35s

Reason	Tests
Failed CTEST tests	test_test_libhdfs_ops_hdfs_static
	test_test_libhdfs_threaded_hdfs_static
	test_test_libhdfs_zerocopy_hdfs_static
	test_test_native_mini_dfs
	test_libhdfs_threaded_hdfspp_test_shim_static
	test_hdfspp_mini_dfs_smoke_hdfspp_test_shim_static
	libhdfs_mini_stress_valgrind_hdfspp_test_static
	memcheck_libhdfs_mini_stress_valgrind_hdfspp_test_static
	test_libhdfs_mini_stress_hdfspp_test_shim_static
	test_hdfs_ext_hdfspp_test_shim_static

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4003/7/artifact/out/Dockerfile
GITHUB PR	#4003
Optional Tests	dupname asflicense compile cc mvnsite javac unit codespell golang
uname	Linux 28ef380f6fa1 4.15.0-163-generic #171-Ubuntu SMP Fri Nov 5 11:55:11 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `c29a231`
Default Java	Red Hat, Inc.-1.8.0_322-b06
CTEST	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4003/7/artifact/out/patch-hadoop-hdfs-project_hadoop-hdfs-native-client-ctest.txt
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4003/7/testReport/
Max. process+thread count	596 (vs. ulimit of 5500)
modules	C: hadoop-hdfs-project/hadoop-hdfs-native-client U: hadoop-hdfs-project/hadoop-hdfs-native-client
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4003/7/console
versions	git=2.9.5 maven=3.6.3
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2022-03-03T11:05:52Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	39m 16s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 4 new or modified test files.
			_ trunk Compile Tests _
-1 ❌	mvninstall	4m 59s	/branch-mvninstall-root.txt	root in trunk failed.
+1 💚	compile	3m 36s		trunk passed
+1 💚	mvnsite	0m 21s		trunk passed
+1 💚	shadedclient	30m 1s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	0m 16s		the patch passed
+1 💚	compile	3m 13s		the patch passed
+1 💚	cc	3m 13s		the patch passed
+1 💚	golang	3m 13s		the patch passed
+1 💚	javac	3m 13s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	mvnsite	0m 18s		the patch passed
+1 💚	shadedclient	18m 35s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	85m 56s		hadoop-hdfs-native-client in the patch passed.
+1 💚	asflicense	0m 38s		The patch does not generate ASF License warnings.
		180m 36s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4003/9/artifact/out/Dockerfile
GITHUB PR	#4003
Optional Tests	dupname asflicense compile cc mvnsite javac unit codespell golang
uname	Linux 9f07cb6071e9 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / a5437123151c595b6f4095b04ea9b115881ceb2f
Default Java	Red Hat, Inc.-1.8.0_322-b06
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4003/9/testReport/
Max. process+thread count	722 (vs. ulimit of 5500)
modules	C: hadoop-hdfs-project/hadoop-hdfs-native-client U: hadoop-hdfs-project/hadoop-hdfs-native-client
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4003/9/console
versions	git=2.9.5 maven=3.6.3
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2022-03-03T16:43:04Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	51m 38s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 1s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 4 new or modified test files.
			_ trunk Compile Tests _
+1 💚	mvninstall	39m 6s		trunk passed
+1 💚	compile	4m 19s		trunk passed
+1 💚	mvnsite	0m 31s		trunk passed
+1 💚	shadedclient	68m 48s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	0m 16s		the patch passed
+1 💚	compile	3m 49s		the patch passed
+1 💚	cc	3m 49s		the patch passed
+1 💚	golang	3m 49s		the patch passed
+1 💚	javac	3m 49s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	mvnsite	0m 18s		the patch passed
+1 💚	shadedclient	25m 20s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	117m 7s		hadoop-hdfs-native-client in the patch passed.
+1 💚	asflicense	0m 31s		The patch does not generate ASF License warnings.
		270m 21s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4003/10/artifact/out/Dockerfile
GITHUB PR	#4003
Optional Tests	dupname asflicense compile cc mvnsite javac unit codespell golang
uname	Linux 480d780e8b9d 4.15.0-153-generic #160-Ubuntu SMP Thu Jul 29 06:54:29 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / a5437123151c595b6f4095b04ea9b115881ceb2f
Default Java	Red Hat, Inc.-1.8.0_322-b06
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4003/10/testReport/
Max. process+thread count	523 (vs. ulimit of 5500)
modules	C: hadoop-hdfs-project/hadoop-hdfs-native-client U: hadoop-hdfs-project/hadoop-hdfs-native-client
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4003/10/console
versions	git=2.9.5 maven=3.6.3
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2022-03-03T16:45:11Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 0s		Docker mode activated.
-1 ❌	docker	0m 30s		Docker failed to build yetus/hadoop:c42441a1ec4.

Subsystem	Report/Notes
GITHUB PR	#4003
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4003/10/console
versions	git=2.17.1
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2022-03-05T05:40:36Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	43m 39s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 4 new or modified test files.
			_ trunk Compile Tests _
+1 💚	mvninstall	35m 38s		trunk passed
+1 💚	compile	3m 30s		trunk passed
+1 💚	mvnsite	0m 23s		trunk passed
+1 💚	shadedclient	61m 29s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	0m 14s		the patch passed
+1 💚	compile	3m 21s		the patch passed
+1 💚	cc	3m 21s		the patch passed
+1 💚	golang	3m 21s		the patch passed
+1 💚	javac	3m 21s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	mvnsite	0m 15s		the patch passed
+1 💚	shadedclient	22m 7s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	99m 14s		hadoop-hdfs-native-client in the patch passed.
+1 💚	asflicense	0m 31s		The patch does not generate ASF License warnings.
		233m 15s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4003/11/artifact/out/Dockerfile
GITHUB PR	#4003
Optional Tests	dupname asflicense compile cc mvnsite javac unit codespell golang
uname	Linux 87488cbcc0d8 4.15.0-153-generic #160-Ubuntu SMP Thu Jul 29 06:54:29 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `40e9eaa`
Default Java	Red Hat, Inc.-1.8.0_322-b06
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4003/11/testReport/
Max. process+thread count	573 (vs. ulimit of 5500)
modules	C: hadoop-hdfs-project/hadoop-hdfs-native-client U: hadoop-hdfs-project/hadoop-hdfs-native-client
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4003/11/console
versions	git=2.9.5 maven=3.6.3
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2022-03-05T09:25:21Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	27m 19s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 4 new or modified test files.
			_ trunk Compile Tests _
+1 💚	mvninstall	25m 14s		trunk passed
+1 💚	compile	3m 41s		trunk passed
+1 💚	mvnsite	0m 34s		trunk passed
+1 💚	shadedclient	52m 1s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	0m 20s		the patch passed
+1 💚	compile	3m 23s		the patch passed
+1 💚	cc	3m 23s		the patch passed
+1 💚	golang	3m 23s		the patch passed
+1 💚	javac	3m 23s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	mvnsite	0m 22s		the patch passed
+1 💚	shadedclient	24m 13s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	114m 4s		hadoop-hdfs-native-client in the patch passed.
+1 💚	asflicense	0m 39s		The patch does not generate ASF License warnings.
		224m 34s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4003/11/artifact/out/Dockerfile
GITHUB PR	#4003
Optional Tests	dupname asflicense compile cc mvnsite javac unit codespell golang
uname	Linux 1140c06f4704 4.15.0-153-generic #160-Ubuntu SMP Thu Jul 29 06:54:29 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `40e9eaa`
Default Java	Red Hat, Inc.-1.8.0_312-b07
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4003/11/testReport/
Max. process+thread count	588 (vs. ulimit of 5500)
modules	C: hadoop-hdfs-project/hadoop-hdfs-native-client U: hadoop-hdfs-project/hadoop-hdfs-native-client
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4003/11/console
versions	git=2.27.0 maven=3.6.3
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2022-03-05T13:14:32Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	16m 18s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 1s		codespell was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 4 new or modified test files.
			_ trunk Compile Tests _
+1 💚	mvninstall	32m 43s		trunk passed
+1 💚	compile	3m 17s		trunk passed
+1 💚	mvnsite	0m 21s		trunk passed
+1 💚	shadedclient	67m 4s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	0m 15s		the patch passed
+1 💚	compile	3m 9s		the patch passed
+1 💚	cc	3m 9s		the patch passed
+1 💚	golang	3m 9s		the patch passed
+1 💚	javac	3m 9s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	mvnsite	0m 15s		the patch passed
+1 💚	shadedclient	30m 43s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	108m 11s		hadoop-hdfs-native-client in the patch passed.
+1 💚	asflicense	0m 30s		The patch does not generate ASF License warnings.
		228m 52s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4003/11/artifact/out/Dockerfile
GITHUB PR	#4003
Optional Tests	dupname asflicense compile cc mvnsite javac unit codespell golang
uname	Linux 7782e331541c 4.15.0-153-generic #160-Ubuntu SMP Thu Jul 29 06:54:29 UTC 2021 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `40e9eaa`
Default Java	Debian-11.0.14+9-post-Debian-1deb10u1
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4003/11/testReport/
Max. process+thread count	532 (vs. ulimit of 5500)
modules	C: hadoop-hdfs-project/hadoop-hdfs-native-client U: hadoop-hdfs-project/hadoop-hdfs-native-client
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4003/11/console
versions	git=2.20.1 maven=3.6.0
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2022-03-05T16:49:40Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	17m 48s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 1s		codespell was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 4 new or modified test files.
			_ trunk Compile Tests _
+1 💚	mvninstall	24m 21s		trunk passed
+1 💚	compile	3m 45s		trunk passed with JDK Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04
+1 💚	compile	3m 49s		trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚	mvnsite	0m 24s		trunk passed
+1 💚	shadedclient	55m 13s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	0m 14s		the patch passed
+1 💚	compile	3m 45s		the patch passed with JDK Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04
+1 💚	cc	3m 45s		the patch passed
+1 💚	golang	3m 45s		the patch passed
+1 💚	javac	3m 45s		the patch passed
+1 💚	compile	3m 43s		the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚	cc	3m 43s		the patch passed
+1 💚	golang	3m 43s		the patch passed
+1 💚	javac	3m 43s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	mvnsite	0m 16s		the patch passed
+1 💚	shadedclient	22m 36s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	108m 40s		hadoop-hdfs-native-client in the patch passed.
+1 💚	asflicense	0m 30s		The patch does not generate ASF License warnings.
		214m 57s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4003/11/artifact/out/Dockerfile
GITHUB PR	#4003
Optional Tests	dupname asflicense compile cc mvnsite javac unit codespell golang
uname	Linux e6b6969132ae 4.15.0-153-generic #160-Ubuntu SMP Thu Jul 29 06:54:29 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `40e9eaa`
Default Java	Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4003/11/testReport/
Max. process+thread count	577 (vs. ulimit of 5500)
modules	C: hadoop-hdfs-project/hadoop-hdfs-native-client U: hadoop-hdfs-project/hadoop-hdfs-native-client
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4003/11/console
versions	git=2.25.1 maven=3.6.3
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

GauthamBanasandra requested a review from goiri February 20, 2022 07:08

GauthamBanasandra self-assigned this Feb 20, 2022

GauthamBanasandra added the HDFS label Feb 20, 2022

This comment was marked as outdated.

Sign in to view

goiri approved these changes Feb 21, 2022

View reviewed changes

GauthamBanasandra force-pushed the get-x-platform branch from f53d0bf to c29a231 Compare February 23, 2022 10:52

GauthamBanasandra force-pushed the get-x-platform branch from c29a231 to a543712 Compare March 3, 2022 08:04

GauthamBanasandra added 5 commits March 5, 2022 07:15

Make HDFS get tool cross platform

e96de9e

Add unit tests

b265528

Add documentation

8dba904

Remove redundant file

81d5bd1

Add documentation

40e9eaa

GauthamBanasandra force-pushed the get-x-platform branch from a543712 to 40e9eaa Compare March 5, 2022 01:46

GauthamBanasandra marked this pull request as ready for review March 5, 2022 17:21

GauthamBanasandra merged commit 8f07081 into apache:trunk Mar 5, 2022

GauthamBanasandra deleted the get-x-platform branch March 5, 2022 17:35

HarshitGupta11 pushed a commit to HarshitGupta11/hadoop that referenced this pull request Nov 28, 2022

HDFS-16462. Make HDFS get tool cross platform (apache#4003)

9709a3e

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

HDFS-16462. Make HDFS get tool cross platform #4003

HDFS-16462. Make HDFS get tool cross platform #4003

GauthamBanasandra commented Feb 20, 2022

This comment was marked as outdated.

hadoop-yetus commented Feb 23, 2022

hadoop-yetus commented Feb 24, 2022

hadoop-yetus commented Feb 24, 2022

hadoop-yetus commented Mar 3, 2022

hadoop-yetus commented Mar 3, 2022

hadoop-yetus commented Mar 3, 2022

hadoop-yetus commented Mar 5, 2022

hadoop-yetus commented Mar 5, 2022

hadoop-yetus commented Mar 5, 2022

hadoop-yetus commented Mar 5, 2022

HDFS-16462. Make HDFS get tool cross platform #4003

HDFS-16462. Make HDFS get tool cross platform #4003

Conversation

GauthamBanasandra commented Feb 20, 2022

Description of PR

How was this patch tested?

hdfs_copyToLocal

hdfs_get

For code changes:

This comment was marked as outdated.

hadoop-yetus commented Feb 23, 2022

hadoop-yetus commented Feb 24, 2022

hadoop-yetus commented Feb 24, 2022

hadoop-yetus commented Mar 3, 2022

hadoop-yetus commented Mar 3, 2022

hadoop-yetus commented Mar 3, 2022

hadoop-yetus commented Mar 5, 2022

hadoop-yetus commented Mar 5, 2022

hadoop-yetus commented Mar 5, 2022

hadoop-yetus commented Mar 5, 2022