HDFS-16446. Consider ioutils of disk when choosing volume #3960

tomscut · 2022-02-05T02:40:36Z

Consider ioutils of disk when choosing volume to avoid busy disks.

Document: https://docs.google.com/document/d/1Ko1J7shz8hVLnNACT6PKVQ_leIHf_YaIFA2s3yJMZHQ/edit?usp=sharing

Architecture is as follows:

hadoop-yetus · 2022-02-05T12:37:36Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	44m 21s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 1s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 4 new or modified test files.
			_ trunk Compile Tests _
+0 🆗	mvndep	12m 40s		Maven dependency ordering for branch
+1 💚	mvninstall	25m 12s		trunk passed
+1 💚	compile	20m 57s		trunk passed
+1 💚	checkstyle	3m 57s		trunk passed
+1 💚	mvnsite	3m 18s		trunk passed
+1 💚	javadoc	3m 20s		trunk passed
+1 💚	spotbugs	6m 8s		trunk passed
+1 💚	shadedclient	25m 56s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 28s		Maven dependency ordering for patch
+1 💚	mvninstall	2m 27s		the patch passed
+1 💚	compile	20m 6s		the patch passed
-1 ❌	cc	20m 6s	/results-compile-cc-root.txt	root generated 18 new + 189 unchanged - 16 fixed = 207 total (was 205)
+1 💚	golang	20m 6s		the patch passed
+1 💚	javac	20m 6s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	3m 50s	/results-checkstyle-root.txt	root: The patch generated 10 new + 519 unchanged - 0 fixed = 529 total (was 519)
+1 💚	mvnsite	3m 17s		the patch passed
+1 💚	xml	0m 1s		The patch has no ill-formed XML file.
+1 💚	javadoc	3m 20s		the patch passed
-1 ❌	spotbugs	3m 50s	/new-spotbugs-hadoop-hdfs-project_hadoop-hdfs.html	hadoop-hdfs-project/hadoop-hdfs generated 7 new + 0 unchanged - 0 fixed = 7 total (was 0)
-1 ❌	shadedclient	26m 38s		patch has errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	17m 31s	/patch-unit-hadoop-common-project_hadoop-common.txt	hadoop-common in the patch passed.
-1 ❌	unit	367m 35s	/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs in the patch passed.
-1 ❌	asflicense	1m 19s	/results-asflicense.txt	The patch generated 1 ASF License warnings.
		595m 52s

Reason	Tests
SpotBugs	module:hadoop-hdfs-project/hadoop-hdfs
	org.apache.hadoop.hdfs.server.datanode.DiskIOUtilManager.getDiskIoUtils() may fail to close stream At DiskIOUtilManager.java:stream At DiskIOUtilManager.java:[line 203]
	Possible null pointer dereference in new org.apache.hadoop.hdfs.server.datanode.DiskIOUtilManager$DiskLocation(DiskIOUtilManager, StorageLocation) due to return value of called method Dereferenced at DiskIOUtilManager.java:new org.apache.hadoop.hdfs.server.datanode.DiskIOUtilManager$DiskLocation(DiskIOUtilManager, StorageLocation) due to return value of called method Dereferenced at DiskIOUtilManager.java:[line 56]
	Possible null pointer dereference in new org.apache.hadoop.hdfs.server.datanode.DiskIOUtilManager$DiskLocation(DiskIOUtilManager, StorageLocation) due to return value of called method Dereferenced at DiskIOUtilManager.java:new org.apache.hadoop.hdfs.server.datanode.DiskIOUtilManager$DiskLocation(DiskIOUtilManager, StorageLocation) due to return value of called method Dereferenced at DiskIOUtilManager.java:[line 61]
	Should org.apache.hadoop.hdfs.server.datanode.DiskIOUtilManager$DiskLocation be a static inner class? At DiskIOUtilManager.java:inner class? At DiskIOUtilManager.java:[lines 43-71]
	Should org.apache.hadoop.hdfs.server.datanode.DiskIOUtilManager$IOStat be a static inner class? At DiskIOUtilManager.java:inner class? At DiskIOUtilManager.java:[lines 85-110]
	Unread field:DiskIOUtilManager.java:[line 86]
	org.apache.hadoop.hdfs.server.datanode.fsdataset.AvailableSpaceVolumeChoosingPolicy$IOUtilVolumePair defines compareTo(AvailableSpaceVolumeChoosingPolicy$IOUtilVolumePair) and uses Object.equals() At AvailableSpaceVolumeChoosingPolicy.java:Object.equals() At AvailableSpaceVolumeChoosingPolicy.java:[lines 442-450]
Failed junit tests	hadoop.util.TestNativeLibraryChecker
	hadoop.service.launcher.TestServiceInterruptHandling
	hadoop.metrics2.sink.TestFileSink
	hadoop.crypto.TestCryptoCodec
	hadoop.crypto.TestCryptoStreamsWithOpensslSm4CtrCryptoCodec
	hadoop.tools.TestHdfsConfigFields
	hadoop.hdfs.server.datanode.fsdataset.TestAvailableSpaceVolumeChoosingPolicy

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3960/1/artifact/out/Dockerfile
GITHUB PR	#3960
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell cc golang xml
uname	Linux ffdbe3525f75 4.15.0-162-generic #170-Ubuntu SMP Mon Oct 18 11:38:05 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / 24e98c8ab8d1596f102c7de7d28553d866bae9ca
Default Java	Red Hat, Inc.-1.8.0_322-b06
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3960/1/testReport/
Max. process+thread count	2137 (vs. ulimit of 5500)
modules	C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3960/1/console
versions	git=2.9.5 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

jojochuang

Some quick comments. I'm still trying to understand how the disk io stats get considered. Some documentation would be helpful.

Instead of being part of AvailableSpaceVolumeChoosingPolicy, would it make sense to become a new volume choosing policy?

...op-common-project/hadoop-common/src/main/native/src/org/apache/hadoop/io/nativeio/NativeIO.c

hadoop-hdfs-project/hadoop-hdfs/pom.xml

...ject/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DiskIOUtilManager.java

hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java

jojochuang · 2022-02-09T06:26:46Z

@ayushtkn fyi

tomscut · 2022-02-09T08:12:19Z

AvailableSpaceVolumeChoosingPolicy

tomscut · 2022-02-09T08:24:21Z

Some quick comments. I'm still trying to understand how the disk io stats get considered. Some documentation would be helpful.

Instead of being part of AvailableSpaceVolumeChoosingPolicy, would it make sense to become a new volume choosing policy?

Thanks @jojochuang for your comments, I will add documentation and update the code.

When doing the POC testing, we define a new policy, named IOUtilVolumeChoosingPolicy, only considering the ioutils without considering the disk space usage, it will cause uneven disk utilization.

So we change AvailableSpaceVolumeChoosingPolicy, under the same disk usage, preferred ioutils smaller disks, filtering the busy disks.

What you said about offering a new policy is a good idea.

hadoop-yetus · 2022-02-09T18:12:17Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 59s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 4 new or modified test files.
			_ trunk Compile Tests _
+0 🆗	mvndep	25m 6s		Maven dependency ordering for branch
+1 💚	mvninstall	29m 19s		trunk passed
+1 💚	compile	25m 35s		trunk passed
+1 💚	checkstyle	4m 24s		trunk passed
+1 💚	mvnsite	3m 52s		trunk passed
+1 💚	javadoc	3m 48s		trunk passed
+1 💚	spotbugs	7m 7s		trunk passed
+1 💚	shadedclient	27m 26s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 38s		Maven dependency ordering for patch
+1 💚	mvninstall	2m 44s		the patch passed
+1 💚	compile	23m 52s		the patch passed
-1 ❌	cc	23m 52s	/results-compile-cc-root.txt	root generated 22 new + 185 unchanged - 20 fixed = 207 total (was 205)
+1 💚	golang	23m 52s		the patch passed
+1 💚	javac	23m 52s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	4m 6s	/results-checkstyle-root.txt	root: The patch generated 10 new + 519 unchanged - 0 fixed = 529 total (was 519)
+1 💚	mvnsite	3m 46s		the patch passed
+1 💚	xml	0m 1s		The patch has no ill-formed XML file.
+1 💚	javadoc	3m 43s		the patch passed
-1 ❌	spotbugs	4m 24s	/new-spotbugs-hadoop-hdfs-project_hadoop-hdfs.html	hadoop-hdfs-project/hadoop-hdfs generated 7 new + 0 unchanged - 0 fixed = 7 total (was 0)
-1 ❌	shadedclient	27m 37s		patch has errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	19m 19s	/patch-unit-hadoop-common-project_hadoop-common.txt	hadoop-common in the patch passed.
-1 ❌	unit	375m 36s	/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs in the patch passed.
-1 ❌	asflicense	1m 12s	/results-asflicense.txt	The patch generated 1 ASF License warnings.
		594m 10s

Reason	Tests
SpotBugs	module:hadoop-hdfs-project/hadoop-hdfs
	org.apache.hadoop.hdfs.server.datanode.DiskIOUtilManager.getDiskIoUtils() may fail to close stream At DiskIOUtilManager.java:stream At DiskIOUtilManager.java:[line 203]
	Possible null pointer dereference in new org.apache.hadoop.hdfs.server.datanode.DiskIOUtilManager$DiskLocation(DiskIOUtilManager, StorageLocation) due to return value of called method Dereferenced at DiskIOUtilManager.java:new org.apache.hadoop.hdfs.server.datanode.DiskIOUtilManager$DiskLocation(DiskIOUtilManager, StorageLocation) due to return value of called method Dereferenced at DiskIOUtilManager.java:[line 56]
	Possible null pointer dereference in new org.apache.hadoop.hdfs.server.datanode.DiskIOUtilManager$DiskLocation(DiskIOUtilManager, StorageLocation) due to return value of called method Dereferenced at DiskIOUtilManager.java:new org.apache.hadoop.hdfs.server.datanode.DiskIOUtilManager$DiskLocation(DiskIOUtilManager, StorageLocation) due to return value of called method Dereferenced at DiskIOUtilManager.java:[line 61]
	Should org.apache.hadoop.hdfs.server.datanode.DiskIOUtilManager$DiskLocation be a static inner class? At DiskIOUtilManager.java:inner class? At DiskIOUtilManager.java:[lines 43-71]
	Should org.apache.hadoop.hdfs.server.datanode.DiskIOUtilManager$IOStat be a static inner class? At DiskIOUtilManager.java:inner class? At DiskIOUtilManager.java:[lines 85-110]
	Unread field:DiskIOUtilManager.java:[line 86]
	org.apache.hadoop.hdfs.server.datanode.fsdataset.AvailableSpaceVolumeChoosingPolicy$IOUtilVolumePair defines compareTo(AvailableSpaceVolumeChoosingPolicy$IOUtilVolumePair) and uses Object.equals() At AvailableSpaceVolumeChoosingPolicy.java:Object.equals() At AvailableSpaceVolumeChoosingPolicy.java:[lines 442-450]
Failed junit tests	hadoop.util.TestNativeLibraryChecker
	hadoop.service.launcher.TestServiceInterruptHandling
	hadoop.crypto.TestCryptoCodec
	hadoop.crypto.TestCryptoStreamsWithOpensslSm4CtrCryptoCodec
	hadoop.tools.TestHdfsConfigFields
	hadoop.hdfs.server.datanode.fsdataset.TestAvailableSpaceVolumeChoosingPolicy

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3960/2/artifact/out/Dockerfile
GITHUB PR	#3960
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell cc golang xml
uname	Linux 011c688e0d67 4.15.0-162-generic #170-Ubuntu SMP Mon Oct 18 11:38:05 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / 24e98c8ab8d1596f102c7de7d28553d866bae9ca
Default Java	Red Hat, Inc.-1.8.0_322-b06
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3960/2/testReport/
Max. process+thread count	3137 (vs. ulimit of 5500)
modules	C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3960/2/console
versions	git=2.9.5 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2022-03-24T04:08:21Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 0s		Docker mode activated.
-1 ❌	docker	19m 36s		Docker failed to build yetus/hadoop:13467f45240.

Subsystem	Report/Notes
GITHUB PR	#3960
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3960/3/console
versions	git=2.17.1
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2022-03-24T04:14:34Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 0s		Docker mode activated.
-1 ❌	docker	14m 54s		Docker failed to build yetus/hadoop:13467f45240.

Subsystem	Report/Notes
GITHUB PR	#3960
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3960/4/console
versions	git=2.17.1
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2022-03-24T15:33:41Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	26m 18s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 4 new or modified test files.
			_ trunk Compile Tests _
+0 🆗	mvndep	12m 15s		Maven dependency ordering for branch
+1 💚	mvninstall	26m 40s		trunk passed
+1 💚	compile	21m 39s		trunk passed
+1 💚	checkstyle	3m 58s		trunk passed
+1 💚	mvnsite	3m 55s		trunk passed
+1 💚	javadoc	3m 56s		trunk passed
+0 🆗	spotbugs	0m 32s		branch/hadoop-project no spotbugs output file (spotbugsXml.xml)
+1 💚	shadedclient	25m 55s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 23s		Maven dependency ordering for patch
+1 💚	mvninstall	2m 34s		the patch passed
+1 💚	compile	20m 28s		the patch passed
-1 ❌	cc	20m 28s	/results-compile-cc-root.txt	root generated 30 new + 177 unchanged - 28 fixed = 207 total (was 205)
+1 💚	golang	20m 28s		the patch passed
+1 💚	javac	20m 28s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	3m 53s		the patch passed
+1 💚	mvnsite	3m 47s		the patch passed
+1 💚	xml	0m 3s		The patch has no ill-formed XML file.
+1 💚	javadoc	3m 49s		the patch passed
+0 🆗	spotbugs	0m 29s		hadoop-project has no data from spotbugs
-1 ❌	spotbugs	3m 50s	/new-spotbugs-hadoop-hdfs-project_hadoop-hdfs.html	hadoop-hdfs-project/hadoop-hdfs generated 7 new + 0 unchanged - 0 fixed = 7 total (was 0)
+1 💚	shadedclient	26m 19s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	0m 28s		hadoop-project in the patch passed.
-1 ❌	unit	17m 51s	/patch-unit-hadoop-common-project_hadoop-common.txt	hadoop-common in the patch passed.
-1 ❌	unit	353m 8s	/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs in the patch passed.
-1 ❌	asflicense	1m 1s	/results-asflicense.txt	The patch generated 1 ASF License warnings.
		569m 16s

Reason	Tests
SpotBugs	module:hadoop-hdfs-project/hadoop-hdfs
	Inconsistent synchronization of org.apache.hadoop.hdfs.server.datanode.DataNode.diskIOUtilManager; locked 50% of time Unsynchronized access at DataNode.java:50% of time Unsynchronized access at DataNode.java:[line 2473]
	org.apache.hadoop.hdfs.server.datanode.DiskIOUtilManager.getDiskIoUtils() may fail to close stream At DiskIOUtilManager.java:stream At DiskIOUtilManager.java:[line 209]
	Possible null pointer dereference in new org.apache.hadoop.hdfs.server.datanode.DiskIOUtilManager$DiskLocation(StorageLocation) due to return value of called method Dereferenced at DiskIOUtilManager.java:new org.apache.hadoop.hdfs.server.datanode.DiskIOUtilManager$DiskLocation(StorageLocation) due to return value of called method Dereferenced at DiskIOUtilManager.java:[line 54]
	Possible null pointer dereference in new org.apache.hadoop.hdfs.server.datanode.DiskIOUtilManager$DiskLocation(StorageLocation) due to return value of called method Dereferenced at DiskIOUtilManager.java:new org.apache.hadoop.hdfs.server.datanode.DiskIOUtilManager$DiskLocation(StorageLocation) due to return value of called method Dereferenced at DiskIOUtilManager.java:[line 59]
	Should org.apache.hadoop.hdfs.server.datanode.DiskIOUtilManager$IOStat be a static inner class? At DiskIOUtilManager.java:inner class? At DiskIOUtilManager.java:[lines 87-112]
	Unread field:DiskIOUtilManager.java:[line 88]
	org.apache.hadoop.hdfs.server.datanode.fsdataset.AvailableSpaceVolumeChoosingPolicy$IOUtilVolumePair defines compareTo(AvailableSpaceVolumeChoosingPolicy$IOUtilVolumePair) and uses Object.equals() At AvailableSpaceVolumeChoosingPolicy.java:Object.equals() At AvailableSpaceVolumeChoosingPolicy.java:[lines 443-451]
Failed junit tests	hadoop.service.launcher.TestServiceInterruptHandling
	hadoop.crypto.TestCryptoCodec
	hadoop.crypto.TestCryptoStreamsWithOpensslSm4CtrCryptoCodec
	hadoop.tools.TestHdfsConfigFields
	hadoop.hdfs.server.datanode.fsdataset.TestAvailableSpaceVolumeChoosingPolicy

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3960/5/artifact/out/Dockerfile
GITHUB PR	#3960
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell cc golang xml
uname	Linux b0c15dd70f74 4.15.0-162-generic #170-Ubuntu SMP Mon Oct 18 11:38:05 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / c59e97d3bc9b63a578ebfed3021eb18170dbba91
Default Java	Red Hat, Inc.-1.8.0_322-b06
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3960/5/testReport/
Max. process+thread count	2425 (vs. ulimit of 5500)
modules	C: hadoop-project hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3960/5/console
versions	git=2.9.5 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2022-03-25T14:14:55Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	44m 58s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 4 new or modified test files.
			_ trunk Compile Tests _
+0 🆗	mvndep	12m 26s		Maven dependency ordering for branch
+1 💚	mvninstall	27m 12s		trunk passed
+1 💚	compile	21m 11s		trunk passed
+1 💚	checkstyle	3m 56s		trunk passed
+1 💚	mvnsite	3m 53s		trunk passed
+1 💚	javadoc	3m 59s		trunk passed
+0 🆗	spotbugs	0m 32s		branch/hadoop-project no spotbugs output file (spotbugsXml.xml)
+1 💚	shadedclient	26m 24s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 23s		Maven dependency ordering for patch
+1 💚	mvninstall	2m 35s		the patch passed
+1 💚	compile	20m 37s		the patch passed
-1 ❌	cc	20m 37s	/results-compile-cc-root.txt	root generated 34 new + 173 unchanged - 32 fixed = 207 total (was 205)
+1 💚	golang	20m 37s		the patch passed
-1 ❌	javac	20m 37s	/results-compile-javac-root.txt	root generated 1 new + 1677 unchanged - 1 fixed = 1678 total (was 1678)
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	3m 51s	/results-checkstyle-root.txt	root: The patch generated 3 new + 515 unchanged - 0 fixed = 518 total (was 515)
+1 💚	mvnsite	3m 48s		the patch passed
+1 💚	xml	0m 3s		The patch has no ill-formed XML file.
+1 💚	javadoc	3m 56s		the patch passed
+0 🆗	spotbugs	0m 29s		hadoop-project has no data from spotbugs
-1 ❌	spotbugs	3m 53s	/new-spotbugs-hadoop-hdfs-project_hadoop-hdfs.html	hadoop-hdfs-project/hadoop-hdfs generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0)
+1 💚	shadedclient	26m 5s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	0m 29s		hadoop-project in the patch passed.
-1 ❌	unit	18m 2s	/patch-unit-hadoop-common-project_hadoop-common.txt	hadoop-common in the patch passed.
-1 ❌	unit	201m 2s	/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs in the patch passed.
+0 🆗	asflicense	1m 10s		ASF License check generated no output?
		436m 42s

Reason	Tests
SpotBugs	module:hadoop-hdfs-project/hadoop-hdfs
	Possible null pointer dereference in new org.apache.hadoop.hdfs.server.datanode.DiskIOUtilManager$DiskLocation(StorageLocation) due to return value of called method Dereferenced at DiskIOUtilManager.java:new org.apache.hadoop.hdfs.server.datanode.DiskIOUtilManager$DiskLocation(StorageLocation) due to return value of called method Dereferenced at DiskIOUtilManager.java:[line 49]
	org.apache.hadoop.hdfs.server.datanode.fsdataset.AvailableSpaceVolumeChoosingPolicy$IOUtilVolumePair defines equals and uses Object.hashCode() At AvailableSpaceVolumeChoosingPolicy.java:Object.hashCode() At AvailableSpaceVolumeChoosingPolicy.java:[lines 459-463]
Failed junit tests	hadoop.service.launcher.TestServiceInterruptHandling
	hadoop.crypto.TestCryptoStreamsWithOpensslSm4CtrCryptoCodec
	hadoop.crypto.TestCryptoCodec
	hadoop.hdfs.server.datanode.TestDataNodeTcpNoDelay
	hadoop.hdfs.server.datanode.TestDataNodeFaultInjector
	hadoop.hdfs.server.namenode.ha.TestRetryCacheWithHA
	hadoop.hdfs.server.namenode.ha.TestHASafeMode
	hadoop.hdfs.server.datanode.TestDataNodeReconfiguration
	hadoop.tools.TestHdfsConfigFields

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3960/6/artifact/out/Dockerfile
GITHUB PR	#3960
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell cc golang xml
uname	Linux d603716e9e49 4.15.0-153-generic #160-Ubuntu SMP Thu Jul 29 06:54:29 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / d1c38ea9554ffc26820b022e30ad6aefc4fa3575
Default Java	Red Hat, Inc.-1.8.0_322-b06
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3960/6/testReport/
Max. process+thread count	3098 (vs. ulimit of 5500)
modules	C: hadoop-project hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3960/6/console
versions	git=2.9.5 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2022-03-27T17:47:19Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 54s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+1 💚	@author	0m 1s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 4 new or modified test files.
			_ trunk Compile Tests _
+0 🆗	mvndep	12m 13s		Maven dependency ordering for branch
+1 💚	mvninstall	27m 6s		trunk passed
+1 💚	compile	21m 20s		trunk passed
+1 💚	checkstyle	3m 56s		trunk passed
+1 💚	mvnsite	3m 50s		trunk passed
+1 💚	javadoc	3m 56s		trunk passed
+0 🆗	spotbugs	0m 32s		branch/hadoop-project no spotbugs output file (spotbugsXml.xml)
+1 💚	shadedclient	26m 18s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 23s		Maven dependency ordering for patch
+1 💚	mvninstall	2m 36s		the patch passed
+1 💚	compile	20m 36s		the patch passed
-1 ❌	cc	20m 36s	/results-compile-cc-root.txt	root generated 6 new + 201 unchanged - 4 fixed = 207 total (was 205)
+1 💚	golang	20m 36s		the patch passed
+1 💚	javac	20m 36s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	3m 56s	/results-checkstyle-root.txt	root: The patch generated 1 new + 516 unchanged - 0 fixed = 517 total (was 516)
+1 💚	mvnsite	3m 48s		the patch passed
+1 💚	xml	0m 2s		The patch has no ill-formed XML file.
+1 💚	javadoc	3m 58s		the patch passed
+0 🆗	spotbugs	0m 29s		hadoop-project has no data from spotbugs
-1 ❌	spotbugs	3m 49s	/new-spotbugs-hadoop-hdfs-project_hadoop-hdfs.html	hadoop-hdfs-project/hadoop-hdfs generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0)
+1 💚	shadedclient	26m 11s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	0m 28s		hadoop-project in the patch passed.
-1 ❌	unit	17m 50s	/patch-unit-hadoop-common-project_hadoop-common.txt	hadoop-common in the patch passed.
-1 ❌	unit	347m 43s	/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs in the patch passed.
-1 ❌	asflicense	1m 0s	/results-asflicense.txt	The patch generated 1 ASF License warnings.
		538m 45s

Reason	Tests
SpotBugs	module:hadoop-hdfs-project/hadoop-hdfs
	Possible null pointer dereference in new org.apache.hadoop.hdfs.server.datanode.DiskIOUtilManager$DiskLocation(StorageLocation) due to return value of called method Dereferenced at DiskIOUtilManager.java:new org.apache.hadoop.hdfs.server.datanode.DiskIOUtilManager$DiskLocation(StorageLocation) due to return value of called method Dereferenced at DiskIOUtilManager.java:[line 58]
	Possible null pointer dereference in new org.apache.hadoop.hdfs.server.datanode.DiskIOUtilManager$DiskLocation(StorageLocation) due to return value of called method Dereferenced at DiskIOUtilManager.java:new org.apache.hadoop.hdfs.server.datanode.DiskIOUtilManager$DiskLocation(StorageLocation) due to return value of called method Dereferenced at DiskIOUtilManager.java:[line 63]
Failed junit tests	hadoop.service.launcher.TestServiceInterruptHandling
	hadoop.crypto.TestCryptoCodec
	hadoop.crypto.TestCryptoStreamsWithOpensslSm4CtrCryptoCodec
	hadoop.tools.TestHdfsConfigFields
	hadoop.hdfs.server.datanode.fsdataset.TestAvailableSpaceVolumeChoosingPolicy

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3960/7/artifact/out/Dockerfile
GITHUB PR	#3960
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell cc golang xml
uname	Linux 4666557d64a5 4.15.0-162-generic #170-Ubuntu SMP Mon Oct 18 11:38:05 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / e6565d20d31e73c3b9475eed790b30e802134de2
Default Java	Red Hat, Inc.-1.8.0_322-b06
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3960/7/testReport/
Max. process+thread count	1899 (vs. ulimit of 5500)
modules	C: hadoop-project hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3960/7/console
versions	git=2.9.5 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2022-03-28T09:09:13Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 55s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 1s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 4 new or modified test files.
			_ trunk Compile Tests _
+0 🆗	mvndep	12m 19s		Maven dependency ordering for branch
+1 💚	mvninstall	26m 44s		trunk passed
+1 💚	compile	21m 19s		trunk passed
+1 💚	checkstyle	4m 0s		trunk passed
+1 💚	mvnsite	3m 51s		trunk passed
+1 💚	javadoc	4m 2s		trunk passed
+0 🆗	spotbugs	0m 32s		branch/hadoop-project no spotbugs output file (spotbugsXml.xml)
+1 💚	shadedclient	25m 58s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 23s		Maven dependency ordering for patch
+1 💚	mvninstall	2m 35s		the patch passed
+1 💚	compile	20m 33s		the patch passed
-1 ❌	cc	20m 33s	/results-compile-cc-root.txt	root generated 20 new + 187 unchanged - 18 fixed = 207 total (was 205)
+1 💚	golang	20m 33s		the patch passed
+1 💚	javac	20m 33s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	3m 54s	/results-checkstyle-root.txt	root: The patch generated 1 new + 516 unchanged - 0 fixed = 517 total (was 516)
+1 💚	mvnsite	3m 50s		the patch passed
+1 💚	xml	0m 3s		The patch has no ill-formed XML file.
+1 💚	javadoc	3m 49s		the patch passed
+0 🆗	spotbugs	0m 30s		hadoop-project has no data from spotbugs
+1 💚	shadedclient	26m 25s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	0m 29s		hadoop-project in the patch passed.
-1 ❌	unit	17m 53s	/patch-unit-hadoop-common-project_hadoop-common.txt	hadoop-common in the patch passed.
-1 ❌	unit	337m 26s	/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs in the patch passed.
+1 💚	asflicense	1m 2s		The patch does not generate ASF License warnings.
		528m 12s

Reason	Tests
Failed junit tests	hadoop.service.launcher.TestServiceInterruptHandling
	hadoop.crypto.TestCryptoCodec
	hadoop.crypto.TestCryptoStreamsWithOpensslSm4CtrCryptoCodec
	hadoop.tools.TestHdfsConfigFields

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3960/8/artifact/out/Dockerfile
GITHUB PR	#3960
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell cc golang xml
uname	Linux c600376df19b 4.15.0-162-generic #170-Ubuntu SMP Mon Oct 18 11:38:05 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / c7882a443a8e7c002a2489d980602f4da36cf310
Default Java	Red Hat, Inc.-1.8.0_322-b06
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3960/8/testReport/
Max. process+thread count	3137 (vs. ulimit of 5500)
modules	C: hadoop-project hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3960/8/console
versions	git=2.9.5 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2022-03-30T21:37:49Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 55s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 4 new or modified test files.
			_ trunk Compile Tests _
+0 🆗	mvndep	12m 17s		Maven dependency ordering for branch
+1 💚	mvninstall	26m 59s		trunk passed
+1 💚	compile	21m 22s		trunk passed
+1 💚	checkstyle	4m 2s		trunk passed
+1 💚	mvnsite	3m 50s		trunk passed
+1 💚	javadoc	3m 58s		trunk passed
+0 🆗	spotbugs	0m 33s		branch/hadoop-project no spotbugs output file (spotbugsXml.xml)
+1 💚	shadedclient	26m 9s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 23s		Maven dependency ordering for patch
+1 💚	mvninstall	2m 33s		the patch passed
+1 💚	compile	20m 32s		the patch passed
-1 ❌	cc	20m 32s	/results-compile-cc-root.txt	root generated 22 new + 185 unchanged - 20 fixed = 207 total (was 205)
+1 💚	golang	20m 32s		the patch passed
+1 💚	javac	20m 32s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	3m 54s	/results-checkstyle-root.txt	root: The patch generated 1 new + 516 unchanged - 0 fixed = 517 total (was 516)
+1 💚	mvnsite	3m 49s		the patch passed
+1 💚	xml	0m 2s		The patch has no ill-formed XML file.
+1 💚	javadoc	3m 55s		the patch passed
+0 🆗	spotbugs	0m 30s		hadoop-project has no data from spotbugs
+1 💚	shadedclient	26m 5s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	0m 28s		hadoop-project in the patch passed.
-1 ❌	unit	17m 51s	/patch-unit-hadoop-common-project_hadoop-common.txt	hadoop-common in the patch passed.
-1 ❌	unit	352m 32s	/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs in the patch passed.
+1 💚	asflicense	1m 0s		The patch does not generate ASF License warnings.
		543m 40s

Reason	Tests
Failed junit tests	hadoop.service.launcher.TestServiceInterruptHandling
	hadoop.crypto.TestCryptoCodec
	hadoop.crypto.TestCryptoStreamsWithOpensslSm4CtrCryptoCodec
	hadoop.hdfs.server.datanode.TestDataXceiverBackwardsCompat

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3960/9/artifact/out/Dockerfile
GITHUB PR	#3960
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell cc golang xml
uname	Linux 0a731bb5fd2a 4.15.0-162-generic #170-Ubuntu SMP Mon Oct 18 11:38:05 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / 512a5ba75cde11a93fa66e3b4d40e086803414cc
Default Java	Red Hat, Inc.-1.8.0_322-b06
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3960/9/testReport/
Max. process+thread count	2394 (vs. ulimit of 5500)
modules	C: hadoop-project hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3960/9/console
versions	git=2.9.5 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

Co-authored-by: liuhongtong <hongtongliu@126.com> Co-authored-by: hfutatzhanghb <1036798979@qq.com>

hadoop-yetus · 2022-03-31T11:12:09Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 55s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 4 new or modified test files.
			_ trunk Compile Tests _
+0 🆗	mvndep	12m 29s		Maven dependency ordering for branch
+1 💚	mvninstall	30m 22s		trunk passed
+1 💚	compile	21m 16s		trunk passed
+1 💚	checkstyle	4m 5s		trunk passed
+1 💚	mvnsite	3m 21s		trunk passed
+1 💚	javadoc	3m 31s		trunk passed
+1 💚	spotbugs	6m 9s		trunk passed
+1 💚	shadedclient	26m 19s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 23s		Maven dependency ordering for patch
+1 💚	mvninstall	2m 24s		the patch passed
+1 💚	compile	20m 37s		the patch passed
-1 ❌	cc	20m 37s	/results-compile-cc-root.txt	root generated 9 new + 198 unchanged - 7 fixed = 207 total (was 205)
+1 💚	golang	20m 37s		the patch passed
+1 💚	javac	20m 37s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	3m 56s	/results-checkstyle-root.txt	root: The patch generated 1 new + 516 unchanged - 0 fixed = 517 total (was 516)
+1 💚	mvnsite	3m 20s		the patch passed
+1 💚	xml	0m 2s		The patch has no ill-formed XML file.
+1 💚	javadoc	3m 24s		the patch passed
+1 💚	spotbugs	6m 31s		the patch passed
+1 💚	shadedclient	26m 17s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	17m 53s	/patch-unit-hadoop-common-project_hadoop-common.txt	hadoop-common in the patch passed.
-1 ❌	unit	360m 26s	/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs in the patch passed.
+1 💚	asflicense	1m 13s		The patch does not generate ASF License warnings.
		552m 14s

Reason	Tests
Failed junit tests	hadoop.service.launcher.TestServiceInterruptHandling
	hadoop.crypto.TestCryptoCodec
	hadoop.crypto.TestCryptoStreamsWithOpensslSm4CtrCryptoCodec
	hadoop.hdfs.server.namenode.TestFileTruncate

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3960/10/artifact/out/Dockerfile
GITHUB PR	#3960
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell cc golang xml
uname	Linux f8b5ed4ec11f 4.15.0-162-generic #170-Ubuntu SMP Mon Oct 18 11:38:05 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `41cfc18`
Default Java	Red Hat, Inc.-1.8.0_322-b06
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3960/10/testReport/
Max. process+thread count	1911 (vs. ulimit of 5500)
modules	C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3960/10/console
versions	git=2.9.5 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

tomscut · 2022-04-06T01:39:59Z

Hi @jojochuang @ayushtkn @tasanuma , I added a document for this. Could you please take a look at this? Thanks a lot.

The failed unit tests seem unrelated to this change.

whbing · 2022-07-14T03:56:42Z

Hi, @tomscut , we also face slow nodes caused by io, but in most cases it is slow to read data because the frequency of reading in the cluster far exceeds that of writing. We have counted many cases, when reading slowly, the iowait is basically high, but only one or a few disks are full of ioutil (12 disks per dn). Therefore, I have the following points to discuss.

Is there any further consideration for slow reading?
Have you considered reporting iowait to nn, like XmitsInProgress, ActiveTransferThreadCount, etc, so that it is taken into account when choosing a DN?

singer-bin · 2024-01-10T07:35:10Z

@whbing Do you have any strategies to deal with the above two points?

uflswe · 2024-08-29T03:45:46Z

@whbing

Hi, @tomscut , we also face slow nodes caused by io, but in most cases it is slow to read data because the frequency of reading in the cluster far exceeds that of writing. We have counted many cases, when reading slowly, the iowait is basically high, but only one or a few disks are full of ioutil (12 disks per dn). Therefore, I have the following points to discuss.

Is there any further consideration for slow reading?

Have you considered reporting iowait to nn, like XmitsInProgress, ActiveTransferThreadCount, etc, so that it is taken into account when choosing a DN?

Hi, @whbing. Your first suggestion is good. But sorry I've some other opinions, VolumeChoosingPolicy is only relevant during the write operation, determining which disk on a DataNode should store the data block. When reading files from HDFS, the focus is on selecting the appropriate DataNode that contains the required data blocks, not on selecting specific disks within a DataNode. Sorry if I'm wrong.

jojochuang reviewed Feb 9, 2022

View reviewed changes

tomscut closed this Feb 9, 2022

tomscut reopened this Feb 9, 2022

tomscut force-pushed the HDFS-16446 branch 2 times, most recently from 4190ea9 to 754f133 Compare March 24, 2022 03:56

tomscut force-pushed the HDFS-16446 branch from 754f133 to c59e97d Compare March 24, 2022 06:03

tomscut force-pushed the HDFS-16446 branch from d1c38ea to e6565d2 Compare March 27, 2022 08:47

tomscut force-pushed the HDFS-16446 branch from e6565d2 to c7882a4 Compare March 28, 2022 00:19

tomscut force-pushed the HDFS-16446 branch from c7882a4 to 512a5ba Compare March 30, 2022 12:32

HDFS-16446. Consider ioutils of disk when choosing volume

41cfc18

Co-authored-by: liuhongtong <hongtongliu@126.com> Co-authored-by: hfutatzhanghb <1036798979@qq.com>

tomscut force-pushed the HDFS-16446 branch from 512a5ba to 41cfc18 Compare March 31, 2022 01:58

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

HDFS-16446. Consider ioutils of disk when choosing volume #3960

HDFS-16446. Consider ioutils of disk when choosing volume #3960

tomscut commented Feb 5, 2022 •

edited

Loading

hadoop-yetus commented Feb 5, 2022

jojochuang left a comment

jojochuang commented Feb 9, 2022

tomscut commented Feb 9, 2022

tomscut commented Feb 9, 2022

hadoop-yetus commented Feb 9, 2022

hadoop-yetus commented Mar 24, 2022

hadoop-yetus commented Mar 24, 2022

hadoop-yetus commented Mar 24, 2022

hadoop-yetus commented Mar 25, 2022

hadoop-yetus commented Mar 27, 2022

hadoop-yetus commented Mar 28, 2022

hadoop-yetus commented Mar 30, 2022

hadoop-yetus commented Mar 31, 2022

tomscut commented Apr 6, 2022

whbing commented Jul 14, 2022

singer-bin commented Jan 10, 2024

uflswe commented Aug 29, 2024 •

edited

Loading

HDFS-16446. Consider ioutils of disk when choosing volume #3960

Are you sure you want to change the base?

HDFS-16446. Consider ioutils of disk when choosing volume #3960

Conversation

tomscut commented Feb 5, 2022 • edited Loading

hadoop-yetus commented Feb 5, 2022

jojochuang left a comment

Choose a reason for hiding this comment

jojochuang commented Feb 9, 2022

tomscut commented Feb 9, 2022

tomscut commented Feb 9, 2022

hadoop-yetus commented Feb 9, 2022

hadoop-yetus commented Mar 24, 2022

hadoop-yetus commented Mar 24, 2022

hadoop-yetus commented Mar 24, 2022

hadoop-yetus commented Mar 25, 2022

hadoop-yetus commented Mar 27, 2022

hadoop-yetus commented Mar 28, 2022

hadoop-yetus commented Mar 30, 2022

hadoop-yetus commented Mar 31, 2022

tomscut commented Apr 6, 2022

whbing commented Jul 14, 2022

singer-bin commented Jan 10, 2024

uflswe commented Aug 29, 2024 • edited Loading

tomscut commented Feb 5, 2022 •

edited

Loading

uflswe commented Aug 29, 2024 •

edited

Loading