Skip to content

Conversation

@huangzhaobo99
Copy link
Contributor

@huangzhaobo99 huangzhaobo99 commented Jan 6, 2024

Description of PR

JIRA: https://issues.apache.org/jira/browse/HDFS-17327

  1. If it is a null value, after reconfig, It may be that the current system value is in effect, not necessarily the default value.
  2. ReconfigurableBase#reconfigureProperty judges the condition incorrectly, When the effectValue is not null, the system value needs to be changed.
  3. In other words, certain variables actually have effected, but the current configuration of the system is refreshed to null.

How was this patch tested?

Add Unit Test

@hadoop-yetus
Copy link

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 20s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 0s codespell was not available.
+0 🆗 detsecrets 0m 0s detect-secrets was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 1 new or modified test files.
_ trunk Compile Tests _
+0 🆗 mvndep 14m 15s Maven dependency ordering for branch
+1 💚 mvninstall 20m 24s trunk passed
+1 💚 compile 8m 39s trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 💚 compile 7m 49s trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚 checkstyle 2m 4s trunk passed
+1 💚 mvnsite 1m 41s trunk passed
+1 💚 javadoc 1m 35s trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 💚 javadoc 1m 37s trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚 spotbugs 3m 11s trunk passed
+1 💚 shadedclient 23m 56s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+0 🆗 mvndep 0m 20s Maven dependency ordering for patch
+1 💚 mvninstall 1m 5s the patch passed
+1 💚 compile 8m 12s the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 💚 javac 8m 12s the patch passed
+1 💚 compile 8m 2s the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚 javac 8m 2s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
+1 💚 checkstyle 2m 15s the patch passed
+1 💚 mvnsite 1m 40s the patch passed
+1 💚 javadoc 1m 28s the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 💚 javadoc 1m 38s the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚 spotbugs 3m 28s the patch passed
+1 💚 shadedclient 23m 46s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 💚 unit 15m 9s hadoop-common in the patch passed.
-1 ❌ unit 214m 32s /patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt hadoop-hdfs in the patch passed.
-1 ❌ asflicense 0m 42s /results-asflicense.txt The patch generated 7 ASF License warnings.
369m 7s
Reason Tests
Failed junit tests hadoop.hdfs.TestDFSStripedInputStream
hadoop.hdfs.TestReplication
hadoop.hdfs.server.datanode.TestDataNodeReconfiguration
hadoop.hdfs.TestFSInputChecker
hadoop.hdfs.TestFetchImage
hadoop.hdfs.server.namenode.ha.TestDFSUpgradeWithHA
hadoop.hdfs.server.namenode.TestProtectedDirectories
hadoop.hdfs.TestDFSStripedOutputStreamWithRandomECPolicy
Subsystem Report/Notes
Docker ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6421/1/artifact/out/Dockerfile
GITHUB PR #6421
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname Linux dad6a3ad0ba8 5.15.0-88-generic #98-Ubuntu SMP Mon Oct 2 15:18:56 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / e3d090b
Default Java Private Build-1.8.0_392-8u392-ga-1~20.04-b08
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_392-8u392-ga-1~20.04-b08
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6421/1/testReport/
Max. process+thread count 3677 (vs. ulimit of 5500)
modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: .
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6421/1/console
versions git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

@hadoop-yetus
Copy link

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 23s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 0s codespell was not available.
+0 🆗 detsecrets 0m 0s detect-secrets was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 1 new or modified test files.
_ trunk Compile Tests _
+0 🆗 mvndep 14m 6s Maven dependency ordering for branch
+1 💚 mvninstall 22m 4s trunk passed
+1 💚 compile 9m 28s trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 💚 compile 7m 47s trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚 checkstyle 2m 3s trunk passed
+1 💚 mvnsite 1m 37s trunk passed
+1 💚 javadoc 1m 15s trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 💚 javadoc 1m 32s trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚 spotbugs 3m 5s trunk passed
+1 💚 shadedclient 24m 4s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+0 🆗 mvndep 0m 20s Maven dependency ordering for patch
+1 💚 mvninstall 1m 7s the patch passed
+1 💚 compile 8m 58s the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 💚 javac 8m 58s the patch passed
+1 💚 compile 7m 51s the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚 javac 7m 51s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
+1 💚 checkstyle 1m 57s the patch passed
+1 💚 mvnsite 1m 34s the patch passed
+1 💚 javadoc 1m 13s the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 💚 javadoc 1m 32s the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚 spotbugs 3m 16s the patch passed
+1 💚 shadedclient 24m 50s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 💚 unit 14m 25s hadoop-common in the patch passed.
-1 ❌ unit 228m 29s /patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt hadoop-hdfs in the patch passed.
-1 ❌ asflicense 0m 38s /results-asflicense.txt The patch generated 9 ASF License warnings.
385m 7s
Reason Tests
Failed junit tests hadoop.hdfs.server.namenode.TestNamenodeRetryCache
hadoop.hdfs.server.namenode.TestNameNodeReconfigure
hadoop.hdfs.server.namenode.sps.TestStoragePolicySatisfierWithStripedFile
hadoop.hdfs.server.datanode.TestDataNodeReconfiguration
hadoop.hdfs.server.namenode.TestAddStripedBlocks
hadoop.hdfs.TestDecommissionWithStripedBackoffMonitor
hadoop.hdfs.server.namenode.TestProtectedDirectories
hadoop.hdfs.TestDFSStorageStateRecovery
hadoop.hdfs.TestFileChecksum
hadoop.hdfs.server.namenode.TestReconstructStripedBlocks
Subsystem Report/Notes
Docker ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6421/2/artifact/out/Dockerfile
GITHUB PR #6421
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname Linux 96dbbc6ef1e4 5.15.0-88-generic #98-Ubuntu SMP Mon Oct 2 15:18:56 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 2e2769d
Default Java Private Build-1.8.0_392-8u392-ga-1~20.04-b08
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_392-8u392-ga-1~20.04-b08
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6421/2/testReport/
Max. process+thread count 4832 (vs. ulimit of 5500)
modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: .
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6421/2/console
versions git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

@huangzhaobo99 huangzhaobo99 marked this pull request as draft January 7, 2024 01:03
@huangzhaobo99 huangzhaobo99 changed the title HDFS-17327. Fix reconfig disk balancer parameters for datanode. HDFS-17327. Fix the judgment logic of the ReconfigurableBase class method reconfigureProperty. Jan 7, 2024
@huangzhaobo99
Copy link
Contributor Author

Hi @ayushtkn If you have time, review the code logic.
Due to product design issues, a large number of UT do not match the actual situation.
If this logic is correct, I will fix the remaining UT according to the modified code logic.

@huangzhaobo99 huangzhaobo99 marked this pull request as ready for review January 7, 2024 02:20
@huangzhaobo99 huangzhaobo99 changed the title HDFS-17327. Fix the judgment logic of the ReconfigurableBase class method reconfigureProperty. HDFS-17327. Fix ReconfigurableBase class's reconfigureProperty method judgment logic. Jan 7, 2024
@hadoop-yetus
Copy link

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
_ Prechecks _
+1 💚 dupname 0m 01s No case conflicting files found.
+0 🆗 spotbugs 0m 00s spotbugs executables are not available.
+0 🆗 codespell 0m 00s codespell was not available.
+0 🆗 detsecrets 0m 00s detect-secrets was not available.
+1 💚 @author 0m 00s The patch does not contain any @author tags.
+1 💚 test4tests 0m 00s The patch appears to include 1 new or modified test files.
_ trunk Compile Tests _
+0 🆗 mvndep 2m 19s Maven dependency ordering for branch
+1 💚 mvninstall 85m 24s trunk passed
+1 💚 compile 37m 18s trunk passed
+1 💚 checkstyle 5m 36s trunk passed
-1 ❌ mvnsite 4m 11s /branch-mvnsite-hadoop-common-project_hadoop-common.txt hadoop-common in trunk failed.
+1 💚 javadoc 10m 11s trunk passed
+1 💚 shadedclient 153m 48s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+0 🆗 mvndep 2m 11s Maven dependency ordering for patch
+1 💚 mvninstall 8m 55s the patch passed
+1 💚 compile 34m 54s the patch passed
+1 💚 javac 34m 54s the patch passed
+1 💚 blanks 0m 00s The patch has no blanks issues.
+1 💚 checkstyle 5m 49s the patch passed
-1 ❌ mvnsite 4m 12s /patch-mvnsite-hadoop-common-project_hadoop-common.txt hadoop-common in the patch failed.
+1 💚 javadoc 10m 20s the patch passed
+1 💚 shadedclient 166m 40s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 💚 asflicense 5m 31s The patch does not generate ASF License warnings.
503m 26s
Subsystem Report/Notes
GITHUB PR #6421
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname MINGW64_NT-10.0-17763 33900c775089 3.4.10-87d57229.x86_64 2024-02-14 20:17 UTC x86_64 Msys
Build tool maven
Personality /c/hadoop/dev-support/bin/hadoop.sh
git revision trunk / 2e2769d
Default Java Azul Systems, Inc.-1.8.0_332-b09
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch-windows-10/job/PR-6421/1/testReport/
modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: .
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch-windows-10/job/PR-6421/1/console
versions git=2.44.0.windows.1
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

@github-actions
Copy link
Contributor

github-actions bot commented Oct 4, 2025

We're closing this stale PR because it has been open for 100 days with no activity. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue manageable.
If you feel like this was a mistake, or you would like to continue working on it, please feel free to re-open it and ask for a committer to remove the stale tag and review again.
Thanks all for your contribution.

@github-actions github-actions bot added the Stale label Oct 4, 2025
@github-actions github-actions bot closed this Oct 5, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants