Skip to content

Conversation

@haiyang1987
Copy link
Contributor

Description of PR

HDFS-16899

TestAddOverReplicatedStripedBlocks#testProcessOverReplicatedAndCorruptStripedBlock occasionally appear failed.

Failed code Line is

// verify that all internal blocks exists except b0
// the redundant internal blocks will not be deleted before the corrupted
// block gets reconstructed. but since we set
// DFS_NAMENODE_REPLICATION_MAX_STREAMS_KEY to 0, the reconstruction will
// not happen
lbs = cluster.getNameNodeRpc().getBlockLocations(filePath.toString(), 0,
 fileLen);
bg = (LocatedStripedBlock) (lbs.get(0)); 
assertEquals(groupSize + 1, bg.getBlockIndices().length); //here not equals. 

From the perspective of normal logic, the internal blocks that need to be obtained are 10, 8 live and 2 redundant internal blocks,
but due to the processing logic that occasionally triggers invalidate to remove redundant internal block,
the final obtained internal blocks value does not meet expectations.

So need avoid the redundant internal blocks will be deleted.

@hadoop-yetus
Copy link

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 44s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 1s No case conflicting files found.
+0 🆗 codespell 0m 0s codespell was not available.
+0 🆗 detsecrets 0m 0s detect-secrets was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 1 new or modified test files.
_ trunk Compile Tests _
+1 💚 mvninstall 43m 30s trunk passed
+1 💚 compile 1m 27s trunk passed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04
+1 💚 compile 1m 22s trunk passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08
+1 💚 checkstyle 1m 7s trunk passed
+1 💚 mvnsite 1m 30s trunk passed
+1 💚 javadoc 1m 8s trunk passed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04
+1 💚 javadoc 1m 34s trunk passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08
+1 💚 spotbugs 3m 23s trunk passed
+1 💚 shadedclient 25m 53s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+1 💚 mvninstall 1m 27s the patch passed
+1 💚 compile 1m 19s the patch passed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04
+1 💚 javac 1m 19s the patch passed
+1 💚 compile 1m 15s the patch passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08
+1 💚 javac 1m 15s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
+1 💚 checkstyle 0m 52s the patch passed
+1 💚 mvnsite 1m 19s the patch passed
+1 💚 javadoc 0m 50s the patch passed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04
+1 💚 javadoc 1m 28s the patch passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08
+1 💚 spotbugs 3m 16s the patch passed
+1 💚 shadedclient 25m 34s patch has no errors when building and testing our client artifacts.
_ Other Tests _
-1 ❌ unit 202m 48s /patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt hadoop-hdfs in the patch passed.
+1 💚 asflicense 0m 52s The patch does not generate ASF License warnings.
320m 57s
Reason Tests
Failed junit tests hadoop.hdfs.server.namenode.TestAuditLogger
hadoop.hdfs.server.namenode.TestFsck
hadoop.hdfs.server.namenode.TestAuditLogs
hadoop.hdfs.server.namenode.TestFSNamesystemLockReport
Subsystem Report/Notes
Docker ClientAPI=1.42 ServerAPI=1.42 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5414/1/artifact/out/Dockerfile
GITHUB PR #5414
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname Linux d0f1d8f5bbda 4.15.0-200-generic #211-Ubuntu SMP Thu Nov 24 18:16:04 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / b0e3c65
Default Java Private Build-1.8.0_352-8u352-ga-1~20.04-b08
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_352-8u352-ga-1~20.04-b08
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5414/1/testReport/
Max. process+thread count 3243 (vs. ulimit of 5500)
modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5414/1/console
versions git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

@haiyang1987
Copy link
Contributor Author

The failed unit test seems unrelated to the change, it is traced by HADOOP-18628 and HDFS-16925 is solving.

@haiyang1987
Copy link
Contributor Author

@ZanderXu @tomscut please help me review this PR when you are available, Thanks.

@haiyang1987
Copy link
Contributor Author

hi sir @ZanderXu @Hexiaoqiao @ayushtkn @tomscut please help me review this occasionally trigger failed UT fix when you are available, Thanks.

@github-actions
Copy link
Contributor

We're closing this stale PR because it has been open for 100 days with no activity. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue manageable.
If you feel like this was a mistake, or you would like to continue working on it, please feel free to re-open it and ask for a committer to remove the stale tag and review again.
Thanks all for your contribution.

@github-actions github-actions bot added the Stale label Oct 26, 2025
@github-actions github-actions bot closed this Oct 27, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants