Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HDFS-16939. Fix the thread safety bug in LowRedundancyBlocks. (#5450)… #5471

Merged
merged 1 commit into from Mar 11, 2023

Conversation

zhangshuyan0
Copy link
Contributor

…. Contributed by Shuyan Zhang.

Description of PR

Cherry-pick from trunk.
The remove method in LowRedundancyBlocks is not protected by synchronized. This method is private and is called by BlockManager. As a result, priorityQueues has the risk of being accessed concurrently by multiple threads.

…#5450). Contributed by Shuyan Zhang.

Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
@hadoop-yetus
Copy link

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 7m 19s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 0s codespell was not available.
+0 🆗 detsecrets 0m 0s detect-secrets was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
-1 ❌ test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ branch-3.3 Compile Tests _
+1 💚 mvninstall 40m 41s branch-3.3 passed
+1 💚 compile 1m 18s branch-3.3 passed
+1 💚 checkstyle 0m 56s branch-3.3 passed
+1 💚 mvnsite 1m 25s branch-3.3 passed
+1 💚 javadoc 1m 39s branch-3.3 passed
+1 💚 spotbugs 3m 24s branch-3.3 passed
+1 💚 shadedclient 26m 28s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+1 💚 mvninstall 1m 16s the patch passed
+1 💚 compile 1m 6s the patch passed
+1 💚 javac 1m 6s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
+1 💚 checkstyle 0m 41s the patch passed
+1 💚 mvnsite 1m 14s the patch passed
+1 💚 javadoc 1m 24s the patch passed
+1 💚 spotbugs 3m 10s the patch passed
+1 💚 shadedclient 25m 39s patch has no errors when building and testing our client artifacts.
_ Other Tests _
-1 ❌ unit 204m 2s /patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt hadoop-hdfs in the patch passed.
+1 💚 asflicense 0m 51s The patch does not generate ASF License warnings.
320m 24s
Reason Tests
Failed junit tests hadoop.hdfs.server.datanode.TestDataNodeRollingUpgrade
Subsystem Report/Notes
Docker ClientAPI=1.42 ServerAPI=1.42 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5471/1/artifact/out/Dockerfile
GITHUB PR #5471
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname Linux acefb07392f6 4.15.0-200-generic #211-Ubuntu SMP Thu Nov 24 18:16:04 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision branch-3.3 / d12c16d
Default Java Private Build-1.8.0_362-8u362-ga-0ubuntu1~18.04.1-b09
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5471/1/testReport/
Max. process+thread count 3387 (vs. ulimit of 5500)
modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5471/1/console
versions git=2.17.1 maven=3.6.0 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

Copy link
Contributor

@Hexiaoqiao Hexiaoqiao left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. +1 from my side. The failed unit test looks not relate with this PR.

@Hexiaoqiao Hexiaoqiao merged commit 8cc57f5 into apache:branch-3.3 Mar 11, 2023
1 check failed
Hexiaoqiao pushed a commit that referenced this pull request Mar 11, 2023
…5471). Contributed by Shuyan Zhang.

Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
(cherry picked from commit 8cc57f5)
@Hexiaoqiao
Copy link
Contributor

Committed to branch-3.3 and branch-3.3.5

ShimmerGlass pushed a commit to ShimmerGlass/hadoop that referenced this pull request Oct 30, 2023
…ks. (apache#5450 apache#5471).

Contributed by Shuyan Zhang.

Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
(cherry picked from commit 8cc57f5)
ShimmerGlass added a commit to criteo-forks/hadoop that referenced this pull request Oct 30, 2023
…ks. (apache#5450 apache#5471). (#60)

Contributed by Shuyan Zhang.

Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
(cherry picked from commit 8cc57f5)

Co-authored-by: zhangshuyan <81411509+zhangshuyan0@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
3 participants