Skip to content

Conversation

@yui2010
Copy link

@yui2010 yui2010 commented Oct 28, 2020

Thread hangs when other thread call filesystem.close(). log as below

971 [main] INFO org.apache.hadoop.fs.cosn.CosNFileSystem - Open the file: [cosn://bigdata-test

1075 [main] ERROR org.apache.hadoop.util.BlockingThreadPoolExecutorService - Could not submit task to executor java.util.concurrent.ThreadPoolExecutor

the reason of thread hangs is :

filesystem is cached when multiple thread using FileSystem#get method for creating instance
CosNInputStream#read method is not check CosNFileSystem is closed
CosNInputStream is using multiple thread(CosNFileReadTask) for read and main thread will await when readbuffer status is init
CosNFileSystem#close method will shutdown boundedIOThreadPool . so CosNInputStream#CosNFileReadTask will not notify main thread and cause thread hangs

@hadoop-yetus
Copy link

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 32m 12s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
-1 ❌ test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ trunk Compile Tests _
+0 🆗 mvndep 14m 50s Maven dependency ordering for branch
+1 💚 mvninstall 20m 44s trunk passed
+1 💚 compile 20m 4s trunk passed with JDK Ubuntu-11.0.9+11-Ubuntu-0ubuntu1.18.04.1
+1 💚 compile 17m 21s trunk passed with JDK Private Build-1.8.0_272-8u272-b10-0ubuntu1~18.04-b10
+1 💚 checkstyle 2m 42s trunk passed
+1 💚 mvnsite 2m 13s trunk passed
+1 💚 shadedclient 20m 27s branch has no errors when building and testing our client artifacts.
+1 💚 javadoc 1m 43s trunk passed with JDK Ubuntu-11.0.9+11-Ubuntu-0ubuntu1.18.04.1
+1 💚 javadoc 2m 16s trunk passed with JDK Private Build-1.8.0_272-8u272-b10-0ubuntu1~18.04-b10
+0 🆗 spotbugs 0m 54s Used deprecated FindBugs config; considering switching to SpotBugs.
+1 💚 findbugs 3m 12s trunk passed
_ Patch Compile Tests _
+0 🆗 mvndep 0m 30s Maven dependency ordering for patch
+1 💚 mvninstall 1m 15s the patch passed
+1 💚 compile 19m 27s the patch passed with JDK Ubuntu-11.0.9+11-Ubuntu-0ubuntu1.18.04.1
+1 💚 javac 19m 27s the patch passed
+1 💚 compile 17m 12s the patch passed with JDK Private Build-1.8.0_272-8u272-b10-0ubuntu1~18.04-b10
+1 💚 javac 17m 12s the patch passed
-0 ⚠️ checkstyle 2m 37s /diff-checkstyle-root.txt root: The patch generated 1 new + 6 unchanged - 0 fixed = 7 total (was 6)
+1 💚 mvnsite 2m 10s the patch passed
+1 💚 whitespace 0m 0s The patch has no whitespace issues.
+1 💚 shadedclient 15m 8s patch has no errors when building and testing our client artifacts.
+1 💚 javadoc 1m 42s the patch passed with JDK Ubuntu-11.0.9+11-Ubuntu-0ubuntu1.18.04.1
+1 💚 javadoc 2m 17s the patch passed with JDK Private Build-1.8.0_272-8u272-b10-0ubuntu1~18.04-b10
+1 💚 findbugs 3m 23s the patch passed
_ Other Tests _
-1 ❌ unit 9m 42s /patch-unit-hadoop-common-project_hadoop-common.txt hadoop-common in the patch passed.
+1 💚 unit 0m 40s hadoop-cos in the patch passed.
+1 💚 asflicense 0m 57s The patch does not generate ASF License warnings.
215m 34s
Reason Tests
Failed junit tests hadoop.security.TestLdapGroupsMapping
Subsystem Report/Notes
Docker ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2420/1/artifact/out/Dockerfile
GITHUB PR #2420
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle
uname Linux af0443d71aa5 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / d0c786d
Default Java Private Build-1.8.0_272-8u272-b10-0ubuntu1~18.04-b10
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.9+11-Ubuntu-0ubuntu1.18.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_272-8u272-b10-0ubuntu1~18.04-b10
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2420/1/testReport/
Max. process+thread count 2194 (vs. ulimit of 5500)
modules C: hadoop-common-project/hadoop-common hadoop-cloud-storage-project/hadoop-cos U: .
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2420/1/console
versions git=2.17.1 maven=3.6.0 findbugs=4.1.3
Powered by Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

@github-actions
Copy link
Contributor

github-actions bot commented Dec 9, 2025

We're closing this stale PR because it has been open for 100 days with no activity. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue manageable.
If you feel like this was a mistake, or you would like to continue working on it, please feel free to re-open it and ask for a committer to remove the stale tag and review again.
Thanks all for your contribution.

@github-actions github-actions bot added the Stale label Dec 9, 2025
@github-actions github-actions bot closed this Dec 10, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants