HADOOP-17334 CosNInputStream should throw exception when filesystem i… #2420
+27
−2
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Thread hangs when other thread call filesystem.close(). log as below
971 [main] INFO org.apache.hadoop.fs.cosn.CosNFileSystem - Open the file: [cosn://bigdata-test
1075 [main] ERROR org.apache.hadoop.util.BlockingThreadPoolExecutorService - Could not submit task to executor java.util.concurrent.ThreadPoolExecutor
the reason of thread hangs is :
filesystem is cached when multiple thread using FileSystem#get method for creating instance
CosNInputStream#read method is not check CosNFileSystem is closed
CosNInputStream is using multiple thread(CosNFileReadTask) for read and main thread will await when readbuffer status is init
CosNFileSystem#close method will shutdown boundedIOThreadPool . so CosNInputStream#CosNFileReadTask will not notify main thread and cause thread hangs