You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
In the implementation of receiver-based connection, there is a race condition when the receiver is restarted.
The current implementation restart a new receiver in a new thread (within Spark Streaming internal) and in the original MessageHandler thread, it will close the FileSystem
This is an undefined behavior according to the API doc of HDFS
In the implementation of receiver-based connection, there is a race condition when the receiver is restarted.
The current implementation restart a new receiver in a new thread (within Spark Streaming internal) and in the original MessageHandler thread, it will close the FileSystem
This is an undefined behavior according to the API doc of HDFS
https://hadoop.apache.org/docs/current/api/org/apache/hadoop/fs/FileSystem.html#close--
Urgent fixes are needed
The text was updated successfully, but these errors were encountered: