-
Notifications
You must be signed in to change notification settings - Fork 3.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Temporary working folders are left behind on Middle Managers after tasks complete #12332
Comments
We saw this too in our clusters. Also timed out queries are also leaving tmp folders. |
This issue has been marked as stale due to 280 days of inactivity. |
This issue has been closed due to lack of activity. If you think that |
I recommend reopening this issue, as I've also encountered such problem, which could lead to the failure of ingestion task if the disk space is fully, and this could be a significant concern that this belongs to the resource leak problem 😅 |
Hi @sergioferragut , have you had a chance to check the |
@abhishekagarwal87 Do you have any idea on this one 😄 |
What version are you on? I don't see such folders on my local box. Can you post your ingestion spec that you are running? |
Hi @abhishekagarwal87 , same as the version that @sergioferragut mentioned in this issue, yes, this indeed is a very low probability event. Now that we are using the MoK mode with the latest version of Druid, this issue no longer affects us 😅 |
Affected Version
Apache Druid 0.22.1
Description
This problem was originally reported here: https://www.druidforum.org/t/temp-folder-size-was-increasing-due-to-that-peons-processing-taking-more-time-how-to-clear-temp-folder-automatically/7139
I was able to reproduce it by running on a small minikube deployment by running the vanilla wikipedia index_parallel ingestion a few times, each with a different target datasource name and confirmed that after the jobs completed the temporary folders for the tasks are not being removed, after 3 runs, the ~/var/tmp folder still contained the three empty folders:
The original report on Druid Forum spoke of thousands of such folders left behind.
The text was updated successfully, but these errors were encountered: