-
Notifications
You must be signed in to change notification settings - Fork 9
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
zarr temp store files filling up local drive #67
Comments
Hi, I can see that this is a bit of an issue. We create these temporary files to enable processing of large files that do not fit in memory - everything is processed in chunks taken from the temp file. However, the temp files should be deleted when you close the audio file in DAS or at the very least when you close DAS. But I guess you did that and the temp files persisted? |
I am using the GUI to predict on a whole folder, so DAS isn't closing between opening files. The zarr temp files do not delete after finishing predictions for one file in the folder and starting the next file. |
Hi, in the newest version 0.32.1, we now explicitly delete the temporary directory after each file was predicted. Can you give this a try and let me know whether it fixes your issue? Thanks. |
Hello, Thank you |
We now manually delete the temporary folders after each file was predicted but maybe we've missed sth. |
Thank you for the quick reply. |
I have an external drive (D:) on my desktop PC that I am using to store my large datasets (typically a directory containing 15 1-hour long wavfiles) that I want to predict using a trained DAS model. However, I am often running into a storage error "No space left on device".
My D: drive has ~1TB of available space, but my local drive is filling up with temp files when DAS is running the predictions. Is there something I did during installation that is making the temp file path be on my local C: drive instead of my larger external D: drive? Is there a way to reroute these temp files to the D: drive so my local drive doesn't fill up? My temporary solution is to go into my temp directory (C:/.../AppData/Local/Temp/) and manually delete the directories created ('zarr...'), which are ~5GB per 1 hour file I am predicting on. I think the code that is creating these temp files is the TempStore class in das/lib/site-packages/zarr/storage.py, I just don't know if there is a way in DAS to redirect these files.
Thank you!
The text was updated successfully, but these errors were encountered: