Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Compress old results #458

Closed
4 tasks done
donkirkby opened this issue Dec 3, 2018 · 1 comment
Closed
4 tasks done

Compress old results #458

donkirkby opened this issue Dec 3, 2018 · 1 comment

Comments

@donkirkby
Copy link
Member

donkirkby commented Dec 3, 2018

The raw data folder is almost full. We purged old image files, but that didn't save much room.

  • Scan old results folders to see how much room each version takes up.
  • Try compressing some of the old versions to see how much room that would save.
  • Consider deleting some of the old versions completely, particularly 6.7 and later that can be regenerated in Kive. (Not worth it after compression.)
  • Compress old version folders before starting a new run folder. Leave the newest version uncompressed, so there will be two uncompressed versions when you finish processing the run folder.
@donkirkby
Copy link
Member Author

Today, there is 2.3TB of free space on the raw data drive (a 14TB drive).
I scanned one out of every 50 runs, and I estimate that there are approximately 1.3TB of output files. I zipped those runs, and I estimate that zipping all the outputs would save approximately 1.1TB.
For comparison, each run generates roughly 6GB of raw data, which adds up to about 300GB per year.

Conclusion: it's worth compressing the old versions before you start processing a folder.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant