Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove run directories after a delay, if they are empty #109

Merged
merged 1 commit into from
Mar 21, 2022

Conversation

kjsanger
Copy link
Member

This prevents an accumulation of empty directories on the instrument
after data have been transferred elsewhere. The run directories
contain the bulk of the empty directories, particularly if barcoding
has been used.

Directories are not removed if they or their descedants contain one or
more files.

@kjsanger kjsanger added the enhancement New feature or request label Feb 16, 2022
@kjsanger kjsanger force-pushed the feature/cleanup-empty-directories branch 3 times, most recently from f0bc5a0 to 2b8b126 Compare February 16, 2022 11:53
This prevents an accumulation of empty directories on the instrument
after data have been transferred elsewhere. The run directories
contain the bulk of the empty directories, particularly if barcoding
has been used.

Directories are not removed if they or their descendants contain one or
more files.

A new command line option for the archive command allows the age
threshold of a deletable run directory to be specified, defaulting to
14 days, with a minimum of 60 seconds.
@kjsanger kjsanger force-pushed the feature/cleanup-empty-directories branch from 2b8b126 to 65087d1 Compare March 8, 2022 17:00
@kjsanger kjsanger removed the request for review from mksanger March 9, 2022 09:38
@dkj dkj self-requested a review March 14, 2022 17:09
@dkj
Copy link
Member

dkj commented Mar 15, 2022

Tech Q. - should we be removing the inotify listeners [watches] on the directories being removed? Or does that happen automatically?

@dkj
Copy link
Member

dkj commented Mar 16, 2022

Tech Q. - should we be removing the inotify listeners [watches] on the directories being removed? Or does that happen automatically?

Should be okay:
fsnotify/fsnotify#238

@kjsanger
Copy link
Member Author

Tech Q. - should we be removing the inotify listeners [watches] on the directories being removed? Or does that happen automatically?

There is a final cleanup here

if err := w.Close(); err != nil {
but that's when everything is over. My understanding is that the library handles this. Doing some rough experiments (trying to exhaust the limit) seemed to confirm this.

Copy link
Member

@dkj dkj left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tests cover new functionality.
14 day wait for clean up of dirs seems safe.
Understand watchers on dirs will go when dirs are removed (OS level).
Review limited by reviewer's understanding of go...

@kjsanger kjsanger merged commit 92c6b90 into wtsi-npg:devel Mar 21, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants