Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

S3 Output should clean up empty temp folders #34

Closed
robbavey opened this issue Jun 20, 2023 · 0 comments · Fixed by #39
Closed

S3 Output should clean up empty temp folders #34

robbavey opened this issue Jun 20, 2023 · 0 comments · Fixed by #39
Assignees
Labels
bug Something isn't working int-shortlist

Comments

@robbavey
Copy link
Contributor

robbavey commented Jun 20, 2023

#28 describes a situation where the s3 output cannot delete a temporary folder, due to the folder being non-empty.

If we fail to delete the folder, we never attempt to delete again, raising a warning and moving on, leading to a proliferation of empty temp folders on systems that are prone to this issue (we believe that Windows boxes, in particular those running antivirus scans, are more prone to this issue than other systems).

The current cleanup implementation deliberately skips empty folders -

https://github.com/logstash-plugins/logstash-integration-aws/blob/main/lib/logstash/outputs/s3.rb#L399

and only cleans up folders containing partially completed upload files, and only if the restore option is selected in the configuration.

We should improve the cleanup process to also delete empty temporary folders.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working int-shortlist
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants