s3 output: resolve stale-detection races #19
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This is largely a forward-port of logstash-plugins/logstash-output-s3#252 with some minor changes to deal with the integrated plugin's usage of the java-native `ConcurrentHashMap` in place of the stand-alone plugin's ruby-native `Concurrent::Map`. Refactor of `S3::FileRepository` to avoid several closely-related race conditions: - prevent `get_factory()` from yielding a factory that was mid-deletion by the stale watcher, which could cause the plugin crash due to the file no longer existing on disk. This is solved by marking a factory's prefix wrapper as deleted while the stale watcher has exclusive access to it, and checking for deletion status before yielding exclusive access to a prefix wrapper's factory. - introduce `each_factory`, which _avoids_ creating new factories or yielding deleted ones. - refactor `each_files` to use new `each_factory` to avoid yielding files whose factories have been deleted. - void-return methods now explicitly emit `nil` to prevent accidental leaks of synchronization-required resources. Additionally, `S3#rotate_if_needed` was migrated to use the now-safer `S3::FileRepository#each_factory` that _avoids_ initializing new factories (and therefore avoids creating empty files on disk after the existing ones had been stale-reaped).
mashhurs
approved these changes
Dec 23, 2022
There was a problem hiding this comment.
LGTM!
Aligns with https://github.com/logstash-plugins/logstash-output-s3/pull/252/files changes
mashhurs
reviewed
Dec 23, 2022
yaauie
commented
Dec 23, 2022
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is largely a forward-port of logstash-plugins/logstash-output-s3#252 with some minor changes to deal with the integrated plugin's usage of the java-native
ConcurrentHashMapin place of the stand-alone plugin's ruby-nativeConcurrent::Map.Refactor of
S3::FileRepositoryto avoid several closely-related race conditions:get_factory()from yielding a factory that was mid-deletion by the stale watcher, which could cause the plugin crash due to the file no longer existing on disk. This is solved by marking a factory's prefix wrapper as deleted while the stale watcher has exclusive access to it, and checking for deletion status before yielding exclusive access to a prefix wrapper's factory.each_factory, which avoids creating new factories or yielding deleted ones.each_filesto use neweach_factoryto avoid yielding files whose factories have been deleted.nilto prevent accidental leaks of synchronization-required resources.Additionally,
S3#rotate_if_neededwas migrated to use the now-saferS3::FileRepository#each_factorythat avoids initializing new factories (and therefore avoids creating empty files on disk after the existing ones had been stale-reaped).