fix(file source): sleep to avoid split reads #1236
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Closes #1125
The file watcher works by repeatedly reading from the file being tailed until it sees a newline or
EOF
(i.e. theread
syscall returns0
). Whatever data it has accumulated to that point is returned as a line.When reads are closely interleaved with writes, however, it's possible for
read
to return part of a singlewrite
, and thenEOF
, and then the rest of that samewrite
sometime later. This causes the file watcher to split what should be a single line into two. We've seen this happen ~3 times over 30M+ reads in our test environment.This change causes the file watcher to wait 1 ms for any in-flight writes to become fully visible before returning a line that was never actually terminated with a newline.