Join GitHub today
GitHub is home to over 40 million developers working together to host and review code, manage projects, and build software together.Sign up
Logstash lost data during log rotate #214
I have use logstash-input-file(4.1.4) to ingest from file. I found data loss during log rotation.
My configuration of log rotate:
My script to generate log and rotate:
if ! ((i % 1000)); then
if ! ((i % 30000 || i == 100000)); then
My configuration of logstash:
Data loss happened as below:
Please give me some advice on this issue.
@lrbsunday I have test with input path as "/tmp/log/test" and "/tmp/log/test*". And data loss is the same.
I have add a flag(:rotate_flag) in "watced_file.rb" to avoid data loss. But i am not sure whether my change will bring other issues. Maybe you can give me some advice.
The temporary work around is to use