Join GitHub today
When the input file is moved, do not set @reopen_on_eof right away. #23
Setting @reopen_on_eof on a file move notification could result in data not being read from the input file.
I wrote a C program (writer.c) to write lines to a file. When it receives a SIGUSR1 signal it closes the file, reopens another file with the same name, and keeps writing.
I wrote a ruby script (reader.rb) to tail the file with EventMachine::file_tail and write what it reads to another file.
I wrote a perl script to start both programs and periodically rename the file opened by the writer, and then send SIGUSR1 to the writer. What should happen is that the output file of reader.rb should be all complete, consecutive lines. Instead, some lines are missing and cut off because EventMachine::file_tail is losing input.
Put these 4 files in a directory and run make. Then, run perl tester.pl. It runs writer for 10 seconds, sending the signal every 0.2 seconds.
Before my change:
After my change:
You might need to run it more than once to see this happen.
This was a bug that plagued my team for some months before I tracked it down to this gem.
Just to explain this a little more on a macro level, I wanted the proof of concept to be somewhat realistic (although it rotates logs much faster than a typical setup). writer acts like a daemon that writes to a log and responds to a signal to rotate the log. tester.pl acts like logrotate in that it renames the log and sends a signal. reader.rb is tailing the log.