Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
Add a feature to clean up position informations on in_tail #1126
Google Container Engine uses fluentd pods to collect container log files via the in_tail plugin and forward the logs to Stackdriver logging.
When a container is deleted, kubernetes also deletes the containers log file and there will never be a log file at this filesystem path again.
However the position file will never clean up the obsolete line in the position file although the position value is ffffffffffffffff.
We see production clusters with position files of over 10000 lines.
Can this cause performance problems with fluentd? Should this be fixed?
The config stanza for the containers log files is:
in_tail removes untracked file positions at start phase.
I'm not sure. I didn't receive any report of pos_file releated performance issue.
You can see pos_file implementation is here.
The file which grows infinitely sounds terrible in production environment (even if that growing speed is slow, or if there are no reports about performance regression).
But currently, we have a plan to switch to use storage plugin from pos file for that purpose.
changed the title from
in_tail plugin does not clean deleted log files from position file
Add a feature to clean up position informations on in_tail
Jul 29, 2016
referenced this issue
Jan 4, 2017
How do we get this feature (defect) fixed? This is creating a situation for us - as the POS file is growing rather quickly and the process has to continue to comb thru this file. We see constant CPU cycles even after no additional data has been written to the tailed log.