in_tail: follow symlink target change

[bmalynovytch]

Bug Report

Summary
in_tail doesn't follow target symlink changes

To Reproduce

• Configure INPUT:

[INPUT]
    Name        tail
    Path        /opt/my_app/releases/current/var/log/exceptions.log

• Configure /opt/my_app/releases/current to be a symlink of /opt/my_app/releases/v1.2.3
• Start agent
• Rotate symlink to point to somewhere else (but still valid target) /opt/my_app/releases/v1.2.4
• No more logs will be processed as file descriptor still points to initial target of symlink

Expected behavior
in_tail should probe regularly (or listen for event) to check if rotation occurred and reload it's FD to point to the right file.

Your Environment

• Version used: Fluent Bit v1.0.4
• Configuration: N/A
• Server type and version: N/A
• Operating System and version: Linux Ubuntu 16.04
• Filters and plugins: N/A

Additional context
Using symlinks to handle running version of an application with its logs stored in a subfolder prevents the log collection to continue working after release a new version without restarting agent, which could cause privilege troubles as the releasing user may not have enough privileges to trigger a restart of the agent.

[wulfgar93]

Hello! Got the same issue with 1.2.2 on CentOS 7. Is there any chance to fix this?

As workaround I use cron for restart td-agent-bit every 15 minutes...

[wtan825]

Hi. Got the same issue with 1.3. Is there any chance to fix this?

[wtan825]

@edsiper

[yusufgungor]

@bmalynovytch in our case, even file is not rotated fluentd recognize it like rotated and tail the same log from head. Is there any relation with this commits?

2020-04-28 21:19:24 +0000 [debug]: [my-containers.log] tailing paths: target = /var/log/containers/myFirstApp-8d74cb847-q9zl9_myNameSpace_myFirstApp-container-79f420d1a203b758f723f6346d7a5f8d84530fdbb6ab009ad196230e4f4baa83.log,/var/log/containers/mySecondApp-7f59799d7d-dl72n_myNameSpace_mySecondApp-container-c01fe8f43d78c8b471dd61267cf8888612f5011ed9bd1c3c7f736b98af320f0b.log | existing = /var/log/containers/myFirstApp-8d74cb847-q9zl9_myNameSpace_myFirstApp-container-79f420d1a203b758f723f6346d7a5f8d84530fdbb6ab009ad196230e4f4baa83.log,/var/log/containers/mySecondApp-7f59799d7d-dl72n_myNameSpace_mySecondApp-container-c01fe8f43d78c8b471dd61267cf8888612f5011ed9bd1c3c7f736b98af320f0b.log

We can see that same files exist on target and existing lists.

[pbn4]

Same problem here.

[pbn4]

@edsiper Can I help you with that somehow?

[pbn4]

Also looking for a workaround. :)

[edsiper]

we have a couple of improvements on in_tail that will land in v1.4.6 (few days) and v1.5 (couple of weeks). Right now the team is working on porting the fixes and adjustments for Windows

we will have an update soon

[github-actions]

This issue is stale because it has been open 90 days with no activity. Remove stale label or comment or this will be closed in 5 days. Maintainers can add the exempt-stale label.

[github-actions]

This issue was closed because it has been stalled for 5 days with no activity.

[bmalynovytch]

@edsiper I think this issue should stay opened. Would you mind reopening it ? 🙏

[adapasuresh]

+1 issue