Skip to content

Commit

Permalink
in_tail: add warning for silent stop on !follow_inodes case
Browse files Browse the repository at this point in the history
For `refresh_watcher`, if an exisiting TailWatcher already
follows a target path with the different inode, it means that
the TailWatcher following the rotated file still exists.

In this case, `refresh_watcher` can't start the new TailWatcher
for the new current file.
So, we should output a warning log in order to prevent silent
collection stops, such as #4327.

The similar warning may work for follow_inodes too.
Just limiting the case to suppress the impact to existing logic.

Signed-off-by: Daijiro Fukuda <fukuda@clear-code.com>
  • Loading branch information
daipom committed Nov 10, 2023
1 parent 36220e5 commit d49cfba
Showing 1 changed file with 23 additions and 1 deletion.
24 changes: 23 additions & 1 deletion lib/fluent/plugin/in_tail.rb
Original file line number Diff line number Diff line change
Expand Up @@ -385,14 +385,36 @@ def refresh_watchers
# So that inode can't be contained in `removed_hash`, and can't be unwatched by `stop_watchers`.
#
# This logic may work for `@follow_inodes false` too.
# Just limiting the case to supress the impact to existing logics.
# Just limiting the case to suppress the impact to existing logics.
@pf&.unwatch_removed_targets(target_paths_hash)
need_unwatch_in_stop_watchers = false
end

removed_hash = existence_paths_hash.reject {|key, value| target_paths_hash.key?(key)}
added_hash = target_paths_hash.reject {|key, value| existence_paths_hash.key?(key)}

# If an exisiting TailWatcher already follows a target path with the different inode,
# it means that the TailWatcher following the rotated file still exists. In this case,
# `refresh_watcher` can't start the new TailWatcher for the new current file. So, we
# should output a warning log in order to prevent silent collection stops.
# (Such as https://github.com/fluent/fluentd/pull/4327)
# (Usually, such a TailWatcher should be removed from `@tails` in `update_watcher`.)
# (The similar warning may work for `@follow_inodes true` too. Just limiting the case
# to suppress the impact to existing logics.)
unless @follow_inodes
target_paths_hash.each do |path, target|
next unless @tails.key?(path)
# We can't use `existence_paths_hash[path].ino` because it is from `TailWatcher.ino`,
# which is very unstable parameter. (It can be `nil` or old).
# So, we need to use `TailWatcher.pe.read_inode`.
existing_watcher_inode = @tails[path].pe.read_inode
if existing_watcher_inode != target.ino
log.warn "Could not follow a file (inode: #{target.ino}) because an existing watcher for that filepath follows a different inode: #{existing_watcher_inode} (e.g. keeps watching a already rotated file). If you keep getting this message, please restart Fluentd.",
filepath: target.path
end
end
end

stop_watchers(removed_hash, unwatched: need_unwatch_in_stop_watchers) unless removed_hash.empty?
start_watchers(added_hash) unless added_hash.empty?
@startup = false if @startup
Expand Down

0 comments on commit d49cfba

Please sign in to comment.