Filebeat ceases indexing activity after restart on large (~300k) number of files

[dilchenko]

Hi folks,

trying to help a fellow community member with what seems to be a simple filebeat setup in https://discuss.elastic.co/t/filebeat/200967/20 (in russian). The gist of the issue is that filebeat is set to index a dir with ~300k (we have seen ~248k according to registry) files. according to the indexing rate graph they shared, these files are infrequently updated (<1000 an hour).

After a restart, however, there seems to be a “pause” of ~8-9hours before filebeat resumes crawling files. Here is a log file from the moment of restart: https://pastebin.com/raw/1qyD1BaN, restart happens at Sep 27 12:02:38. I cannot spot anything wrong with the logs, aside from filebeat not reporting crawler start or ES connection. We have checked the basics, like filebeat process actually restarting, config files being used, etc.

For confirmed bugs, please report:

• Version: 7.3.2
• Operating System: Ubuntu 16.04
• Discuss Forum URL: https://discuss.elastic.co/t/filebeat/200967
• Steps to Reproduce: have a dir with 245k small files, let filebeat index them, restart filebeat, observe no indexing activity for hours

[Axizdkr]

Hello, today I tested it again.
When i was restart a service in registry been some over 100k files. And after one hour after restart crawling do not started.
filebeat.conf - https://pastebin.com/5Z4ButTB
modules.d/bis.yml - https://pastebin.com/0t7K2LHy
/usr/share/filebeat/module/bis/arch/config/input.yml - https://pastebin.com/be5vGYFM

[botelastic]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[botelastic]

This issue doesn't have a Team:<team> label.