New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[BUG]:容器日志文件收集,一个日志文件打开多个文件描述符 #420
Comments
ilogtail version: |
你好,与此相关的logtail配置文件和应用yaml配置能否脱敏后贴一下?我这里比较关注采集路径的配置,透明时请保留层级信息(如有2层就写a/b/warning.log)。另外问一下是什么容器环境呢? |
日志文件:/a/b/c/d/e/warn.log logtail配置
容器中的这个目录通过hostPath挂载到主机上,容器内和主机上目录结构一样
容器环境: |
这个问题我们会持续跟进 |
问题已经定位,会尽快发版修复 |
Describe the bug
A clear and concise description of what the bug is, ideally within 20 words.
容器文件日志收集,一个日志文件打开多个文件描述符,最后触发以下报错:
[2022-09-21 04:36:24.752141] [warning] [000033] /src/core/event_handler/EventHandler.cpp:582 too many open files:skip this read operation log path:/logtail_host/xxxx/warning-2022-09-20.log
[2022-09-21 04:36:24.752146] [error] [000033] /src/core/reader/LogFileReader.cpp:1007 log file reader fd limit, too many open files:/logtail_host/xxxx/warning-2022-09-21.log : limit:25000
查看文件描述符打开数量
[root@ip-10-53-0-57 ilogtail]# ls -l /proc/11/fd |wc -l
25057
查看/proc/11/fd 内容,可以看到多个文件描述符指向通一个文件
lr-x------ 1 root root 64 Sep 21 01:46 17355 -> /logtail_host/xxxx/warning-2022-09-21.log
lr-x------ 1 root root 64 Sep 21 01:46 17382 -> /logtail_host/xxxx/warning-2022-09-21.log
lr-x------ 1 root root 64 Sep 21 01:46 17401 -> /logtail_host/xxxx/warning-2022-09-21.log
....
每次check container path update flag:true ,文件描述符使用就增长
[2022-09-21 04:42:45.520378] [info] [000011] /src/core/config_manager/ConfigManagerBase.cpp:2642 check container path update flag:true size:1
[2022-09-21 04:42:45.520394] [info] [000011] /src/core/observer/network/NetworkObserver.cpp:54 hold on:observer
[2022-09-21 04:42:45.520408] [info] [000011] /src/core/controller/EventDispatcherBase.cpp:1137 main thread:start update config
[2022-09-21 04:42:45.525535] [info] [000011] /src/core/plugin/LogtailPlugin.cpp:98 logtail plugin HoldOn:start
[2022-09-21 04:42:45.529024] [info] [000011] /src/core/plugin/LogtailPlugin.cpp:102 logtail plugin HoldOn:success cost:4
iLogtail Running Environment
Please provide the following information:
ilogtail version:
1.1.1
Yaml configuration:
ilogtail.LOG:
[2022-09-21 04:36:24.752141] [warning] [000033] /src/core/event_handler/EventHandler.cpp:582 too many open files:skip this read operation log path:/logtail_host/xxxx/warning-2022-09-20.log
[2022-09-21 04:36:24.752146] [error] [000033] /src/core/reader/LogFileReader.cpp:1007 log file reader fd limit, too many open files:/logtail_host/xxxx/warning-2022-09-21.log : limit:25000
logtail_plugin.LOG:
The text was updated successfully, but these errors were encountered: