We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Describe the bug 容器场景,statefulset使用sidecar进行日志采集,日志和ilogtail checkpoint都存储在pv中。 故障复现 删除pod,让pod调度到一台不同的ecs node上,会出现日志重复采集的现象。
iLogtail Running Environment 查阅日志发现,checkpoint无法通过校验,故无法恢复读取进度。 文件实际是存在的,但从读取日志看,file device发生了变更,从14变成了66311,不过file inode是没有变化的,都是14。 查阅代码,可以发现,checkpoint的校验逻辑中,强校验文件都DeviceId和Inode都要相等。故checkpoint被判断为无法与当前的文件匹配而丢弃。
Fix Proposal 因为checkpoint已经会校验文件名和文件签名,误判的可能性非常低,而文件的inode是不会变化的,提议将这里的DevInode校验弱化为Inode校验。 这样,checkpoint方案将适配sidecar + pv场景的日志采集。
The text was updated successfully, but these errors were encountered:
fix alibaba#1081, skip dev check to avoid duplicated data collection …
6b07f85
…when run as statefulset sidecar Signed-off-by: Tao Yu <yyuuttaaoo@gmail.com>
fix #1081, skip dev check to avoid duplicated data collection when ru…
8a50e12
…n as statefulset sidecar (#1082) Signed-off-by: Tao Yu <yyuuttaaoo@gmail.com>
1a2a357
Successfully merging a pull request may close this issue.
Describe the bug
容器场景,statefulset使用sidecar进行日志采集,日志和ilogtail checkpoint都存储在pv中。
故障复现
删除pod,让pod调度到一台不同的ecs node上,会出现日志重复采集的现象。
iLogtail Running Environment
查阅日志发现,checkpoint无法通过校验,故无法恢复读取进度。
文件实际是存在的,但从读取日志看,file device发生了变更,从14变成了66311,不过file inode是没有变化的,都是14。
查阅代码,可以发现,checkpoint的校验逻辑中,强校验文件都DeviceId和Inode都要相等。故checkpoint被判断为无法与当前的文件匹配而丢弃。
Fix Proposal
因为checkpoint已经会校验文件名和文件签名,误判的可能性非常低,而文件的inode是不会变化的,提议将这里的DevInode校验弱化为Inode校验。
这样,checkpoint方案将适配sidecar + pv场景的日志采集。
The text was updated successfully, but these errors were encountered: