Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG]: K8s场景下使用statefulset + pv + ilogtail sidecar采集时数据重复 #1081

Closed
yyuuttaaoo opened this issue Aug 22, 2023 · 0 comments · Fixed by #1082
Closed
Labels
bug Something isn't working
Milestone

Comments

@yyuuttaaoo
Copy link
Collaborator

Describe the bug
容器场景,statefulset使用sidecar进行日志采集,日志和ilogtail checkpoint都存储在pv中。
故障复现
删除pod,让pod调度到一台不同的ecs node上,会出现日志重复采集的现象。

iLogtail Running Environment
查阅日志发现,checkpoint无法通过校验,故无法恢复读取进度。
文件实际是存在的,但从读取日志看,file device发生了变更,从14变成了66311,不过file inode是没有变化的,都是14。
查阅代码,可以发现,checkpoint的校验逻辑中,强校验文件都DeviceId和Inode都要相等。故checkpoint被判断为无法与当前的文件匹配而丢弃。

Fix Proposal
因为checkpoint已经会校验文件名和文件签名,误判的可能性非常低,而文件的inode是不会变化的,提议将这里的DevInode校验弱化为Inode校验。
这样,checkpoint方案将适配sidecar + pv场景的日志采集。

@yyuuttaaoo yyuuttaaoo added the bug Something isn't working label Aug 22, 2023
yyuuttaaoo added a commit to yyuuttaaoo/ilogtail that referenced this issue Aug 22, 2023
…when run as statefulset sidecar

Signed-off-by: Tao Yu <yyuuttaaoo@gmail.com>
yyuuttaaoo added a commit that referenced this issue Aug 23, 2023
…n as statefulset sidecar (#1082)

Signed-off-by: Tao Yu <yyuuttaaoo@gmail.com>
yyuuttaaoo added a commit that referenced this issue Aug 31, 2023
…n as statefulset sidecar (#1082)

Signed-off-by: Tao Yu <yyuuttaaoo@gmail.com>
@yyuuttaaoo yyuuttaaoo added this to the v1.8 milestone Sep 26, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
1 participant