vector will consume the deleted log

[zzmg]

A note for the community

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Problem

error log:
2023-12-04T11:47:38.166255Z ERROR source{component_kind="source" component_id=in_k8s component_type=kubernetes_logs component_name=in_k8s}: vector::internal_events::kubernetes_logs: Internal log [Failed to annotate event with pod metadata.] has been suppressed 30502 times.
2023-12-04T11:47:38.166275Z ERROR source{component_kind="source" component_id=in_k8s component_type=kubernetes_logs component_name=in_k8s}: vector::internal_events::kubernetes_logs: Failed to annotate event with pod metadata. event=Log(LogEvent { inner: Inner { fields: Object({"file": Bytes(b"/var/log/pods/traefik_traefik-847668b4dc-4r48l_dc75eb95-b854-4138-b883-41cd818bf642/traefik/0.log"), "message": Bytes(b"2023-12-04T11:19:05.291522693Z stdout F {"ClientAddr\

this file: /var/log/pods/traefik_traefik-847668b4dc-4r48l_dc75eb95-b854-4138-b883-41cd818bf642/traefik/0.log， no longer exists on the k8s worker node.

Configuration

in_k8s:
    auto_partial_merge: true
    data_dir: /var/lib/vector
    delay_deletion_ms: 60000
    exclude_paths_glob_patterns:
      - "**/vector/**"
      - "**/istio-init/**"
      - "**/filebeat/**"
      - "**/vmagent/**"
      - "**/nats/**"
      - "**/*.gz"
      - "**/*.log.*"
      - "**/vector-logfilter/**"
      - "**/geoip/**"
    fingerprint_lines: 1
    glob_minimum_cooldown_ms: 60000
    max_line_bytes: 32768
    max_read_bytes: 4096
    self_node_name: "${VECTOR_SELF_NODE_NAME}"
    timezone: local
    type: kubernetes_logs
    use_apiserver_cache: true

Version

0.33

Debug Output

No response

Example Data

No response

Additional Context

No response

References

No response

[jszwedko]

Hi @zzmg ,

This is expected behavior. Vector will hold open handles to any files that it has not yet fully read.

I'll close this out, but let us know if you have any further questions!