Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Logs can't be captured after some time for certain pods #12033

Open
wenlinhe opened this issue Feb 22, 2024 · 0 comments
Open

Logs can't be captured after some time for certain pods #12033

wenlinhe opened this issue Feb 22, 2024 · 0 comments
Labels
component/promtail type/bug Somehing is not working as expected

Comments

@wenlinhe
Copy link

Describe the bug
Kubernetes env, using Promtail collecting pods logs and send to Loki and view the logs in Grafana.
Logs can't be captured after some time for certain pods. For these certain pods, there was logs shown in the beginning but then no new logs captured by Promtail, however there is new logs keep creating in the log file, the only workaround is the copy the log file to a new file then Promtail agent will read the log from new file, but this problem will repeat and repeat. There is no such problem for the other pods in the same Kubernetes(Openshift).

To Reproduce
Steps to reproduce the behavior:

  1. Openshift v4.12+ env.
  2. Promtail agent + kinds of applications
  3. For a give application 'xxxx-jumpbox''s pods log, it can be shown in the beginning but then no logs collected
  4. Try to copy the log to a new log file, Promtail can load the logs from new file

Screenshots, Promtail config, or terminal output
image
image

@JStickler JStickler added component/promtail type/bug Somehing is not working as expected labels Feb 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component/promtail type/bug Somehing is not working as expected
Projects
None yet
Development

No branches or pull requests

2 participants