Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Addon: Collect logs to Kafka topic #40

Closed
wants to merge 8 commits into from
Closed

Addon: Collect logs to Kafka topic #40

wants to merge 8 commits into from

Conversation

solsson
Copy link
Contributor

@solsson solsson commented Jul 7, 2017

Note that unlike #39 we don't want a test consumer in this addon, becauase it would cause an infinite loop of logging. Use kubectl exec to some kafka(cat) container with a console consumer.

Currently two blockers:

  • tail -f *.log won't discover new files
  • can't use the filename headers from tail -f as delimiters because kafkacat supports only single char delimiters.

Maybe a dedicated tool like fluent-bit is required for proper log folder processing.

@solsson solsson added the addon label Jul 7, 2017
@solsson solsson changed the title Addon logs Addon: Collect logs to Kafka topic Jul 7, 2017
@solsson
Copy link
Contributor Author

solsson commented Jul 27, 2017

@solsson solsson changed the base branch from kafka-011 to master November 4, 2017 09:21
@solsson
Copy link
Contributor Author

solsson commented Nov 4, 2017

The "raw" collector got a huge improvement for #84 in da19538 and 8528f0b.

@solsson
Copy link
Contributor Author

solsson commented Jan 22, 2018

Keeping this addon open for 2.x users, but I think the way forward is #131.

@solsson
Copy link
Contributor Author

solsson commented Apr 21, 2021

Keeping this addon open for 2.x users, but I think the way forward is #131.

And there's Loki.

@solsson solsson closed this Apr 21, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant