Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Expose unavailable Kafka source broker #1752

Closed
Jeffail opened this issue Feb 8, 2020 · 1 comment · Fixed by #1918
Closed

Expose unavailable Kafka source broker #1752

Jeffail opened this issue Feb 8, 2020 · 1 comment · Fixed by #1918
Labels
domain: sources Anything related to the Vector's sources meta: good first issue Anything that is good for new contributors. source: kafka Anything `kafka` source related type: enhancement A value-adding code change that enhances its existing functionality.

Comments

@Jeffail
Copy link
Contributor

Jeffail commented Feb 8, 2020

Currently, you can start Vector with an unavailable Kafka broker as the source and see no errors:

[sources.kafka]
    type = "kafka"
    bootstrap_servers = "this-doesnt-exist-fam"
    group_id = "foo"
    topics = ["bar"]

[sinks.out]
    inputs   = ["kafka"]
    type     = "console"
    encoding = "text"

The service starts up and you get no feedback that the connection is failing at all. We should be at the very least intermittently logging errors to indicate the connection is not open.

We should also consider adding healthchecks to sources, eventually we'll want to add endpoints (#1073) for checking the health of the service, and we should offer a probe for whether our components (all of them) are successfully connected.

@Jeffail Jeffail added source: kafka Anything `kafka` source related domain: sources Anything related to the Vector's sources labels Feb 8, 2020
@binarylogic binarylogic added type: enhancement A value-adding code change that enhances its existing functionality. meta: good first issue Anything that is good for new contributors. labels Feb 8, 2020
@binarylogic
Copy link
Contributor

So between this and the silent http failures, I'm wondering if it makes sense to do an audit of our logging across all components, again?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
domain: sources Anything related to the Vector's sources meta: good first issue Anything that is good for new contributors. source: kafka Anything `kafka` source related type: enhancement A value-adding code change that enhances its existing functionality.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants