Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Connection test problem with apache-airflow-providers-apache-kafka #40204

Closed
2 tasks done
riccardoforzan opened this issue Jun 12, 2024 · 2 comments · Fixed by #40284
Closed
2 tasks done

Connection test problem with apache-airflow-providers-apache-kafka #40204

riccardoforzan opened this issue Jun 12, 2024 · 2 comments · Fixed by #40284

Comments

@riccardoforzan
Copy link
Contributor

Apache Airflow Provider(s)

apache-kafka

Versions of Apache Airflow Providers

1.4.1

Apache Airflow version

2.9.2

Operating System

Debian 11

Deployment

Docker-Compose

Deployment details

No response

What happened

I am encountering an issue with Apache Airflow 2.9.2 when attempting to test a connection to Kafka.
I followed the integration guide as per the documentation here.

After setting up the connection to Kafka, testing the connection with the command airflow connections test <connection-name> results in a failure.

It appears that the test utilizes KafkaBaseHook, and it fails when reaching this specific statement: base.py#L56

Here is the configuration dictionary I am using

{
    "bootstrap.servers": "server",
    "security.protocol": "SASL_SSL",
    "sasl.mechanisms": "PLAIN",
    "sasl.username": "username",
    "sasl.password": "password"
}

What you think should happen instead

I think it should use an instance of KafkaAdminClientHook instead of an instance of the generic class, which throws when invoking _get_client

How to reproduce

Anything else

No response

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

@riccardoforzan riccardoforzan added area:providers kind:bug This is a clearly a bug needs-triage label for new issues that we didn't triage yet labels Jun 12, 2024
Copy link

boring-cyborg bot commented Jun 12, 2024

Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.

@potiuk potiuk added good first issue and removed needs-triage label for new issues that we didn't triage yet labels Jun 13, 2024
@potiuk
Copy link
Member

potiuk commented Jun 13, 2024

Yes. There is a base Kafka connection and when "Test connection" is used, it instantiates Hook based on the type of connection - so it will instantiate the Base Kafka Hook and there _get_client is Not Implemented - easy fix will be to turn the NotImplemented into a client that can do a basic connection test.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants