Skip to content

Airflow CLI throwing plugin logs as output #31479

@abhishekbhakat

Description

@abhishekbhakat

Apache Airflow version

2.6.1

What happened

Tried this with Airflow 2.6.1 and some plugins installed (eg: openlineage-airflow)

When we try to run a command airflow variables get test it shows Warning logs with the output.

docker run -it quay.io/astronomer/astro-runtime:8.2.0 bash
astro@b823037e7206:/usr/local/airflow$ a=`airflow variables get test`
Variable test does not exist
astro@b823037e7206:/usr/local/airflow$ echo $a
[2023-05-23T14:33:38.987+0000] {plugin.py:88} INFO - Creating DB tables for astronomer.airflow.version_check.plugin [2023-05-23T14:33:38.998+0000] {plugin.py:97} INFO - Created [2023-05-23T14:33:39.129+0000] {utils.py:429} WARNING - No module named 'paramiko' [2023-05-23T14:33:39.138+0000] {utils.py:429} WARNING - No module named 'airflow.providers.dbt'
astro@b823037e7206:/usr/local/airflow$ b=$(airflow variables get test)
Variable test does not exist
astro@b823037e7206:/usr/local/airflow$ echo $b
[2023-05-23T14:34:01.806+0000] {utils.py:429} WARNING - No module named 'paramiko' [2023-05-23T14:34:01.814+0000] {utils.py:429} WARNING - No module named 'airflow.providers.dbt'
astro@b823037e7206:/usr/local/airflow$

I think the issue is how Airflow plugins handle logs.

What you think should happen instead

The expected output should be:

docker run -it quay.io/astronomer/astro-runtime:8.2.0 bash
astro@23baa05cdc14:/usr/local/airflow$ a = $(airflow variables get test)
Variable test does not exist
bash: a: command not found
astro@23baa05cdc14:/usr/local/airflow$ echo $a

astro@23baa05cdc14:/usr/local/airflow$

How to reproduce

Try fetching a variable using airflow cli with Runtime:8.2.0

Operating System

Debian GNU/Linux 11 (bullseye)

Versions of Apache Airflow Providers

apache-airflow-providers-amazon==8.0.0
apache-airflow-providers-celery==3.1.0
apache-airflow-providers-cncf-kubernetes==6.1.0
apache-airflow-providers-common-sql==1.4.0
apache-airflow-providers-elasticsearch==4.4.0
apache-airflow-providers-ftp==3.3.1
apache-airflow-providers-google==10.0.0
apache-airflow-providers-http==4.3.0
apache-airflow-providers-imap==3.1.1
apache-airflow-providers-microsoft-azure==6.0.0
apache-airflow-providers-postgres==5.4.0
apache-airflow-providers-redis==3.1.0
apache-airflow-providers-sqlite==3.3.2
astronomer-providers==1.15.5

Deployment

Astronomer

Deployment details

Just using runtime image with a simple docker run.

Anything else

No response

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:corekind:bugThis is a clearly a bugneeds-triagelabel for new issues that we didn't triage yet

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions