Invalid argument(s) 'encoding' sent to create_engine()

[yougyoung94]

Apache Airflow version

Other Airflow 2 version (please specify below)

What happened

Airflow Version: 2.4.3

Python Version: 3.9

I installed airflow via pip install apache-airflow==2.4.3 but whenever I tried to run airflow cli, it gave the following error

Traceback (most recent call last):
  File "/usr/local/bin/airflow", line 5, in <module>
    from airflow.__main__ import main
  File "/usr/local/lib/python3.9/dist-packages/airflow/__init__.py", line 46, in <module>
    settings.initialize()
  File "/usr/local/lib/python3.9/dist-packages/airflow/settings.py", line 573, in initialize
    configure_orm()
  File "/usr/local/lib/python3.9/dist-packages/airflow/settings.py", line 276, in configure_orm
    engine = create_engine(SQL_ALCHEMY_CONN, connect_args=connect_args, **engine_args)
  File "<string>", line 2, in create_engine
  File "/usr/local/lib/python3.9/dist-packages/sqlalchemy/util/deprecations.py", line 281, in warned
    return fn(*args, **kwargs)  # type: ignore[no-any-return]
  File "/usr/local/lib/python3.9/dist-packages/sqlalchemy/engine/create.py", line 680, in create_engine
    raise TypeError(
TypeError: Invalid argument(s) 'encoding' sent to create_engine(), using configuration SQLiteDialect_pysqlite/QueuePool/Engine.  Please check that the keyword arguments are appropriate for this combination of components.

What you think should happen instead

airflow command must normally run

How to reproduce

docker run -it --rm ubuntu:20.04

--- Inside the contaier

apt-get update -y
&& apt-get install software-properties-common=0.99.9.11 -y
&& add-apt-repository ppa:deadsnakes/ppa
&& apt-get install -y --no-install-recommends
python3.9=*
python3.9-dev=*

apt-get update -y && apt-get install python3-pip

pip install apache-airflow==2.4.3

airflow

--- gives following error

Traceback (most recent call last):
  File "/usr/local/bin/airflow", line 5, in <module>
    from airflow.__main__ import main
  File "/usr/local/lib/python3.9/dist-packages/airflow/__init__.py", line 46, in <module>
    settings.initialize()
  File "/usr/local/lib/python3.9/dist-packages/airflow/settings.py", line 573, in initialize
    configure_orm()
  File "/usr/local/lib/python3.9/dist-packages/airflow/settings.py", line 276, in configure_orm
    engine = create_engine(SQL_ALCHEMY_CONN, connect_args=connect_args, **engine_args)
  File "<string>", line 2, in create_engine
  File "/usr/local/lib/python3.9/dist-packages/sqlalchemy/util/deprecations.py", line 281, in warned
    return fn(*args, **kwargs)  # type: ignore[no-any-return]
  File "/usr/local/lib/python3.9/dist-packages/sqlalchemy/engine/create.py", line 680, in create_engine
    raise TypeError(
TypeError: Invalid argument(s) 'encoding' sent to create_engine(), using configuration SQLiteDialect_pysqlite/QueuePool/Engine.  Please check that the keyword arguments are appropriate for this combination of components.

Operating System

ubuntu:20.04

Versions of Apache Airflow Providers

None

Deployment

Other

Deployment details

Ran in docker container

Anything else

No response

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[boring-cyborg]

Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.

[yougyoung94]

I solved this problem by installing apache-airflow==2.6.2 first then downgrade by apache-airflow==2.4.3

Does anyone know what was the problem with 2.4.3 version??

[potiuk]

This is not a problem with 2.4.3. What you llkely had problem with is that you have not used the recommended (and the only one that can lead to reproducible install) method of installing airflow - i.e. with constraints: https://airflow.apache.org/docs/apache-airflow/stable/installation/installing-from-pypi.html

The problem is - like with any other Python package that has no fixed requirements, that 3rd-party depenedncy release might break "plain" installation. And constraints is the way how Airflow solves it, you should always use constraints when you are installing airlfow from scratch and expect to get reproducible installaion.