airflow db upgrade for 2.3.4 is not working

[aksoni007]

I have been running airflow db upgrade command to upgrade database accordingly but its getting failed with error - "ERROR: You need to upgrade the database. Please run airflow db upgrade. Make sure the command is run using Airflow version 2.3.4."
The same lines of code and everything is working well in sandbox and preprod environment but in prod its getting failed.
I did followed this document and collation also in mysql8.0 db - https://airflow.apache.org/docs/apache-airflow/stable/installation/upgrading.html but did not get any luck. Can you please let me know how to get rid of this error and what are the other possibilities.

[potiuk]

Likely you have different (lower) version of airlfow when you run airflow db upgrade. This happens when one of your other dependencies causes pip or any other tool you use to upgrade airflow to downgrade airflow due to conflicts in dependencies. If you look here https://airflow.apache.org/docs/apache-airflow/stable/installation/installing-from-pypi.html - when you are upgrading/installing airlfow to a specific version, you should always pin the version of airflow (==2.3.4) whenever you install/upgrade any of your dependencies, otherwise - this how dependency resolution works - you might get pip to downgrade your airflow accidentally.

When you pin airlfow with ==, and some of your dependencies are conflicting, you will see an error message telling which of your dependencies are conflcting and you will have to make sure to solve those conflicts - usually by making sure you remove limitations on some other dependencies and make your code compatible with newer versions of those dependencies.

[aksoni007]

I have been using like this @potiuk - RUN pip install --user setuptools wheel &&
pip install --user 'redis==3.2' &&
pip install --user -U pip wheel pytz pyOpenSSL ndg-httpsclient pyasn1
apache-airflow[flask-admin,certifi,requests,urllib3,charset_normalizer,apache-airflow-providers-microsoft-azure,apache-airflow-providers-databricks,crypto,celery,postgres,hive,jdbc,mysql,ssh${AIRFLOW_DEPS:+,}${AIRFLOW_DEPS}]==${AIRFLOW_VERSION}
&& if [ -n "${PYTHON_DEPS}" ]; then pip install --user ${PYTHON_DEPS}; fi

[potiuk]

Can't help more - that would require me to debug your code - just check if whatever you do (say for eample your PYTHON_DEPS) which you forgot to explain what they are) are not bringing your airflow version down. They could, but since you have not posted what they are, it's hard to guess - you can just print the version of airflow you have right before you run the upgrade. that's what I would do if I were to debug it (but it's you who need to debug it).

[aksoni007]

ARG AIRFLOW_VERSION=2.3.4
ARG AIRFLOW_USER_HOME=/opt/airflow

START Directly-specific installs

ARG AIRFLOW_DEPS="slack"
ARG PYTHON_DEPS="ldap3 azure-cli amqpstorm pandas pyarrow"

I AM ALSO PRINTING VERSION OF AIRFLOW JUST BEFORE EXECUTING upgrade command its 2.3.4

[potiuk]

Maybe you have other misconfiguration and point to a diffferent database. Hard to say.