Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove Python 3.7 support #30963

Merged
merged 1 commit into from
May 29, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/actions/breeze/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -37,8 +37,8 @@ runs:
path: ~/.local/pipx
# README has the latest breeze's hash and python location is used to distinguish between
# different minor versions of python
key: "breeze-${{ env.pythonLocation }}-${{ hashFiles('dev/breeze/README.md') }}"
restore-keys: breeze-${{ env.pythonLocation }}
key: "breeze-3.8-${{ env.pythonLocation }}-${{ hashFiles('dev/breeze/README.md') }}"
restore-keys: breeze-3.8-${{ env.pythonLocation }}
- name: "Install Breeze"
shell: bash
run: ./scripts/ci/install_breeze.sh
Expand Down
2 changes: 1 addition & 1 deletion .github/actions/build-prod-images/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ runs:
shell: bash
run: breeze ci-image pull --tag-as-latest
env:
PYTHON_MAJOR_MINOR_VERSION: "3.7"
PYTHON_MAJOR_MINOR_VERSION: "3.8"
- name: "Cleanup dist and context file"
shell: bash
run: rm -fv ./dist/* ./docker-context-files/*
Expand Down
2 changes: 1 addition & 1 deletion .readthedocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ formats: []
sphinx:
configuration: docs/rtd-deprecation/conf.py
python:
version: "3.7"
version: "3.8"
install:
- method: pip
path: .
Expand Down
43 changes: 37 additions & 6 deletions BREEZE.rst
Original file line number Diff line number Diff line change
Expand Up @@ -245,6 +245,17 @@ Run this command to install Breeze (make sure to use ``-e`` flag):

pipx install -e ./dev/breeze


.. note:: Note for Windows users

The ``./dev/breeze`` in command about is a PATH to sub-folder where breeze source packages are.
If you are on Windows, you should use Windows way to point to the ``dev/breeze`` sub-folder
of Airflow either as absolute or relative path. For example:

.. code-block:: bash

pipx install -e dev\breeze

Once this is complete, you should have ``breeze`` binary on your PATH and available to run by ``breeze``
command.

Expand Down Expand Up @@ -278,6 +289,26 @@ where it was installed.
You can run ``breeze setup version`` command to see where breeze installed from and what are the current sources
that Breeze works on

.. warning:: Upgrading from earlier Python version

If you used Breeze with Python 3.7 and when running it, it will complain that it needs Python 3.8. In this
case you should force-reinstall Breeze with ``pipx``:

.. code-block:: bash

pipx install --force -e ./dev/breeze

.. note:: Note for Windows users

The ``./dev/breeze`` in command about is a PATH to sub-folder where breeze source packages are.
If you are on Windows, you should use Windows way to point to the ``dev/breeze`` sub-folder
of Airflow either as absolute or relative path. For example:

.. code-block:: bash

pipx install --force -e dev\breeze


Running Breeze for the first time
---------------------------------

Expand Down Expand Up @@ -360,12 +391,12 @@ You can use additional ``breeze`` flags to choose your environment. You can spec
version to use, and backend (the meta-data database). Thanks to that, with Breeze, you can recreate the same
environments as we have in matrix builds in the CI.

For example, you can choose to run Python 3.7 tests with MySQL as backend and with mysql version 8
For example, you can choose to run Python 3.8 tests with MySQL as backend and with mysql version 8
as follows:

.. code-block:: bash

breeze --python 3.7 --backend mysql --mysql-version 8
breeze --python 3.8 --backend mysql --mysql-version 8

The choices you make are persisted in the ``./.build/`` cache directory so that next time when you use the
``breeze`` script, it could use the values that were used previously. This way you do not have to specify
Expand Down Expand Up @@ -526,15 +557,15 @@ When you are starting airflow from local sources, www asset compilation is autom

.. code-block:: bash

breeze --python 3.7 --backend mysql start-airflow
breeze --python 3.8 --backend mysql start-airflow


You can also use it to start any released version of Airflow from ``PyPI`` with the
``--use-airflow-version`` flag.

.. code-block:: bash

breeze start-airflow --python 3.7 --backend mysql --use-airflow-version 2.2.5
breeze start-airflow --python 3.8 --backend mysql --use-airflow-version 2.2.5

Those are all available flags of ``start-airflow`` command:

Expand Down Expand Up @@ -1422,10 +1453,10 @@ suffix and they need to also be paired with corresponding runtime dependency add

.. code-block:: bash

breeze prod-image build --python 3.7 --additional-dev-deps "libasound2-dev" \
breeze prod-image build --python 3.8 --additional-dev-deps "libasound2-dev" \
--additional-runtime-apt-deps "libasound2"

Same as above but uses python 3.7.
Same as above but uses python 3.8.

Building PROD image
...................
Expand Down
6 changes: 3 additions & 3 deletions CI.rst
Original file line number Diff line number Diff line change
Expand Up @@ -145,7 +145,7 @@ have to be percent-encoded when you access them via UI (/ = %2F)
+--------------+----------------------------------------------------------+----------------------------------------------------------+

* <BRANCH> might be either "main" or "v2-*-test"
* <X.Y> - Python version (Major + Minor).Should be one of ["3.7", "3.8", "3.9"].
* <X.Y> - Python version (Major + Minor).Should be one of ["3.8", "3.9", "3.10", "3.11"].
* <COMMIT_SHA> - full-length SHA of commit either from the tip of the branch (for pushes/schedule) or
commit from the tip of the branch used for the PR.

Expand Down Expand Up @@ -523,9 +523,9 @@ For example knowing that the CI job was for commit ``cd27124534b46c9688a1d89e75f

.. code-block:: bash

docker pull ghcr.io/apache/airflow/main/ci/python3.7:cd27124534b46c9688a1d89e75fcd137ab5137e3
docker pull ghcr.io/apache/airflow/main/ci/python3.8:cd27124534b46c9688a1d89e75fcd137ab5137e3

docker run -it ghcr.io/apache/airflow/main/ci/python3.7:cd27124534b46c9688a1d89e75fcd137ab5137e3
docker run -it ghcr.io/apache/airflow/main/ci/python3.8:cd27124534b46c9688a1d89e75fcd137ab5137e3


But you usually need to pass more variables and complex setup if you want to connect to a database or
Expand Down
10 changes: 5 additions & 5 deletions CONTRIBUTING.rst
Original file line number Diff line number Diff line change
Expand Up @@ -414,7 +414,7 @@ these guidelines:
- Run tests locally before opening PR.

- You can use any supported python version to run the tests, but the best is to check
if it works for the oldest supported version (Python 3.7 currently). In rare cases
if it works for the oldest supported version (Python 3.8 currently). In rare cases
tests might fail with the oldest version when you use features that are available in newer Python
versions. For that purpose we have ``airflow.compat`` package where we keep back-ported
useful features from newer versions.
Expand Down Expand Up @@ -840,7 +840,7 @@ from the PyPI package:
.. code-block:: bash

pip install apache-airflow[google,amazon,async]==2.2.5 \
--constraint "https://raw.githubusercontent.com/apache/airflow/constraints-2.2.5/constraints-3.7.txt"
--constraint "https://raw.githubusercontent.com/apache/airflow/constraints-2.2.5/constraints-3.8.txt"

The last one can be used to install Airflow in "minimal" mode - i.e when bare Airflow is installed without
extras.
Expand All @@ -852,15 +852,15 @@ requirements).
.. code-block:: bash

pip install -e . \
--constraint "https://raw.githubusercontent.com/apache/airflow/constraints-main/constraints-source-providers-3.7.txt"
--constraint "https://raw.githubusercontent.com/apache/airflow/constraints-main/constraints-source-providers-3.8.txt"


This works also with extras - for example:

.. code-block:: bash

pip install ".[ssh]" \
--constraint "https://raw.githubusercontent.com/apache/airflow/constraints-main/constraints-source-providers-3.7.txt"
--constraint "https://raw.githubusercontent.com/apache/airflow/constraints-main/constraints-source-providers-3.8.txt"


There are different set of fixed constraint files for different python major/minor versions and you should
Expand All @@ -872,7 +872,7 @@ If you want to update just airflow dependencies, without paying attention to pro
.. code-block:: bash

pip install . --upgrade \
--constraint "https://raw.githubusercontent.com/apache/airflow/constraints-main/constraints-no-providers-3.7.txt"
--constraint "https://raw.githubusercontent.com/apache/airflow/constraints-main/constraints-no-providers-3.8.txt"


The ``constraints-<PYTHON_MAJOR_MINOR_VERSION>.txt`` and ``constraints-no-providers-<PYTHON_MAJOR_MINOR_VERSION>.txt``
Expand Down
2 changes: 1 addition & 1 deletion Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ ARG AIRFLOW_USER_HOME_DIR=/home/airflow
# latest released version here
ARG AIRFLOW_VERSION="2.6.1"

ARG PYTHON_BASE_IMAGE="python:3.7-slim-bullseye"
ARG PYTHON_BASE_IMAGE="python:3.8-slim-bullseye"

ARG AIRFLOW_PIP_VERSION=23.1.2
ARG AIRFLOW_IMAGE_REPOSITORY="https://github.com/apache/airflow"
Expand Down
4 changes: 2 additions & 2 deletions Dockerfile.ci
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@
#
# WARNING: THIS DOCKERFILE IS NOT INTENDED FOR PRODUCTION USE OR DEPLOYMENT.
#
ARG PYTHON_BASE_IMAGE="python:3.7-slim-bullseye"
ARG PYTHON_BASE_IMAGE="python:3.8-slim-bullseye"

##############################################################################################
# This is the script image where we keep all inlined bash scripts needed in other segments
Expand Down Expand Up @@ -615,7 +615,7 @@ chmod 1777 /tmp

AIRFLOW_SOURCES=$(cd "${IN_CONTAINER_DIR}/../.." || exit 1; pwd)

PYTHON_MAJOR_MINOR_VERSION=${PYTHON_MAJOR_MINOR_VERSION:=3.7}
PYTHON_MAJOR_MINOR_VERSION=${PYTHON_MAJOR_MINOR_VERSION:=3.8}

export AIRFLOW_HOME=${AIRFLOW_HOME:=${HOME}}

Expand Down