Skip to content

Cleanup CI after Airflow 2.10.4 release #45014

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
kaxil merged 1 commit into from
Dec 18, 2024
Merged

Cleanup CI after Airflow 2.10.4 release #45014

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 0 additions & 16 deletions Dockerfile.ci
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -993,22 +993,6 @@ function determine_airflow_to_use() {
--constraint https://raw.githubusercontent.com/apache/airflow/constraints-main/constraints-${PYTHON_MAJOR_MINOR_VERSION}.txt
# Some packages might leave legacy typing module which causes test issues
pip uninstall -y typing || true
# We need to install `eval-type-backport` to avoid problems with Pydantic 2.10.+ released in
# November 2024 for python 3.8 and 3.9. While Pydantic 2.10.0/2.10.1 completely broke Airflow 2
# installation and Pydantic 2.10.2 fixed the issue for past versions of Airflow, there are still
# Some Typing constructs that are not handled well by Pydantic and in case Pydantic fails with
# those errors, it will STILL fall back to `eval-type-backport` to handle those cases (if
# if `eval-type-backport` is installed. Therefore - until we have Airflow 2.10.3 for backwards
# compatibility tests and we attempt to install "edge" provider that might use such breaking
# constructs, we need to install `eval-type-backport` to avoid problems with Pydantic 2.10.2+
# as well. As soon as we move to Airflow 2.10.4, we can remove this workaround because Airflow
# 2.10.4 adds "eval-type-backport" as a dependency and it will be installed automatically.
if [[ ${PYTHON_MAJOR_MINOR_VERSION} == "3.8" || ${PYTHON_MAJOR_MINOR_VERSION} == "3.9" ]]; then
echo
echo "${COLOR_BLUE}Installing eval-type-backport for Python ${PYTHON_MAJOR_MINOR_VERSION} to workaround Pydantic 2.10.0/2.10.1 issue with new typing style.${COLOR_RESET}"
echo
pip install eval-type-backport>=0.2.0
fi
if [[ ${LINK_PROVIDERS_TO_AIRFLOW_PACKAGE=} == "true" ]]; then
echo
echo "${COLOR_BLUE}Linking providers to airflow package as we are using them from mounted sources.${COLOR_RESET}"
Expand Down
3 changes: 2 additions & 1 deletion dev/breeze/src/airflow_breeze/global_constants.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -421,6 +421,7 @@ def get_default_platform_machine() -> str:
"2.10.1": ["3.8", "3.9", "3.10", "3.11", "3.12"],
"2.10.2": ["3.8", "3.9", "3.10", "3.11", "3.12"],
"2.10.3": ["3.8", "3.9", "3.10", "3.11", "3.12"],
"2.10.4": ["3.8", "3.9", "3.10", "3.11", "3.12"],
}

DB_RESET = False
Expand Down Expand Up @@ -619,7 +620,7 @@ def get_airflow_extras():
},
{
"python-version": "3.9",
"airflow-version": "2.10.3",
"airflow-version": "2.10.4",
"remove-providers": "cloudant fab",
"run-tests": "true",
},
Expand Down
16 changes: 0 additions & 16 deletions scripts/docker/entrypoint_ci.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -226,22 +226,6 @@ function determine_airflow_to_use() {
--constraint https://raw.githubusercontent.com/apache/airflow/constraints-main/constraints-${PYTHON_MAJOR_MINOR_VERSION}.txt
# Some packages might leave legacy typing module which causes test issues
pip uninstall -y typing || true
# We need to install `eval-type-backport` to avoid problems with Pydantic 2.10.+ released in
# November 2024 for python 3.8 and 3.9. While Pydantic 2.10.0/2.10.1 completely broke Airflow 2
# installation and Pydantic 2.10.2 fixed the issue for past versions of Airflow, there are still
# Some Typing constructs that are not handled well by Pydantic and in case Pydantic fails with
# those errors, it will STILL fall back to `eval-type-backport` to handle those cases (if
# if `eval-type-backport` is installed. Therefore - until we have Airflow 2.10.3 for backwards
# compatibility tests and we attempt to install "edge" provider that might use such breaking
# constructs, we need to install `eval-type-backport` to avoid problems with Pydantic 2.10.2+
# as well. As soon as we move to Airflow 2.10.4, we can remove this workaround because Airflow
# 2.10.4 adds "eval-type-backport" as a dependency and it will be installed automatically.
if [[ ${PYTHON_MAJOR_MINOR_VERSION} == "3.8" || ${PYTHON_MAJOR_MINOR_VERSION} == "3.9" ]]; then
echo
echo "${COLOR_BLUE}Installing eval-type-backport for Python ${PYTHON_MAJOR_MINOR_VERSION} to workaround Pydantic 2.10.0/2.10.1 issue with new typing style.${COLOR_RESET}"
echo
pip install eval-type-backport>=0.2.0
fi
if [[ ${LINK_PROVIDERS_TO_AIRFLOW_PACKAGE=} == "true" ]]; then
echo
echo "${COLOR_BLUE}Linking providers to airflow package as we are using them from mounted sources.${COLOR_RESET}"
Expand Down