Status of testing of Apache Airflow 2.3.1rc1

[ephraimbuddy]

Body

We have a kind request for all the contributors to the latest Apache Airflow RC 2.3.1rc1.

Could you please help us to test the RC versions of Airflow?

Please let us know in the comment if the issue is addressed in the latest RC.

• Add tags inside try block. (#21784): @jhtimmins
• Fix secrets rendered in UI when task is not executed. (#22754): @m1racoli @tirkarthi
  Linked issues:
  • Webserver doesn't mask rendered fields for pending tasks (#22738)
• Fix Graph overflow, position and padding (#23044): @bbovenzi @eladkal
  Linked issues:
  • Graph view: Nodes arrows are cut (#23042)
• Use kubernetes queue in kubernetes hybrid executors (#23048): @tanelk
• Show warning if '/' is used in a DAG run ID (#23106): @davidavdav @uranusjr
  Linked issues:
  • Forward slash in dag_run_id gives rise to trouble accessing things through the REST API (#20063)
• Pools with negative open slots should not block other pools (#23143): @tanelk
• Handle invalid date parsing in webserver views. (#23161): @tirkarthi
  Linked issues:
  • Invalid execution_date crashes pages accepting the query parameter (#22969)
• Refactor code references from tree to grid (#23254): @bbovenzi
• Override pool for TaskInstance when pool is passed from cli. (#23258): @tirkarthi @Swalloow
  Linked issues:
  • Pool option does not work in backfill command (#23249)
• Remove custom signal handling in Triggerer (#23274): @dstandish @potiuk
  Linked issues:
  • Airflow standalone command does not exit gracefully (#19260)
• Store grid view selection in url params (#23290): @bbovenzi
• Fix: Exception when parsing log #20966 (#23301): @microhuang @jakubno
  Linked issues:
  • Fix: Exception when parsing log #20966 (#21053)
• Fix duplicated Kubernetes DeprecationWarnings (#23302): @jedcunningham
• Fix empty image preparation (#23304): @potiuk
• Fix Mapped Instance start/end date timezones (#23313): @bbovenzi
• Fix update user auth stats (#23314): @blag
• Use run_id for ti.mark_success_url (#23330): @bbovenzi
• Don't show grid actions if server would reject with permission denied (#23332): @bbovenzi
• Hide some task instance attributes in details page (#23338): @bbovenzi
• fix cli airflow show dags for mapped operator (#23339): @MatrixManAtYrService @RNHTTR
  Linked issues:
  • airflow dags show Exception: "The node ... should be TaskGroup and is not" (#23315)
• Fix connection test button (#23345): @jedcunningham
• Fix broken task instance link in xcom list (#23367): @bbovenzi
• Grid view status filters (#23392): @pierrejeambrun
• [FIX] remove python 3.6 (#23409): @raphaelauv
• Fix literal cross product expansion (#23434): @aspain @jedcunningham
  Linked issues:
  • Mapping over multiple parameters results in 1 task fewer than expected (#23425)
• Add backward compatibility for core__sql_alchemy_conn__cmd (#23441): @c-thiel
• Optimize 2.3.0 pre-upgrade check queries (#23458): @dstandish
• Fix broken dagrun links when many runs start at the same time (#23462): @repl-chris
• Remove color change for highly nested groups (#23482): @bbovenzi
• Fix scheduler crash when expanding with mapped task that returned none (#23486): @ephraimbuddy
• Expand/collapse all groups (#23487): @bbovenzi
• Visually distinguish task group summaries (#23488): @bbovenzi
• Only count bad refs when moved table exists (#23491): @dstandish
• Added postgres 14 to support versions(including breeze) (#23506): @alexInhert @subkanthi
  Linked issues:
  • Add Postgres 14 support (#23460)
• Prevent KubernetesJobWatcher getting stuck on resource too old (#23521): @SimonOsipov @ecerulm @ephraimbuddy @guillemborrell @cansjt
  Linked issues:
  • Network instabilities are able to freeze KubernetesJobWatcher (#12644)
  • Handle kubernetes watcher stream disconnection (#15500)
  • KubernetesExecutor do not spawn Kubernetes Pod Operator pods. (#17629)
  • KubernetesJobWatcher failing on HTTP 410 errors, jobs stuck in scheduled state (#21087)
• Change approach to finding bad rows to LEFT OUTER JOIN. (#23528): @ashb
• Fix regression in ignoring symlinks (#23535): @ianbuss @NickYadance
  Linked issues:
  • Airflow .airflowignore not handling soft link properly. (#23532)
• Apply specific ID collation to root_dag_id too (#23536): @mpeteuil
• Fix PythonVirtualenvOperator templated_fields (#23559): @eladkal
• Fix dag-processor fetch metabase config (#23575): @allebacco @Taragolis
  Linked issues:
  • cli command dag-processor uses [core] sql_alchemy_conn (#23572)
  • External DAG processor not working (#23447)
• Fix expand/collapse all buttons (#23590): @bbovenzi
• Implement send_callback method for CeleryKubernetesExecutor and LocalKubernetesExecutor (#23617): @eduardchai @mhenc
  Linked issues:
  • AttributeError: 'CeleryKubernetesExecutor' object has no attribute 'send_callback' (#23610)
• Add index for event column in log table (#23625): @gmsantos
• Don't run pre-migration checks for downgrade (#23634): @dstandish
• [FEATURE] update K8S-KIND to 0.13.0 (#23636): @raphaelauv
• [FEATURE] add k8s 1.24 support (#23637): @raphaelauv
• Ensure execution_timeout as timedelta (#23655): @pingzh
• Add 'reschedule' to the serialized fields for the BaseSensorOperator (#23674): @pmrem @davidcaron
  Linked issues:
  • PythonSensor is not considering mode='reschedule', instead marking task UP_FOR_RETRY (#23411)
• Fix task log is not captured (#23684): @pingzh
• Disable Flower by default from docker-compose (#23685): @potiuk
• Automatically reschedule stalled queued tasks in CeleryExecutor (v2) (#23690): @repl-chris
• Modify db clean to also catch the ProgrammingError exception (#23699): @jianyuan
  Linked issues:
  • airflow db clean - table missing exception not captured (#23698)
• Add typing for airflow/configuration.py (#23716): @potiuk
• Fix retrieval of deprecated non-config values (#23723): @potiuk
• Fix grid details header text overlap (#23728): @bbovenzi
• Remove titles from link buttons (#23736): @bbovenzi
• Fix auto upstream dep when expanding non-templated field (#23771): @jedcunningham
• Fix provider import error matching (#23825): @ianbuss
  Linked issues:
  • ModuleNotFoundExceptions not matched as optional features (#23823)
• Fix constraint reference params (#23845): @potiuk
• Add limit for JPype1 (#23847): @potiuk

Thanks to all who contributed to the release (probably not a complete list!):
@pierrejeambrun @mhenc @gmsantos @Swalloow @potiuk @eduardchai @mpeteuil @allebacco @ianbuss @guillemborrell @SimonOsipov @tirkarthi @RNHTTR @alexInhert @davidavdav @blag @NickYadance @c-thiel @Taragolis @ecerulm @jedcunningham @ephraimbuddy @jakubno @MatrixManAtYrService @dstandish @eladkal @pmrem @repl-chris @davidcaron @ashb @aspain @microhuang @m1racoli @uranusjr @pingzh @bbovenzi @tanelk @jhtimmins @subkanthi @jianyuan @cansjt @raphaelauv

Committer

• I acknowledge that I am a maintainer/committer of the Apache Airflow project.

[raphaelauv]

maybe not related to 2.3.1rc1 astronomer/astronomer-providers#372

[pierrejeambrun]

Hello Guys,

#23392 is working fine 👍

[gmsantos]

I see that the apache-airflow-providers-cncf-kubernetes was downgraded to 3.1.2 in 2.3.1rc1 constraints file (it was 4.0.1 in 2.3.0 constraints).

Is it not good yet to go 4.0 for this provider?

[potiuk]

It should not be the case. Looking at it.

[potiuk]

I fixed it in the constraints (and rebuilding the images).

@raphaelauv -> the astronomer/astronomer-providers#372 could be related if you installed rc1 with the constraints containing 3.1.2.

I will look in detail why the downgrade happened. It could be related to the rc* behaviour of pip (previously we bumped it manually to 4.0 because of this but it should not be the case). I double checked all other constraints and it seems that it has only happened for cncf.kubernetes.

[potiuk]

Thanks for spotting it @gmsantos !

[potiuk]

Yeah. 3.7 image already pushed and has the right cncf.kubernetes provider:

[jarek@Hyperion:~] 3s 2 % docker run -it apache/airflow:2.3.1rc1-python3.7 providers list

package_name                             | description                                                                                  | version
=========================================+==============================================================================================+========
apache-airflow-providers-amazon          | Amazon integration (including Amazon Web Services (AWS) https://aws.amazon.com/)             | 3.4.0
apache-airflow-providers-celery          | Celery http://www.celeryproject.org/                                                         | 2.1.4
apache-airflow-providers-cncf-kubernetes | Kubernetes https://kubernetes.io/                                                            | 4.0.2
apache-airflow-providers-docker          | Docker https://docs.docker.com/install/                                                      | 2.7.0
apache-airflow-providers-elasticsearch   | Elasticsearch https://www.elastic.co/elasticsearch                                           | 3.0.3
apache-airflow-providers-ftp             | File Transfer Protocol (FTP) https://tools.ietf.org/html/rfc114                              | 2.1.2
apache-airflow-providers-google          | Google services including:                                                                   | 7.0.0
                                         |                                                                                              |
                                         |   - Google Ads https://ads.google.com/                                                       |
                                         |   - Google Cloud (GCP) https://cloud.google.com/                                             |
                                         |   - Google Firebase https://firebase.google.com/                                             |
                                         |   - Google LevelDB https://github.com/google/leveldb/                                        |
                                         |   - Google Marketing Platform https://marketingplatform.google.com/                          |
                                         |   - Google Workspace https://workspace.google.pl/ (formerly Google Suite)                    |
apache-airflow-providers-grpc            | gRPC https://grpc.io/                                                                        | 2.0.4
apache-airflow-providers-hashicorp       | Hashicorp including Hashicorp Vault https://www.vaultproject.io/                             | 2.2.0
apache-airflow-providers-http            | Hypertext Transfer Protocol (HTTP) https://www.w3.org/Protocols/                             | 2.1.2
apache-airflow-providers-imap            | Internet Message Access Protocol (IMAP) https://tools.ietf.org/html/rfc3501                  | 2.2.3
apache-airflow-providers-microsoft-azure | Microsoft Azure https://azure.microsoft.com/                                                 | 3.9.0
apache-airflow-providers-mysql           | MySQL https://www.mysql.com/products/                                                        | 2.2.3
apache-airflow-providers-odbc            | ODBC https://github.com/mkleehammer/pyodbc/wiki                                              | 2.0.4
apache-airflow-providers-postgres        | PostgreSQL https://www.postgresql.org/                                                       | 4.1.0
apache-airflow-providers-redis           | Redis https://redis.io/                                                                      | 2.0.4
apache-airflow-providers-sendgrid        | Sendgrid https://sendgrid.com/                                                               | 2.0.4
apache-airflow-providers-sftp            | SSH File Transfer Protocol (SFTP) https://tools.ietf.org/wg/secsh/draft-ietf-secsh-filexfer/ | 2.6.0
apache-airflow-providers-slack           | Slack https://slack.com/                                                                     | 4.2.3
apache-airflow-providers-sqlite          | SQLite https://www.sqlite.org/                                                               | 2.1.3
apache-airflow-providers-ssh             | Secure Shell (SSH) https://tools.ietf.org/html/rfc4251                                       | 2.4.4

[potiuk]

All images are refreshed - I also opened #23858 to investigate why it happened.

[potiuk]

Tested all change I've been involved in. All good :).

The most important #23723 (passing deprecated parameters to "run_as_user" - works as expected).

[2022-05-22, 22:40:11 UTC] {base_task_runner.py:141} INFO - Running on host: 01b98cdf7c4d
[2022-05-22, 22:40:11 UTC] {base_task_runner.py:142} INFO - Running: ['sudo', '-E', '-H', '-u', '***', '***', 'tasks', 'run', 'example_bash_operator', 'runme_0', 'manual__2022-05-22T22:40:11.200418+00:00', '--job-id', '44', '--raw', '--subdir', '/usr/local/lib/python3.8/site-packages/***/example_dags/example_bash_operator.py', '--cfg-path', '/tmp/tmpqmbfe35d', '--error-file', '/tmp/tmpvqn56f5f']
[2022-05-22, 22:40:11 UTC] {base_task_runner.py:123} INFO - Job 44: Subtask runme_0 /usr/local/lib/python3.8/site-packages/***/configuration.py:525: DeprecationWarning: The sql_alchemy_conn option in [core] has been moved to the sql_alchemy_conn option in [database] - the old setting has been used, but please update your config.
[2022-05-22, 22:40:11 UTC] {base_task_runner.py:123} INFO - Job 44: Subtask runme_0   option = self._get_environment_variables(deprecated_key, deprecated_section, key, section)

Killing standalone #23274 also works:

[2022-05-22, 22:40:11 UTC] {base_task_runner.py:141} INFO - Running on host: 01b98cdf7c4d
[2022-05-22, 22:40:11 UTC] {base_task_runner.py:142} INFO - Running: ['sudo', '-E', '-H', '-u', '***', '***', 'tasks', 'run', 'example_bash_operator', 'runme_0', 'manual__2022-05-22T22:40:11.200418+00:00', '--job-id', '44', '--raw', '--subdir', '/usr/local/lib/python3.8/site-packages/***/example_dags/example_bash_operator.py', '--cfg-path', '/tmp/tmpqmbfe35d', '--error-file', '/tmp/tmpvqn56f5f']
[2022-05-22, 22:40:11 UTC] {base_task_runner.py:123} INFO - Job 44: Subtask runme_0 /usr/local/lib/python3.8/site-packages/***/configuration.py:525: DeprecationWarning: The sql_alchemy_conn option in [core] has been moved to the sql_alchemy_conn option in [database] - the old setting has been used, but please update your config.
[2022-05-22, 22:40:11 UTC] {base_task_runner.py:123} INFO - Job 44: Subtask runme_0   option = self._get_environment_variables(deprecated_key, deprecated_section, key, section)

[cansjt]

I am sorry, but I am alone this week (my colleagues are off), so it's doubtful I will find the time to check.

[gmsantos]

For #23590 (related to #23489 and #23580), tasks with . are shown again in grid view.

I checked that #23521 also fixes the scheduler error loop too.

[gmsantos]

#23625: working fine as well

[Taragolis]

#23575 working fine

[repl-chris]

#23462 and #23690 are working fine

And FWIW, they're not mine but I have also verified #23345 (@jedcunningham) and #23535 (@ianbuss @NickYadance) are also working wonderfully

[RNHTTR]

#23339 looks good 😄

[ephraimbuddy]

Thank you all for testing this release!

[eduardchai]

For #23590 (related to #23489 and #23580), tasks with . are shown again in grid view.

I checked that #23521 also fixes the scheduler error loop too.

The tasks with . appears only on newly created dags. If you are modifying an existing DAG (e.g. task_id from run_0 to run.0, adding a task to an existing DAG, etc) it will not work and you need to delete the DAG first. Modifying DAG task_id may not be a common practice, so this info may be useful for people who do.

[eduardchai]

I have confirmed that #23617 has been fixed 🎉 . Thank you!

[m1racoli]

#22754 has been fixed too. 🥳 Thanks a lot!