chore: Bump Celery to 5.5.3 for Airflow 3.x

[xeniape]

Description

Implementation for stackabletech/airflow-operator#676

Airflow 3.x built locally and ran airflow integrationtests with them:

--- FAIL: kuttl (4311.80s)
    --- FAIL: kuttl/harness (0.00s)
        --- PASS: kuttl/harness/smoke_airflow-3.0.1_openshift-false_executor-celery (256.60s)
        --- PASS: kuttl/harness/logging_airflow-3.0.1_openshift-false_executor-celery (334.39s)
        --- PASS: kuttl/harness/external-access_airflow-3.0.1_openshift-false_executor-kubernetes (199.20s)
        --- PASS: kuttl/harness/mount-dags-gitsync_airflow-latest-3.0.6_openshift-false_executor-kubernetes (193.47s)
        --- PASS: kuttl/harness/orphaned-resources_airflow-latest-3.0.6_openshift-false (211.67s)
        --- PASS: kuttl/harness/mount-dags-gitsync_airflow-latest-3.0.6_openshift-false_executor-celery (342.22s)
        --- PASS: kuttl/harness/remote-logging_airflow-latest-3.0.6_openshift-false_executor-kubernetes (257.22s)
        --- PASS: kuttl/harness/remote-logging_airflow-latest-3.0.6_openshift-false_executor-celery (274.78s)
        --- PASS: kuttl/harness/external-access_airflow-3.0.6_openshift-false_executor-kubernetes (137.54s)
        --- PASS: kuttl/harness/overrides_airflow-latest-3.0.6_openshift-false (222.64s)
        --- PASS: kuttl/harness/external-access_airflow-3.0.6_openshift-false_executor-celery (435.33s)
        --- PASS: kuttl/harness/external-access_airflow-3.0.1_openshift-false_executor-celery (200.81s)
        --- PASS: kuttl/harness/resources_airflow-latest-3.0.6_openshift-false (223.76s)
        --- PASS: kuttl/harness/oidc_airflow-latest-3.0.6_openshift-false (214.42s)
        --- PASS: kuttl/harness/logging_airflow-3.0.6_openshift-false_executor-celery (331.96s)
        --- PASS: kuttl/harness/ldap_airflow-latest-3.0.6_ldap-authentication-insecure-tls_openshift-false_executor-celery (376.26s)
        --- PASS: kuttl/harness/triggerer_airflow-latest-3.0.6_openshift-false_executor-kubernetes (154.39s)
        --- PASS: kuttl/harness/triggerer_airflow-latest-3.0.6_openshift-false_executor-celery (204.45s)
        --- PASS: kuttl/harness/ldap_airflow-latest-3.0.6_ldap-authentication-no-tls_openshift-false_executor-celery (211.53s)
        --- FAIL: kuttl/harness/logging_airflow-3.0.6_openshift-false_executor-kubernetes (614.94s)
        --- PASS: kuttl/harness/ldap_airflow-latest-3.0.6_ldap-authentication-insecure-tls_openshift-false_executor-kubernetes (167.75s)
        --- PASS: kuttl/harness/ldap_airflow-latest-3.0.6_ldap-authentication-no-tls_openshift-false_executor-kubernetes (223.82s)
        --- PASS: kuttl/harness/mount-dags-configmap_airflow-latest-3.0.6_openshift-false_executor-kubernetes (170.12s)
        --- PASS: kuttl/harness/ldap_airflow-latest-3.0.6_ldap-authentication-server-verification-tls_openshift-false_executor-kubernetes (182.64s)
        --- PASS: kuttl/harness/opa_airflow-3.0.1_opa-latest-1.8.0_openshift-false (214.74s)
        --- PASS: kuttl/harness/opa_airflow-3.0.6_opa-latest-1.8.0_openshift-false (226.10s)
        --- PASS: kuttl/harness/smoke_airflow-3.0.6_openshift-false_executor-kubernetes (254.95s)
        --- PASS: kuttl/harness/cluster-operation_airflow-latest-3.0.6_openshift-false (263.08s)
        --- PASS: kuttl/harness/mount-dags-configmap_airflow-latest-3.0.6_openshift-false_executor-celery (177.10s)
        --- PASS: kuttl/harness/ldap_airflow-latest-3.0.6_ldap-authentication-server-verification-tls_openshift-false_executor-celery (356.73s)
        --- PASS: kuttl/harness/smoke_airflow-3.0.6_openshift-false_executor-celery (249.80s)
        --- FAIL: kuttl/harness/logging_airflow-3.0.1_openshift-false_executor-kubernetes (371.49s)
        --- PASS: kuttl/harness/smoke_airflow-3.0.1_openshift-false_executor-kubernetes (335.40s)
FAIL

Failed tests passed but timed out on namespace deletion

Definition of Done Checklist

Note

Not all of these items are applicable to all PRs, the author should update this template to only leave the boxes in that are relevant.

Please make sure all these things are done and tick the boxes

• Changes are OpenShift compatible
• All added packages (via microdnf or otherwise) have a comment on why they are added
• Things not downloaded from Red Hat repositories should be mirrored in the Stackable repository and downloaded from there
• All packages should have (if available) signatures/hashes verified
• Add an entry to the CHANGELOG.md file
• Integration tests ran successfully

TIP: Running integration tests with a new product image

The image can be built and uploaded to the kind cluster with the following commands:

boil build <IMAGE> --image-version <RELEASE_VERSION> --strip-architecture --load
kind load docker-image <MANIFEST_URI> --name=<name-of-your-test-cluster>

See the output of boil to retrieve the image manifest URI for <MANIFEST_URI>.

[sbernauer]

Thanks!
Ideally we add a more detailed changelog entry to the operator as well.

[sbernauer]

Thanks!

[xeniape]

Release Note

Fixed Celery on Redis problems in combination with Airflow 3. We achieved this by bumping the celery Python package. Before the fix, workers could run into the problem of not executing tasks anymore after a Redis reconnection.