Can't install airflow helm chart using the default image hosted on ecr

[Gaabe]

Official Helm Chart version

1.9.0 (latest released)

Apache Airflow version

2.5.3

Kubernetes Version

v1.25.6-eks-48e63af

Helm Chart configuration

executor: "KubernetesExecutor"

enableBuiltInSecretEnvVars:
AIRFLOW__CELERY__CELERY_RESULT_BACKEND: false
AIRFLOW__CELERY__RESULT_BACKEND: false
AIRFLOW__CELERY__BROKER_URL: false
AIRFLOW__ELASTICSEARCH__HOST: false
AIRFLOW__ELASTICSEARCH__ELASTICSEARCH_HOST: false

data:
metadataConnection:
user: redacted-user
pass: redacted-pass
protocol: postgresql
host: redacted-postgres-host.us-east-1.rds.amazonaws.com
port: 5432
db: airflow_db_2
sslmode: disable

postgresql:
enabled: false

redis:
enabled: false

images:
airflow:
repository: 0000000000.dkr.ecr.us-east-1.amazonaws.com/ctv-de
tag: airflow-test
pullPolicy: Always

registry:
secretName: ecr-credentials

Docker Image customizations

I'm simply trying to run with the default image, tried two different approaches:

first creating a new image with no modifications.
Dockerfile:

FROM apache/airflow:2.5.3-python3.8

Build cmd:
docker build . -t airflow-test

Which didn't work

Also tried just hosting the unmodified airflow image, running the commands below:
docker pull apache/airflow
docker tag apache/airflow:latest 0000000000.dkr.ecr.us-east-1.amazonaws.com/ctv-de:airflow-test
docker push 0000000000.dkr.ecr.us-east-1.amazonaws.com/ctv-de:airflow-test

Which failed the same way.

What happened

The chart failed to install, there isn't anything useful in the logs or in the k8s events:

helm install first-test-airflow apache-airflow/airflow --namespace gabe-test-airflow -f values.yaml

migrations pod log:
exec /usr/bin/dumb-init: exec format error │ │ Stream closed EOF for gabe-test-airflow/first-test-airflow-run-airflow-migrations-54vhx (run-airflow-migrations)

migrations pod events:
Normal Pulling 8s (x4 over 48s) kubelet Pulling image "825357016628.dkr.ecr.us-east-1.amazonaws.com/ctv-de:airflow-test" │ │ Normal Created 8s (x4 over 48s) kubelet Created container run-airflow-migrations │ │ Normal Started 8s (x4 over 48s) kubelet Started container run-airflow-migrations │ │ Normal Pulled 8s kubelet Successfully pulled image "825357016628.dkr.ecr.us-east-1.amazonaws.com/ctv-de:airflow-test" in 125.701036ms (125.7 │ │ 1102ms including waiting) │ │ Warning BackOff 7s (x5 over 47s) kubelet Back-off restarting failed container

The rest of the pods have the following in the logs:
│ Stream closed EOF for gabe-test-airflow/first-test-airflow-scheduler-75b645df5c-p4lnq (wait-for-airflow-migrations)

What you think should happen instead

Since there are no modifications to the image, only it being hosted elsewhere, and from the logs it pulls it successfully, i believe it should install the chart without any problems.

How to reproduce

Host the airflow image in ecr, change values.yaml to fetch the image from the private repo, try to install the chart.

Anything else

No logs or events have any meaningful errors messages to me, I can install the chart just fine if i don't use a custom image.

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[boring-cyborg[bot]]

Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.

[potiuk]

exec /usr/bin/dumb-init: exec format error - this means that you have wrong architecture of your image. You likely built it using your ARM mac and you want to run it using AMD. this is not going to work. You need to make sure that the architecture you build your image is the same you run it on.

[Gaabe]

That was it, i'm building on the m1 mac and by default it was pulling the arm image. I was able to solve by specifying the platform to pull the base image in the dockerfile. Thank you

[potiuk]

Deleted the comments that could be misleading - please in the future avoid creating issues where you have troubleshooting of your problems. You are clearly asked to use discussions instead. In the meantime, fix the way you buld your image.

[kobethuwis]

Like @potiuk mentioned, you are trying to force a docker image with a different architecture on your machine. Are you hosting your EC2's with AWS EKS? You might be using a Graviton enabled machine (r6g, m5g, ...) with ARM architecture as nodes. In that case you'll have to use the ARM docker images as well.