gRPC server crashes when running an amd container on arm (mac m1)

[ThomasRolfsnes-EP]

Dagster version

1.6.2

What's the issue?

I'm developing an extract against a db2-database, which requires libraries that only support x86 at the moment, while my machine is an m1 mac.

I've therefore set platform: linux/amd64 in the docker compose config. This all works, I'm able to build the docker image and work with the database.

However, I get the following code location error in the dagster UI:

Exception: gRPC server exited with return code -4 while starting up with the command: "/root/.cache/pypoetry/virtualenvs/she-dagster-VA82Wl8V-py3.10/bin/python -m dagster api grpc --lazy-load-user-code --socket /tmp/tmpmt76toxl --heartbeat --heartbeat-timeout 45 --fixed-server-id 48124e83-f7d0-45e2-b820-4f6dab49bef0 --log-level info --inject-env-vars-from-instance --instance-ref {"__class__": "InstanceRef", "compute_logs_data": {"__class__": "ConfigurableClassData", "class_name": "LocalComputeLogManager", "config_yaml": "base_dir: /opt/dagster/dagster_home/storage\n", "module_name": "dagster.core.storage.local_compute_log_manager"}, "custom_instance_class_data": null, "event_storage_data": {"__class__": "ConfigurableClassData", "class_name": "SqliteEventLogStorage", "config_yaml": "base_dir: /opt/dagster/dagster_home/history/runs/\n", "module_name": "dagster.core.storage.event_log"}, "local_artifact_storage_data": {"__class__": "ConfigurableClassData", "class_name": "LocalArtifactStorage", "config_yaml": "base_dir: /opt/dagster/dagster_home/\n", "module_name": "dagster.core.storage.root"}, "run_coordinator_data": {"__class__": "ConfigurableClassData", "class_name": "QueuedRunCoordinator", "config_yaml": "max_concurrent_runs: 25\ntag_concurrency_limits:\n- key: dagster/backfill\n  limit: 10\n", "module_name": "dagster.core.run_coordinator"}, "run_launcher_data": {"__class__": "ConfigurableClassData", "class_name": "DefaultRunLauncher", "config_yaml": "{}\n", "module_name": "dagster"}, "run_storage_data": {"__class__": "ConfigurableClassData", "class_name": "SqliteRunStorage", "config_yaml": "base_dir: /opt/dagster/dagster_home/history/\n", "module_name": "dagster.core.storage.runs"}, "schedule_storage_data": {"__class__": "ConfigurableClassData", "class_name": "SqliteScheduleStorage", "config_yaml": "base_dir: /opt/dagster/dagster_home/schedules\n", "module_name": "dagster.core.storage.schedules"}, "scheduler_data": {"__class__": "ConfigurableClassData", "class_name": "DagsterDaemonScheduler", "config_yaml": "{}\n", "module_name": "dagster.core.scheduler"}, "secrets_loader_data": null, "settings": {}, "storage_data": {"__class__": "ConfigurableClassData", "class_name": "DagsterSqliteStorage", "config_yaml": "base_dir: /opt/dagster/dagster_home/\n", "module_name": "dagster.core.storage.sqlite_storage"}} --location-name she_dagster.repos --package-name she_dagster.repos"

What did you expect to happen?

Dagster beeing able to load the code-location without issues

How to reproduce?

Run dagster in a docker container on an arm-machine, setting platform to amd64

Deployment type

Dagster Cloud

Deployment details

Docker version 25.0.3, build 4debf41
Docker Compose version v2.24.5-desktop.1

Additional information

No response

Message from the maintainers

Impacted by this issue? Give it a 👍! We factor engagement into prioritization.

[ThomasRolfsnes-EP]

This was a python package version problem, unrelated to dagster.
I.e., an older python package which had not been updated to work on amd64 in and arm64 environement.