Dagster Running with Celery on Kubernetes, define_asset_job is not respecting dagster-k8s/config.pod_spec_config.tolerations

[henrytseng]

Dagster version

1.7.15

What's the issue?

Using celery_k8s_job_executor

Running Dagster Running with Celery on Kubernetes using the Helm chart and using define_asset_job is not respecting tag defined in

define_asset_job(
    tags={
        "dagster-k8s/config": {
            "pod_spec_config": {
                "node_selector": {
                    "nodegroup": "memory-workers",
                },
                "tolerations": [
                    {
                        "key": "asdf/memory-workers",
                        "effect": "NoSchedule",
                    },
                    {
                        "key": "ops",
                        "value": "only",
                    },
                ],
            },
        },
        ...

Using the celery_k8s_job_executor with Definitions configured. The Job seems to run but without the Kubernetes tolerations.

What did you expect to happen?

Expecting tolerations to be added to job/pod manifest(s) or validation messages but no luck ATM.

How to reproduce?

No response

Deployment type

Dagster Helm chart

Deployment details

Dagster Running with Celery on Kubernetes

Additional information

No response

Message from the maintainers

Impacted by this issue? Give it a 👍! We factor engagement into prioritization.