Skip to content

Separate nodeSelector logic for scheduler+webserver vs ephemeral resources #10761

@aimran-adroll

Description

@aimran-adroll

Description
I would like to separate out the current nodeSelector logic for critical (imo) pods such as the scheduler+webserver vs worker pods.

Use case / motivation

If we look at the current implementation,

# Select certain nodes for airflow pods.
nodeSelector: {}
affinity: {}
tolerations: []

we have only one place to specify the logic above

Subsequently, the above template is applied as is across all the resources (e.g scheduler, webserver, worker, ...)

Typically, I work with an autoscaling group that consists of few stable (ie on-demand) instances along with the more ephemeral spot (preemptible) instances.

During deployment, I would like to assign the critical (TBD but at least scheduler and webserver) pods to the on-demand instances.

Related Issues

grepping for nodeSelector did not turn up anything here or the astronomer chart repo.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions