Join GitHub today
GitHub is home to over 40 million developers working together to host and review code, manage projects, and build software together.Sign up
K8s EKS Scheduler possibly not respecting node score for nodeAffinity & nodeAnti-Affinity #88112
number of stateful apps that we'd PREFER that they run on on-demand, example stateful app could be elasticsearch where the Affinity is as:
Running 6 on-demand and 6 spot in my EKS cluster
If we reduce the number of on-demand from 6 to 3, we're left with 3 on-demand and 6 spot, which is what we want
But my problem is that some of my stateful apps end up on spot, which is NOT what we want.
What you expected to happen:
How to reproduce it (as minimally and precisely as possible):
Anything else we need to know?:
Environment: AWS EKS
I attempted to create a priorityClass to see if it would help the scheduler treat the stateful pods differently
then picked elasticsearch.data to test this priorityClass by setting my yaml as per the helm chart data.priorityClassName:
When I run:
But once again when I test my scenario I still end up with pods binding to spot:
Copying the response I posted on slack:
The default scheduler is configured to prefer nodes with lower utilization, which I think is conflicting with the preferred affinity you are setting. You can fix this by configuring the scheduler to give a higher weight to pod/node affinity priorities.
In general, I think that all priorities that act on pod-specific configuration (affinities and the new spread) should by default be configured with a higher weight than the rest.