-
Notifications
You must be signed in to change notification settings - Fork 39.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
More general scheduling constraints #367
Comments
I have noticed the FirstFit (default?) scheduler co-locates pods when there are open machines available. Each of these machines has a single cpu. It would be nice to use information about available cpu and a pod's expected cpu requirements sed 's/cpu/other_machine_stat/' |
Currently, kubelet could get stats from cAdvisor which would be useful for scheduler. It could provide different percentiles of CPU and memory usage of a container (including the root container, i.e. the machine). |
That's just "scheduling", as opposed to machine constraints, though very On Mon, Jul 7, 2014 at 1:32 PM, Tony Worm notifications@github.com wrote:
|
Labels "seems" like the ideal place to enable a rank & requirements to define constraints. However labels would need to be regularly published by minions. e.g. I have a couple of concerns here:
|
Let's worry about semantics before syntax. We have a similar issue open On Mon, Jul 7, 2014 at 1:42 PM, Timothy St. Clair notifications@github.com
|
FWIW I often view constraints as a SQL query on a nvp store. SELECT Resources The hardest part are the 'fields' in an nvp store. |
I'm ok with doing the selection from a set of offers/resources from the scheduler. Provided the offers have enough NVP information to enable discrimination. |
I don't know about NVP - where can I read more on it?
|
Searching for "NVP SQL" or "name value pair SQL" or "key value pair SQL" comes up with lots of hits. Common arguments against are performance and loss of control over DB schema. But I'm getting the feeling that we're barking up the wrong forest. @timothysc What are you trying to do? Right now, k8s has essentially no intelligent scheduling. However, that's not a desirable end state. If what you want is a scheduler, we should figure out how to support scheduling plugins and/or layers on top of k8s. |
Name Value Pairs? Now I feel dumb :) On Tue, Jul 8, 2014 at 7:51 PM, Tim Hockin thockin@google.com wrote:
|
Something somewhat different than label selectors is per-attribute limits for spreading. Aurora is one system that supports this model: This is more relevant to physical rather than virtual deployments. I'd consider it a distinct mechanism from constraints. @timothysc If you'd like this, we should file a separate issue. However, I'd prefer a a new failure tolerance scheduling policy object that specifies a label selector to identify the set of instances to be spread. We could debate about how to describe what kind and/or how much spreading to apply, but I'd initially just leave it entirely up to the infrastructure. |
I completely agree its more relevant to physical rather then virtual deployments. I was somewhat testing the possibility of enabling the capabilities for more general purpose scheduling, on par with a mini-Condor approach but it's not a requirement. Aurora or Marathon -esk capabilities will fill the gap. |
Note that in order to add constraints, we'd need a way to attach labels to minions/nodes. |
That is what I had alluded to earlier, but it received luke warm attention. In fact, I believe Wilkes had chimed in on a different thread regarding this topic. |
I think we should have labels for worker nodes, but they need to be For now, I think we should use resources on nodes, since they are already You can add resource requests to pods, to achieve appropriate scheduling. Brendan
|
Replication controllers reschedule pods when the machines they are on are no longer available. Seems like replication controller could do the same if the machine becomes infeasible for scheduling. A fairly simple loop can recheck predicates as a background task in the scheduler, and move pods to terminated state if they no longer fit. Questions:
|
Yeah, having the kubelet kill pods that don't match makes the most sense. --brendan On Mon, Oct 20, 2014 at 9:39 AM, bgrant0607 notifications@github.com
|
Constraint evals to simlpe true or false expressions, so it should really be &&, || this way you can connect them including simple predicate matching. GROUPBY (cluster) || rack==funkytown The more I think about it, the less I want to tread into config language space given the cattle idiom on services. Weather they are soft or hard could either be denoted kia keyword or some other semantics. |
So IIUC the way the thing you're proposing would work would like this? GROUPBY expr1 || expr2 => put a virtual label X on all the machines that match expr1 or expr2, and then try to co-locate all the pods of the service on machines with label X It would be good to flesh out some use cases... |
I agree that flavors of affinity and anti-affinity are the basic 2 features that would satisfy most use cases. With respect to As Simple As Possible, specifying just whether to group or spread seams like the simplest possible API. That needs to be associated with some set of pods, via label selector (in which object TBD). Node groups to concentrate in or spread across could be configured in the scheduler in most cases. |
+1 I deploy some fairly hefty celery tasks in our cluster, and definitively do not ever want more than one running on the same host at the same time. I'd rather some get left unscheduled and run a monitoring task to pick them up by scaling my cluster up. |
cc/ @pwittrock |
This is part of #18261 |
@davidopp I think it reasonable to close this issue, in lieu of the assorted proposal. |
@timothysc Let's wait until we merge the proposal. |
Caches container data for 5 seconds before updating it
Baremetal cleanup
Affinity/anti-affinity proposals merged and implementations are underway. |
Baremetal cleanup
Baremetal cleanup
Baremetal cleanup
Baremetal cleanup
change packages to bases in Manifest
Bug 1877793: Force releasing the lock on exit for KS
untangle plugin runner a bit
There have been a few folks who have asked about machine constraints for scheduling. Let's use this issue as a place to gather ideas and requirements.
@timothysc
The text was updated successfully, but these errors were encountered: