Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
Combine offers to schedule tasks more efficiently #1561
This updates the scheduler to group offers by host and use the total resources available on all offers when scheduling tasks. Especially when using the offer cache, it has been common for us to have 10+ offers in the cache at a time from a single host. For tasks that require larger amounts of resources, this should get them scheduled much more quickly.
As far as I can tell, the mesos master will add up all resources on it's side when a task or tasks are scheduled using multiple offers. So, this code is not currently making an effort to match each task with a portion of the resources available, it is launching all tasks with all offers for the host. It could be a future improvement to save chunks of unused offers so the offer cache could hold on to them for the next run.