Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

#764 Add GPU support via --enable-features #810

Open
wants to merge 3 commits into
base: master
from

Conversation

Projects
None yet
4 participants
@reneploetz
Copy link

commented Feb 26, 2017

This patch adds GPU scheduling support to Chronos the same way Marathon does it:

  • you need to specify "--enable-features=gpu_resources" as parameter to enable it
  • you can specify "--mesos_task_gpu=X" where X is an Integer to always allocate X GPUs for new tasks
  • the REST API was extended to allow an optional field "gpus" alongside the usual "cpus/disk/mem" arguments to allow per-task GPU settings

The argument parsing code was adapted from Marathon (Features.scala / SchedulerConfiguration.scala) to provide the same behaviour.

This should fix #764

Signed-off-by: Rene Ploetz reneploetz@gmx.de

@brndnmtthws
Copy link
Member

left a comment

I'd like to see a little more in the unit tests for this, but aside from that, it looks good 👍

@@ -36,7 +36,7 @@ class MesosTaskBuilderSpec extends SpecificationWithJUnit with Mockito {

val parameters = scala.collection.mutable.ListBuffer[Parameter]()

val container = Container("dockerImage", ContainerType.DOCKER, volumes, parameters, NetworkMode.HOST, None, networks, forcePullImage = true)

This comment has been minimized.

Copy link
@brndnmtthws

brndnmtthws Mar 5, 2017

Member

Please add a new test instead of changing this one.

This comment has been minimized.

Copy link
@reneploetz

reneploetz Mar 6, 2017

Author

I'll change the type back to DOCKER.
Do you want some tests for the MesosTaskBuilderSpec, e.g. whether a non-zero value would be applied? And do you want the commits squashed?

This comment has been minimized.

Copy link
@gengmao

gengmao Mar 7, 2017

@reneploetz According to http://mesos.apache.org/documentation/latest/gpu-support/, seems DOCKER type is not the right one

At the time of this writing, Nvidia GPU support is only available for tasks launched through the Mesos containerizer (i.e. no support exists for launching GPU capable tasks through the Docker containerizer)

@reneploetz reneploetz force-pushed the reneploetz:master branch from e9362af to 9fa6e23 Mar 6, 2017

reneploetz and others added some commits Feb 26, 2017

add GPU support via --enable-features
Signed-off-by: Rene Ploetz <reneploetz@gmx.de>

@reneploetz reneploetz force-pushed the reneploetz:master branch from 9fa6e23 to 86779f2 Mar 6, 2017

@brndnmtthws

This comment has been minimized.

Copy link
Member

commented Mar 8, 2017

@reneploetz let me clarify: I was suggesting you add an additional (new) test with the GPUs enabled, and leave the existing test unchanged.

@mindscratch

This comment has been minimized.

Copy link
Contributor

commented Apr 23, 2018

What's the status of this PR? Right now, mesos-agents that have GPUs won't be offered to Chronos, which is problematic when all the agents have GPUs.

@reneploetz

This comment has been minimized.

Copy link
Author

commented Apr 26, 2018

I'm afraid I never had time to create a pull request with proper tests as per review - feel free to do so. We were using Chronos with this patch in some product, but switched to another one a while ago.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.