Add docker runtime support.

[asford]

Adds passthrough support for the docker run --runtime parameter, allowing pipeline-level specification of the target docker runtime. This is primarily intended to provide support for nvidia-docker in contexts where the default docker runtime can not, or should not, be reconfigured, allowing the use of CUDA devices within pipeline steps. A minimal example configuration is given at uw-ipd/buildkite-cuda.

• Expand --runtime documentation.
• Provide tutorial for basic configuration of cuda-enabled docker-based buildkite agent?

[toolmantim]

Thanks @asford! Looks like a great addition, and also the buildkite-cuda example is great. There's definitely room for a tutorial, or guest blog post, about running cuda jobs too.

[asford]

Sounds great. Do you think the current documentation for this plugin needs any expansion?

[toolmantim]

Not really! Perhaps just linking to the docs with a title, rather than inlining the entire link? And then a test (hopefully it’s mostly copy and paste?). Then it should be good to merge!

We don’t assume we’re teaching you much about Docker itself here. So linking to the Docker docs is enough I think.

Let me know if you need a help with the test or doc update, happy to help!

[asford]

Great, I've taken care of the test. I inlined the documentation link to match the current format, perhaps you can take on the documentation reformatting and/or split to a follow-up PR if needed?

I'm happy to expand the buildkite-cuda example into a more full-fledged example and/or blob post. There are, obviously, a few different ways to configure the system around nvidia-docker and it might be nice to provide a single reference covering this plugin as well as the docker-compose plugin. What's a good communication avenue to follow up with you? The buildkite-community slack?

docker-compose run tests
 ✓ Run command
 ✓ Run command without a workdir should not fail
 ✓ Pull image first before running the command with mount-buildkite-agent disabled
 ✓ Runs the command with mount-buildkite-agent disabled
 ✓ Runs command with volumes
 ✓ Runs command with environment
 ✓ Runs command with user
 ✓ Runs command with additional groups
 ✓ Runs command with network that doesn't exist
 ✓ Runs with debug mode
 ✓ Runs with custom runtime

11 tests, 0 failures

[toolmantim]

Awesome, thanks so much @asford! This has been released in v1.2.0.

A guest blog post is probably the way to go? Only because the tools might get out of date, and if it were an official tutorial we might not have the ability to keep it up-to-date. But if we publish it on https://building.buildkite.com (similar to https://building.buildkite.com/continuous-delivery-with-maven-buildkite-and-packagecloud-fc8c86f1e9b1) it could still help plenty of people? Want send me an email (tim@)? And I can invite you as a contributor to the Medium publication.