[WIP] LIBCLOUD-781 docker node driver

[tonybaloney]

Port the node driver for Docker from @mgogoulos initial driver in Mist.IO

TODO:

• Port initial driver code
• Refactor with docstrings, method naming and project standards
• Setup tests and fixtures
• Update docs

[sebgoa]

We don't keep authorship information that way at ASF, so we can remove those lines.

[poliveira89]

@tonybaloney do you use it in a dev or prod environment?
If this PR is accepted, could lead to create providers for Swarm, Kubernetes and other Container implementations as well?

[poliveira89]

Just for curiosity, why do you have used the HTTP API instead their official Python package?

[tonybaloney]

Generally LibCloud does the minimal integration instead of relying on 3rd party Libs. The number of Libs would be huge and lack of control and test ability would be an issue

[poliveira89]

Makes sense! 👍

As I said, I asked for curiosity, and to understand your development choice! :)

[mgogoulos]

hi @poliveira89, we use this driver on https://github.com/mistio/mist.io to run actions on containers and also deploy new ones (passing a name, image, command, ports to expose, port bindings and also env variables)

[sebgoa]

@mgogoulos cool. Can you explain how you use the 'password' defined in the features list ? I am just not getting the workflow

[poliveira89]

@mgogoulos which branch contains the stable version of libcloud with docker support?

I have only found this one: https://github.com/DimensionDataCBUSydney/libcloud/tree/LIBCLOUD-781_Docker

[mgogoulos]

Hi @Runseb, you are right, this line can be omitted

features = {'create_node': ['password']}

The create_node() function can be used without the need of this line of course.

[Kami]

@poliveira89

@tonybaloney do you use it in a dev or prod environment?
If this PR is accepted, could lead to create providers for Swarm, Kubernetes and other Container implementations as well?

Similar to libvirt, a lot of docker functionality doesn't really nicely map to the libcloud.compute.* API which was primarily designed to handle cloud and virtual servers.

If we decide to also add support for other container run-times (docker, rkt, FreeBsD jails, Solaris zones, etc.) it would be worth evaluating if we should create a new libcloud.container API which is primarily targeted and designed for containers.

In any case, if we go with a new API it's also worth noting that that's a much bigger task and commitment.

Also as far as Swarm and Kubernetes goes - those are more of container orchestrator systems so I don't think they fit well in either compute or potential new container API.

[tonybaloney]

@poliveira89 @Kami @Runseb

I can see 3 options for us here (if we target docker and then rkt)

1. proceed with docker as a compute driver, the implementation is a bit wonky but it works
2. create a new driver type, ContainerNodeDriver, which INHERITS from NodeDriver, list_nodes would be overloaded to return ContainerNode's instead of Nodes. a ContainerNode (which inherits from Node) would share the same properties, but also have an additional property stating which Node it belongs to. There would be some other methods like pull_image, etc. That way users can use list_nodes etc. but also take advantage of the other functions.
3. create a new driver type, ContainerDriver and start from scratch, just design and implement which methods based on usage

For the record. I favor option 2.

[poliveira89]

@Kami let me start this comment, by saying that I fully understand that this "upgrade" or embracing Containers will be a much bigger task and require a lot effort from everyone involved (because as you stated some changes - design and implementation must be discussed). I just asked to inquire which was the roadmap regarding this topic - for Container-as-a-Service providers.

But, if you allow me to say this, before anyone start this undertaking, why don't start small? First discuss the acceptance of this PR... as @tonybaloney and @mgogoulos states, this works perfectly for their use-case scenario (which, I assume, is deploy instances/nodes regardless of the provider).
After, that the community and the Libcloud team could start the discussion of a more serious a well-developed solution regarding that alongside this there are some consortiums discussing standards for containers.

After reading, @tonybaloney last post, I still does not have a well defined opinion, because I'm questioning (because I'm not seeing at first) what is the major differences that requires a differente object Driver for Containers?

To finalize, IMHO, I think that considering Docker as a provider of Libcloud is not that farfetch, because, I may be wrong, but you support Digital Ocean and other providers which works with Docker Daemon/Client to deploy containers.
Source: https://docs.docker.com/machine/drivers/

[tetron]

One use case for a Docker compute node driver is a test environment where Docker containers are an adequate stand-in for booting full fledged cloud VMs.

[tonybaloney]

@poliveira89 @tetron @Runseb this PR is now #666