Multiple hosts #9

Open
asbjornenge opened this Issue Jan 25, 2014 · 11 comments

10 participants

@asbjornenge

After thinking some more about multiple host support, I don't think there is an elegant DNS solution to the problem.

One idea I have is to allow listening on multiple docker hosts, e.g. supporting multiple -s parameters, and leaving the networking between the hosts to some other mechanism.

This would keep skydock simple, focused and clean.

A setup similar to http://goldmann.pl/blog/2014/01/21/connecting-docker-containers-on-multiple-hosts/ might work...? Or any other setup. The more flexible skydock would not make assumptions about it and simply get out of it's way.

Thoughts?

@kshep

Was just chatting with @crosbymichael in IRC about this. He mentioned he'd like a container-based approach, but it certainly seems like there's an option that could take advantage of an Open vSwitch VLAN that one sets up on all the hosts in a cluster before launching anything.

@asbjornenge

@goldmann certainly interesting! I have been avoiding ip conflicts by providing a different subnet to the docker daemon for each host (-bip). But, unless I am mistaken, dhcp does not solve the issue with using skydock for multiple host setups...? It is however a nice network setup, and provided that skydock could talk to multiple docker daemons, it would probably work very nicely 👍

@jhulten

I was thinking about this and was wondering how possible it would be to have two levels, private and public. Everything that connects to the bridge would be local and everything that binds to the public ip would be public. This might require two skydns services, one for .local and one for mycorp.com. Then only clustering the mycorp.com instance.

Thoughts?

@crosbymichael
@asbjornenge

So, I had another idea... It requires the introduction of a skydock scheduler (container).

Registering hosts

(this could be done in number of ways, but in the spirit of skydock, lets leverage dns)

Hosts are registered in a DNS SRV record hosts.domain.com with your "external" DNS service. This can be done either via your host spawn script, manually or by enabling the host to self register.

Scheduler

The skydock scheduler would be responsible for querying hosts.domain.com and feeding that information to skydock. The scheduler is also responsible for spawning all containers in the cluster, including skydns and skydock. Since the scheduler knows where skydns lives and it's IP, it can pass a -dns flag along to all other containers.

Requirements

Preconfigured network (all the hosts need to be able to talk to each other).
Separate docker subnets on each host.

Crazy stuff

The scheduler could perhaps even be responsible for starting the docker daemon on each host, thereby removing the last bit of host state (subnet). Taking that even further; I wonder about any potential drawbacks of running docker within docker...? Could perhaps all the hosts run vanilla docker, and the scheduler just deploy a "docker daemon container" on each host (with subnet arguments etc.) and use those to run all the application containers...? I don't know, I'm not a doc(ker).

Thoughts?

@pires

+1

@prologic

/cc @therealprologic

@orlin

Have you seen Serf? Also written in Go. May be a good combination...

@Lennie

Finally had an idea about how to make it easier to do this:
skynetservices/skydns#46

@tpires

+1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment