-
Notifications
You must be signed in to change notification settings - Fork 2.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Discovering and reattaching to running containers #185
Comments
Reattaching stream is kinda hard. I think there're some hacks involving ptrace to let you reattach the stdin/stdout, but im not sure What do you think about runc exposing a set of APIs ( HTTP ? ) that allows the monitor ( docker ) in this case to monitor and attach to the container stream ? The API is probably mirrors docker streaming API, but maybe with some additional healthchecks to make sure that runc is running properly. Something like this:
Running one unix socket per container is also helpful in terms of discovery. A docker engine can reserve its /var/run/docker/engine-id/ directory and symlink the sockets of its managed container there for discovery purpose. So when the daemon restarts, it can find the current runc processes there. Some thoughts:
|
I agree with @dqminh . Actually I have made some progress on implementing docker hot upgrade. If you are interested, see https://github.com/chenchun/docker/commits/upgrades. When I started working on this, I haven't head about runc, so I create an external monitor process per container like the design proposal of moby/moby#13884. The external monitor process is responsible for monitoring the container, notifying daemon of container starting or exiting, and providing extra streaming APIs such as |
Initially, these are two different problems to solve. There could be a layout for discovering running containers, like a Reattaching is surely more tricky, unless there is a per-container supervisor or init. |
@chenchun Good to know that you went ahead with design akin to our Proposal A... Just a question for you, how about the parent child relationship between the daemon and the containers? Do you daemonize Monitor+Container? |
@mudverma Actually, I'm doing Proposal B right now. The relationship between the daemon and the containers is the same with the graph in Proposal B. |
@chenchun thanks .. I pulled your code and ran it. When I killed the daemon, the containers were killed by the daemon on restart ? Also, I don't see any monitor process either in between the daemon and the containers. Am I missing something? | `-docker-1.8.0-de,32642 -D -d | |-sh,32706 -c while true; do echo hello world; sleep 1; done | |-sh,32752 -c while true; do echo hello world; sleep 1; done |
@mudverma, How did you killed the daemon? If you are using |
@mudverma you have to copy the newly compiled docker to Before, I started a ubuntu container
Now kill the daemon directly. Interactive mode is not effected. (docker daemon restarts due to ubuntu upstart)
|
We need to decide if sockets based solution provides too much complexity to runc or not. There's probably also a way to try to achieve it with fifos but it would be hacky and I'm not sure how stable. Keep in mind that this is not only a technical question but we would need a way to define what is going on when there is nothing attached to runc. For example if there is a logger setup we do not want to lose any logs between attaches. When there is no logger we don't want to buffer but simply discard unless there is someone attached. We also dont' want to just block or error on writes because user applications likely don't know how to handle that. This would mean something like As it goes pretty complex, I think we should consider leaving this out of runc. This can always be handled in an upper layer. If you want to crash the daemon you could write something intermediate that only handles the streams. |
On Fri, Aug 14, 2015 at 10:58:58AM -0700, Tõnis Tiigi wrote:
I agree that reattaching a higher-level monitor to streams associated If reattaching to streams or getting nonblocking streams on monitor |
@tiborvass Hi, I was just wondering about how Docker is integrating with runc? Is there some internal work on getting rid of parent child relationship b/w daemon-runc-containers? runc daemon integration was targeted for 1.9? So there must be some work on that already. |
@mudverma It didn't make it in 1.9, but @LK4D4 or @crosbymichael can say more about this. As far as this issue is concerned, I think it makes sense to leave reattaching out of runc like @tonistiigi said. I'm closing this issue. |
We'd like the Docker daemon to be able to die and discover its containers upon restarting.
What are good implementations for achieving this? Having the daemon set some ID (caller ID?) in the containers' state?
It would also need to have a way of reattaching to standard streams of those running containers.
We also need to keep in mind the scenario when multiple docker daemons (or equivalent) spawn their own sets of containers; there shouldn't be interference between those instances.
The text was updated successfully, but these errors were encountered: