New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Windows: The real Windows exec driver is here. #13554

Merged
merged 1 commit into from Jul 10, 2015

Conversation

Projects
None yet
10 participants
@jhowardmsft
Contributor

jhowardmsft commented May 28, 2015

Signed-off-by: John Howard jhoward@microsoft.com
@swernli

This PR is part of the proposal described in docker/docker issue 10662 to port the docker daemon to Windows.

execdriver

This PR adds an alpha-version of the Windows exec driver. Been waiting a long time to be able to finally PR this :) See screenshot above - the daemon is really starting to bootstrap on Windows now! This PR vendors in a Windows named pipe library which is used by the exec driver to communicate with the HCS (Host Compute Service) which is (will be when a build is available publically) the management service for containers on Windows.

Currently the storage parts of the HCS shim are not included - they will come in a following PR with the actual Windows graphdriver. I'm currently developing against vfs and using a dummy mode in HCS which effectively makes docker a process launcher outside of a container.

@runcom

This comment has been minimized.

Show comment
Hide comment
@runcom

runcom May 28, 2015

Member

🐳 🎉

Member

runcom commented May 28, 2015

🐳 🎉

@thaJeztah

This comment has been minimized.

Show comment
Hide comment
@thaJeztah

thaJeztah May 28, 2015

Member

Wow! very very cool!

Member

thaJeztah commented May 28, 2015

Wow! very very cool!

@calavera

This comment has been minimized.

Show comment
Hide comment
@calavera

calavera May 28, 2015

Contributor

🙀

Contributor

calavera commented May 28, 2015

🙀

@LK4D4

This comment has been minimized.

Show comment
Hide comment
@LK4D4

LK4D4 May 28, 2015

Contributor

@jhowardmsft I have a question, can we move all this stuff under libcontainer interface and use native execdriver as proxy? Because I'm not very happy with new execdriver, when I'm going to remove all of them :)

Contributor

LK4D4 commented May 28, 2015

@jhowardmsft I have a question, can we move all this stuff under libcontainer interface and use native execdriver as proxy? Because I'm not very happy with new execdriver, when I'm going to remove all of them :)

@jhowardmsft

This comment has been minimized.

Show comment
Hide comment
@jhowardmsft

jhowardmsft May 28, 2015

Contributor

@LK4D4 Urgh! ;) Hmmm, do you have a timeframe for their removal or something I could look at as an example?

Contributor

jhowardmsft commented May 28, 2015

@LK4D4 Urgh! ;) Hmmm, do you have a timeframe for their removal or something I could look at as an example?

@jhowardmsft

This comment has been minimized.

Show comment
Hide comment
@jhowardmsft

jhowardmsft May 28, 2015

Contributor

Actually - was just looking at this - thought you were referring to something different. To do that would require major refactoring of the native execdriver to _windows and _Linux versions which would be IMO extremely confusing. (Assuming we're now talking about the same thing)

Contributor

jhowardmsft commented May 28, 2015

Actually - was just looking at this - thought you were referring to something different. To do that would require major refactoring of the native execdriver to _windows and _Linux versions which would be IMO extremely confusing. (Assuming we're now talking about the same thing)

@icecrime

This comment has been minimized.

Show comment
Hide comment
@icecrime

icecrime Jun 12, 2015

Contributor

@jhowardmsft, we don't want to build more on execdriver as we mentioned in the past. I'm sorry to ask this but please wrap this driver a type which implements libcontainer.Container interface, and that should be the native driver on Windows.

Contributor

icecrime commented Jun 12, 2015

@jhowardmsft, we don't want to build more on execdriver as we mentioned in the past. I'm sorry to ask this but please wrap this driver a type which implements libcontainer.Container interface, and that should be the native driver on Windows.

@jhowardmsft

This comment has been minimized.

Show comment
Hide comment
@jhowardmsft

jhowardmsft Jun 13, 2015

Contributor

@icecrime As I've explained over the phone and in email now, we really can't see how this would be done. If you can point us to a PR-ready branch which contains an implementation of LXC (or Native) on Linux with replacement exec drivers that we could use as a starting point, that would be the best way forward. We really need additionally guidance and samples on this to move forward and are stuck without guidance from docker folks.

Contributor

jhowardmsft commented Jun 13, 2015

@icecrime As I've explained over the phone and in email now, we really can't see how this would be done. If you can point us to a PR-ready branch which contains an implementation of LXC (or Native) on Linux with replacement exec drivers that we could use as a starting point, that would be the best way forward. We really need additionally guidance and samples on this to move forward and are stuck without guidance from docker folks.

@LK4D4

This comment has been minimized.

Show comment
Hide comment
@LK4D4

LK4D4 Jun 13, 2015

Contributor

@ashahab-altiscale Can you pls share your work with @jhowardmsft

Contributor

LK4D4 commented Jun 13, 2015

@ashahab-altiscale Can you pls share your work with @jhowardmsft

@ashahab-altiscale

This comment has been minimized.

Show comment
Hide comment
@ashahab-altiscale

ashahab-altiscale Jun 13, 2015

Contributor

@LK4D4 Sure.
@jhowardmsft https://github.com/ashahab-altiscale/docker/tree/replace_execdriver_lxc
It's not ready for primetime, but you will get the idea.

Contributor

ashahab-altiscale commented Jun 13, 2015

@LK4D4 Sure.
@jhowardmsft https://github.com/ashahab-altiscale/docker/tree/replace_execdriver_lxc
It's not ready for primetime, but you will get the idea.

@jhowardmsft

This comment has been minimized.

Show comment
Hide comment
@jhowardmsft

jhowardmsft Jun 13, 2015

Contributor

@ashahab-altiscale Thank you, I will take a look.

@icecrime @LK4D4 Can I propose that the current model is still PR'd in to continue unblocking Windows. (We really want to get off the 1.4 code base being used by most people internally here - fixing bugs in it makes no sense at all given how much else has changed in Docker). This PR is at least a parallel of the existing Linux drivers on master as it stands right now, so this does not seem an unreasonable request to me.

This PR (along with graphdriver, about to be PR'd - just a couple of bugs to resolve, my other outstanding PRs, and obviously ignoring SQLite for the moment ;) ), will get us to within ~85 to 90% parity of what was demo'd at build on1.4 - effectively a largely working Windows daemon - compiling, running, and driving real Windows Containers on Windows. I will commit to working on moving the interfaces over to libcontainer for the Windows exec for the next release, and hope that the Linux side will be in unison. A reasonable compromise? Thanks.

Contributor

jhowardmsft commented Jun 13, 2015

@ashahab-altiscale Thank you, I will take a look.

@icecrime @LK4D4 Can I propose that the current model is still PR'd in to continue unblocking Windows. (We really want to get off the 1.4 code base being used by most people internally here - fixing bugs in it makes no sense at all given how much else has changed in Docker). This PR is at least a parallel of the existing Linux drivers on master as it stands right now, so this does not seem an unreasonable request to me.

This PR (along with graphdriver, about to be PR'd - just a couple of bugs to resolve, my other outstanding PRs, and obviously ignoring SQLite for the moment ;) ), will get us to within ~85 to 90% parity of what was demo'd at build on1.4 - effectively a largely working Windows daemon - compiling, running, and driving real Windows Containers on Windows. I will commit to working on moving the interfaces over to libcontainer for the Windows exec for the next release, and hope that the Linux side will be in unison. A reasonable compromise? Thanks.

@crosbymichael

This comment has been minimized.

Show comment
Hide comment
@crosbymichael

crosbymichael Jun 29, 2015

Contributor

We thought about this more and reviewed the implementation and it would be better if we moved this PR forward instead of changing the interface to where docker integrates with windows containers.

Right now the spec and runc work is still being finalized for an initial draft to be released and it would be too much of a burden to have someone build on those projects right now until it is finalized.

As longs as we agree that when the time comes to remove exec drivers and port the container code behind runc we will have help on the Windows side. If so then this PR LGTM.

thanks @jhowardmsft for the patience

Contributor

crosbymichael commented Jun 29, 2015

We thought about this more and reviewed the implementation and it would be better if we moved this PR forward instead of changing the interface to where docker integrates with windows containers.

Right now the spec and runc work is still being finalized for an initial draft to be released and it would be too much of a burden to have someone build on those projects right now until it is finalized.

As longs as we agree that when the time comes to remove exec drivers and port the container code behind runc we will have help on the Windows side. If so then this PR LGTM.

thanks @jhowardmsft for the patience

Show outdated Hide outdated hack/vendor.sh Outdated
@@ -73,6 +73,7 @@ type CommonContainer struct {
MountLabel, ProcessLabel string
RestartCount int
UpdateDns bool
HasBeenStartedBefore bool

This comment has been minimized.

@crosbymichael

crosbymichael Jul 7, 2015

Contributor

What is this used for? Just wondering.

@crosbymichael

crosbymichael Jul 7, 2015

Contributor

What is this used for? Just wondering.

This comment has been minimized.

@jhowardmsft

jhowardmsft Jul 7, 2015

Contributor

It allows us to do a first boot optimization on Windows when starting the container.

@jhowardmsft

jhowardmsft Jul 7, 2015

Contributor

It allows us to do a first boot optimization on Windows when starting the container.

This comment has been minimized.

@crosbymichael

crosbymichael Jul 7, 2015

Contributor

does this apply to first started ever or first started by a machine reboot?

@crosbymichael

crosbymichael Jul 7, 2015

Contributor

does this apply to first started ever or first started by a machine reboot?

This comment has been minimized.

@jhowardmsft

jhowardmsft Jul 7, 2015

Contributor

First time the Windows container OS is booted. Not related to the physical machine.

@jhowardmsft

jhowardmsft Jul 7, 2015

Contributor

First time the Windows container OS is booted. Not related to the physical machine.

@jhowardmsft

This comment has been minimized.

Show comment
Hide comment
@jhowardmsft

jhowardmsft Jul 10, 2015

Contributor

Janky/Experimental failures due to #14520 which fixes flakiness

Contributor

jhowardmsft commented Jul 10, 2015

Janky/Experimental failures due to #14520 which fixes flakiness

@crosbymichael

This comment has been minimized.

Show comment
Hide comment
@crosbymichael

crosbymichael Jul 10, 2015

Contributor

LGTM

Contributor

crosbymichael commented Jul 10, 2015

LGTM

Show outdated Hide outdated daemon/execdriver/windows/run.go Outdated
Show outdated Hide outdated daemon/execdriver/windows/exec.go Outdated
Windows: Exec driver
Signed-off-by: John Howard <jhoward@microsoft.com>
@estesp

This comment has been minimized.

Show comment
Hide comment
@estesp

estesp Jul 10, 2015

Contributor

Thanks @jhowardmsft
LGTM

Contributor

estesp commented Jul 10, 2015

Thanks @jhowardmsft
LGTM

estesp added a commit that referenced this pull request Jul 10, 2015

Merge pull request #13554 from Microsoft/10662-winexec
Windows: The real Windows exec driver is here.

@estesp estesp merged commit b7e8169 into moby:master Jul 10, 2015

4 checks passed

docker/dco-signed All commits signed
Details
experimental Jenkins build Docker-PRs-experimental 2579 has succeeded
Details
janky Jenkins build Docker-PRs 11126 has succeeded
Details
windows Jenkins build Windows-PRs 7779 has succeeded
Details

@jhowardmsft jhowardmsft deleted the Microsoft:10662-winexec branch Jul 10, 2015

@jhowardmsft

This comment has been minimized.

Show comment
Hide comment
@jhowardmsft

jhowardmsft Jul 10, 2015

Contributor

Yay!!! Thanks @crosbymichael @estesp

Contributor

jhowardmsft commented Jul 10, 2015

Yay!!! Thanks @crosbymichael @estesp

@thaJeztah

This comment has been minimized.

Show comment
Hide comment
@thaJeztah
Member

thaJeztah commented Jul 10, 2015

Thanks @jhowardmsft !

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment