New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Proposal: Introduce Dockerfiles for windowsservercore #222

Closed
wants to merge 1 commit into
base: master
from

Conversation

Projects
None yet
7 participants
@StefanScherer

StefanScherer commented Aug 25, 2016

After golang now has its first Windows Docker image on Docker Hub it is time for node to provide an official Windows Docker image as well.

This PR is a proposal to build and push Windows Docker images for the windowsservercore variant which works on Windows 10 + Docker 4 Windows Beta 31++ or Windows Server 2016.

The test-build.ps1 can be called to build the images. It downloads the ZIP files for Node.js 4.7.0, 6.9.2 and 7.2.1 and builds these Docker images:

  • node:4.7.0-windowsservercore
  • node:4.7.0-windowsservercore-onbuild
  • node:6.9.2-windowsservercore
  • node:6.9.2-windowsservercore-onbuild
  • node:7.2.1-windowsservercore
  • node:7.2.1-windowsservercore-onbuild

In my previous work building a Windows Docker image I used the MSI packages which can be found here https://github.com/StefanScherer/dockerfiles-windows/tree/master/node, but this results in a slightly bigger Docker image as Windows caches the MSI packages in a second place even after removing it in the Dockerfile. So I switched to the ZIP files.

The labels for the Docker images are based on the work for the golang Docker images.

See also:

For information about how to get Docker running on Windows, please see the relevant Windows Server Quick Start guide provided by Microsoft.

See also #223 for the nanoserver Dockerfile proposal.

@Starefossen

This comment has been minimized.

Show comment
Hide comment
@Starefossen

Starefossen Aug 25, 2016

Member

This is great! 😄

Now we just need a CI capable of running Docker for Windows so we can make sure things don't break!

Member

Starefossen commented Aug 25, 2016

This is great! 😄

Now we just need a CI capable of running Docker for Windows so we can make sure things don't break!

@Starefossen

This comment has been minimized.

Show comment
Hide comment
@Starefossen

Starefossen Aug 25, 2016

Member

@tianon can Docker Hub build Docker Images for Windows?

Member

Starefossen commented Aug 25, 2016

@tianon can Docker Hub build Docker Images for Windows?

@StefanScherer

This comment has been minimized.

Show comment
Hide comment
@StefanScherer

StefanScherer Aug 25, 2016

@Starefossen not yet, also AppVeyor waits for Windows Server 2016 GA.

StefanScherer commented Aug 25, 2016

@Starefossen not yet, also AppVeyor waits for Windows Server 2016 GA.

@StefanScherer

This comment has been minimized.

Show comment
Hide comment
@StefanScherer

StefanScherer Aug 25, 2016

I'm more concerned if we need more dev dependencies to make it usable for the community.

Maybe we should add Python and probanly https://github.com/felixrieseberg/windows-build-tools to allow compiling native node modules as well.

The golang image without git was also not really usable.

StefanScherer commented Aug 25, 2016

I'm more concerned if we need more dev dependencies to make it usable for the community.

Maybe we should add Python and probanly https://github.com/felixrieseberg/windows-build-tools to allow compiling native node modules as well.

The golang image without git was also not really usable.

@StefanScherer

This comment has been minimized.

Show comment
Hide comment
@StefanScherer

StefanScherer Sep 4, 2016

I just tested the 6.5.0 Dockerfile with another line without any further optimizations

RUN npm install --global --production windows-build-tools

but then the Docker image increases from 7.9 GB to 11.5 GB:

C:\Users\vagrant\dockerfiles-windows\node [master ≡ +0 ~1 -0 !]> docker images
REPOSITORY                    TAG                 IMAGE ID            CREATED             SIZE
stefanscherer/node-windows    6.5.0-full          d564a1d10cab        55 seconds ago      11.47 GB
stefanscherer/node-windows    6.5.0               558a2232ccf5        5 days ago          7.887 GB

I personally don't have the use case of compiling native node modules and build a Docker image with that for distribution. My workflow would look like this: Run npm install --production in AppVeyor in their full installed build images and then COPY deploy own code + node_modules into a Docker image (once they have a Docker build image).

StefanScherer commented Sep 4, 2016

I just tested the 6.5.0 Dockerfile with another line without any further optimizations

RUN npm install --global --production windows-build-tools

but then the Docker image increases from 7.9 GB to 11.5 GB:

C:\Users\vagrant\dockerfiles-windows\node [master ≡ +0 ~1 -0 !]> docker images
REPOSITORY                    TAG                 IMAGE ID            CREATED             SIZE
stefanscherer/node-windows    6.5.0-full          d564a1d10cab        55 seconds ago      11.47 GB
stefanscherer/node-windows    6.5.0               558a2232ccf5        5 days ago          7.887 GB

I personally don't have the use case of compiling native node modules and build a Docker image with that for distribution. My workflow would look like this: Run npm install --production in AppVeyor in their full installed build images and then COPY deploy own code + node_modules into a Docker image (once they have a Docker build image).

@friism

This comment has been minimized.

Show comment
Hide comment
@StefanScherer

This comment has been minimized.

Show comment
Hide comment
@StefanScherer

StefanScherer Sep 28, 2016

@friism I've tried several times to adapt the Dockerfile to PowerShell without the CMD variable names, but it seems that there is a strange problem using quotes for strings.
As this Dockerfile has the ErrorActionPreference it should be good enough for now.

FROM microsoft/windowsservercore
SHELL ["powershell", "-Command", "$ErrorActionPreference = 'Stop';"]
ENV NODE_VERSION 4.6.0

RUN Write-Host "https://nodejs.org/dist/v$($env:NODE_VERSION)/node-v$($env:NODE_VERSION)-win-x64.zip"
RUN $url = "https://nodejs.org/dist/v$($env:NODE_VERSION)/node-v$($env:NODE_VERSION)-win-x64.zip"
RUN (New-Object System.Net.WebClient).DownloadFile("https://nodejs.org/dist/v$($env:NODE_VERSION)/node-v$($env:NODE_VERSION)-win-x64.zip", 'node.zip')

shows the correct string after the Write-Host, but saving that string as a variable or use it in the DownloadFile function does not work:

Step 6/10 : RUN Write-Host "https://nodejs.org/dist/v$($env:NODE_VERSION)/node-v$($env:NODE_VERSION)-win-x64.zip"
 ---> Running in 96040c5827a5
https://nodejs.org/dist/v4.6.0/node-v4.6.0-win-x64.zip
 ---> 092e42203a2f
Removing intermediate container 96040c5827a5
Step 7/10 : RUN $url = "https://nodejs.org/dist/v$($env:NODE_VERSION)/node-v$($env:NODE_VERSION)-win-x64.zip"
 ---> Running in 68bcfb0f2eba
https://nodejs.org/dist/v$($env:NODE_VERSION)/node-v$($env:NODE_VERSION)-win-x6
4.zip : The term 'https://nodejs.org/dist/v$($env:NODE_VERSION)/node-v$($env:NO
DE_VERSION)-win-x64.zip' is not recognized as the name of a cmdlet, function,
script file, or operable program. Check the spelling of the name, or if a path
was included, verify that the path is correct and try again.
At line:1 char:41
+ ... op'; $url = https://nodejs.org/dist/v$($env:NODE_VERSION)/node-v$($en ...
+                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : ObjectNotFound: (https://nodejs....ON)-win-x64.z
   ip:String) [], ParentContainsErrorRecordException
    + FullyQualifiedErrorId : CommandNotFoundException

So I keep it as it is right now.

StefanScherer commented Sep 28, 2016

@friism I've tried several times to adapt the Dockerfile to PowerShell without the CMD variable names, but it seems that there is a strange problem using quotes for strings.
As this Dockerfile has the ErrorActionPreference it should be good enough for now.

FROM microsoft/windowsservercore
SHELL ["powershell", "-Command", "$ErrorActionPreference = 'Stop';"]
ENV NODE_VERSION 4.6.0

RUN Write-Host "https://nodejs.org/dist/v$($env:NODE_VERSION)/node-v$($env:NODE_VERSION)-win-x64.zip"
RUN $url = "https://nodejs.org/dist/v$($env:NODE_VERSION)/node-v$($env:NODE_VERSION)-win-x64.zip"
RUN (New-Object System.Net.WebClient).DownloadFile("https://nodejs.org/dist/v$($env:NODE_VERSION)/node-v$($env:NODE_VERSION)-win-x64.zip", 'node.zip')

shows the correct string after the Write-Host, but saving that string as a variable or use it in the DownloadFile function does not work:

Step 6/10 : RUN Write-Host "https://nodejs.org/dist/v$($env:NODE_VERSION)/node-v$($env:NODE_VERSION)-win-x64.zip"
 ---> Running in 96040c5827a5
https://nodejs.org/dist/v4.6.0/node-v4.6.0-win-x64.zip
 ---> 092e42203a2f
Removing intermediate container 96040c5827a5
Step 7/10 : RUN $url = "https://nodejs.org/dist/v$($env:NODE_VERSION)/node-v$($env:NODE_VERSION)-win-x64.zip"
 ---> Running in 68bcfb0f2eba
https://nodejs.org/dist/v$($env:NODE_VERSION)/node-v$($env:NODE_VERSION)-win-x6
4.zip : The term 'https://nodejs.org/dist/v$($env:NODE_VERSION)/node-v$($env:NO
DE_VERSION)-win-x64.zip' is not recognized as the name of a cmdlet, function,
script file, or operable program. Check the spelling of the name, or if a path
was included, verify that the path is correct and try again.
At line:1 char:41
+ ... op'; $url = https://nodejs.org/dist/v$($env:NODE_VERSION)/node-v$($en ...
+                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : ObjectNotFound: (https://nodejs....ON)-win-x64.z
   ip:String) [], ParentContainsErrorRecordException
    + FullyQualifiedErrorId : CommandNotFoundException

So I keep it as it is right now.

@tianon

This comment has been minimized.

Show comment
Hide comment
@tianon

tianon Sep 28, 2016

Contributor

@StefanScherer in my own testing with PowerShell's string interpolation, it had odd edge cases that made it more trouble than it was worth; I found either ('string' + $var + 'string') or ('Format {0} string {1}' -f $var1, $var2) to be much more reliable

Contributor

tianon commented Sep 28, 2016

@StefanScherer in my own testing with PowerShell's string interpolation, it had odd edge cases that made it more trouble than it was worth; I found either ('string' + $var + 'string') or ('Format {0} string {1}' -f $var1, $var2) to be much more reliable

@StefanScherer

This comment has been minimized.

Show comment
Hide comment
@StefanScherer

StefanScherer Sep 29, 2016

@tianon Thanks. I've added similar workarounds to introduce the SHELL instruction. PR rebased and squashed to use latest Node.js versions 4.6 and 6.7.

StefanScherer commented Sep 29, 2016

@tianon Thanks. I've added similar workarounds to introduce the SHELL instruction. PR rebased and squashed to use latest Node.js versions 4.6 and 6.7.

@chorrell chorrell referenced this pull request Sep 29, 2016

Merged

Improve test #239

@chorrell

My preference would be to simplify things by using a windowsservercore variant directory. So instead of this:

4.6/windows/windowsservercore/Dockerfile

We'd have:

4.6/windowsservercore/Dockerfile

And do the same for onbuild

@friism

This comment has been minimized.

Show comment
Hide comment
@friism

friism Sep 29, 2016

@chorrell note that there may also (likely) be a nanoserver variant alongside windowsservercore.

friism commented Sep 29, 2016

@chorrell note that there may also (likely) be a nanoserver variant alongside windowsservercore.

@StefanScherer

This comment has been minimized.

Show comment
Hide comment
@StefanScherer

StefanScherer Sep 29, 2016

@chorrell Thanks for #239. Tried something similar today morning, but you know your test scripts better than I.
Your comment about removing the windows directory, is it still valid if #223 gets merged in the future to support nanoserver + windowsservercore?
If yes, I'll rebase both PR's, no problem.

StefanScherer commented Sep 29, 2016

@chorrell Thanks for #239. Tried something similar today morning, but you know your test scripts better than I.
Your comment about removing the windows directory, is it still valid if #223 gets merged in the future to support nanoserver + windowsservercore?
If yes, I'll rebase both PR's, no problem.

@chorrell

This comment has been minimized.

Show comment
Hide comment
@chorrell

chorrell Sep 29, 2016

Contributor

At the time I thought it might make things simpler to omit the windows directory, but I wasn't considering tags and looking at docker-library/golang#106 it makes more sense. So yeah, leave it as is withwindows/nanoserver and windows/windowsservercore.

We'll want to probably modify the update.sh script and add new *.template files for the nanoserver and windowsservercore images at some point. I might also need to do more updates to test-build.sh too, but I'll merge #239 first.

Contributor

chorrell commented Sep 29, 2016

At the time I thought it might make things simpler to omit the windows directory, but I wasn't considering tags and looking at docker-library/golang#106 it makes more sense. So yeah, leave it as is withwindows/nanoserver and windows/windowsservercore.

We'll want to probably modify the update.sh script and add new *.template files for the nanoserver and windowsservercore images at some point. I might also need to do more updates to test-build.sh too, but I'll merge #239 first.

@StefanScherer

This comment has been minimized.

Show comment
Hide comment
@StefanScherer

StefanScherer Nov 3, 2016

Anything I can do to push this forward a little bit?

StefanScherer commented Nov 3, 2016

Anything I can do to push this forward a little bit?

misunderstanding, I wasn't thinking about tags

@chorrell

This comment has been minimized.

Show comment
Hide comment
@chorrell

chorrell Nov 3, 2016

Contributor

Was there still an open question about doing CI? I don't have any experience running Docker in Windows and I don't really have a handle on how to test this or what we'd use for automated CI.

Contributor

chorrell commented Nov 3, 2016

Was there still an open question about doing CI? I don't have any experience running Docker in Windows and I don't really have a handle on how to test this or what we'd use for automated CI.

@StefanScherer

This comment has been minimized.

Show comment
Hide comment
@StefanScherer

StefanScherer Nov 3, 2016

Something like .travis.yml + test_build.sh for Windows?
This should be possible on AppVeyor soon: appveyor.yml + test_build.ps1

But I think there is more CI to push to Docker Hub as well, right? Jenkins? So you would need eg. a Windows 2016 Jenkins slave for further steps.

StefanScherer commented Nov 3, 2016

Something like .travis.yml + test_build.sh for Windows?
This should be possible on AppVeyor soon: appveyor.yml + test_build.ps1

But I think there is more CI to push to Docker Hub as well, right? Jenkins? So you would need eg. a Windows 2016 Jenkins slave for further steps.

@chorrell

This comment has been minimized.

Show comment
Hide comment
@chorrell

chorrell Nov 3, 2016

Contributor

The push to docker hub is all manual via a PR e.g., docker-library/official-images#2281

AppVeyor sounds like it might be an option for validating the images then?

Contributor

chorrell commented Nov 3, 2016

The push to docker hub is all manual via a PR e.g., docker-library/official-images#2281

AppVeyor sounds like it might be an option for validating the images then?

@StefanScherer

This comment has been minimized.

Show comment
Hide comment
@StefanScherer

StefanScherer Nov 3, 2016

Yes, they plan to setup Win2016 + nanoserver build agents with preloaded base images microsoft/windowsservercore + microsoft/nanoserver.

StefanScherer commented Nov 3, 2016

Yes, they plan to setup Win2016 + nanoserver build agents with preloaded base images microsoft/windowsservercore + microsoft/nanoserver.

@chorrell

This comment has been minimized.

Show comment
Hide comment
@chorrell

chorrell Nov 3, 2016

Contributor

Also, I think a few remaining tasks are:

  • Update the Dockerfiles with the latest Node.js versions (v4.6.1, v7.0.0, etc)
  • Create .template files for the windowsservercore Dockerfiles
  • Update the update.sh script for updating the Dockerfiles or create a new script just for updating the Windows images.
Contributor

chorrell commented Nov 3, 2016

Also, I think a few remaining tasks are:

  • Update the Dockerfiles with the latest Node.js versions (v4.6.1, v7.0.0, etc)
  • Create .template files for the windowsservercore Dockerfiles
  • Update the update.sh script for updating the Dockerfiles or create a new script just for updating the Windows images.
@chorrell

This comment has been minimized.

Show comment
Hide comment
@chorrell

chorrell Nov 3, 2016

Contributor

OK, so we probably want to wait until AppVeyor has Windows server 2016. Afaik, the build group doesn't have any 2016 machines in the jenkins/ci setup: https://ci.nodejs.org/computer/

Contributor

chorrell commented Nov 3, 2016

OK, so we probably want to wait until AppVeyor has Windows server 2016. Afaik, the build group doesn't have any 2016 machines in the jenkins/ci setup: https://ci.nodejs.org/computer/

@StefanScherer

This comment has been minimized.

Show comment
Hide comment
@StefanScherer

StefanScherer Nov 3, 2016

For example Azure has prebuilt Win2016+Containers including Docker engine preinstalled, there is also an up-to-date "Deploy to Azure" template here.
So this should be easy to extend new Jenkins slaves.

I can have a look at the update.sh

StefanScherer commented Nov 3, 2016

For example Azure has prebuilt Win2016+Containers including Docker engine preinstalled, there is also an up-to-date "Deploy to Azure" template here.
So this should be easy to extend new Jenkins slaves.

I can have a look at the update.sh

@chorrell

This comment has been minimized.

Show comment
Hide comment
@chorrell

chorrell Nov 3, 2016

Contributor

OK, that would be up to the @nodejs/build group to setup I guess.

Contributor

chorrell commented Nov 3, 2016

OK, that would be up to the @nodejs/build group to setup I guess.

@StefanScherer

This comment has been minimized.

Show comment
Hide comment
@StefanScherer

StefanScherer Nov 3, 2016

The update.sh could be enhanced like the one for docker-library/mongo/update.sh fetching the sha256 sums for the Windows Dockerfiles.

StefanScherer commented Nov 3, 2016

The update.sh could be enhanced like the one for docker-library/mongo/update.sh fetching the sha256 sums for the Windows Dockerfiles.

@StefanScherer

This comment has been minimized.

Show comment
Hide comment
@StefanScherer

StefanScherer Dec 7, 2016

Updated to Node 4.7.0, 6.9.2 and 7.2.1 :-)

StefanScherer commented Dec 7, 2016

Updated to Node 4.7.0, 6.9.2 and 7.2.1 :-)

@computeronix

This comment has been minimized.

Show comment
Hide comment
@computeronix

computeronix Dec 27, 2016

FYI streamlined the windows side with the msi (of course wouldn't work with nano though)

#297

computeronix commented Dec 27, 2016

FYI streamlined the windows side with the msi (of course wouldn't work with nano though)

#297

@computeronix

This comment has been minimized.

Show comment
Hide comment
@computeronix

computeronix Dec 27, 2016

Python has a splitter between Windows and Linux so could be possible to easily implement with the proper tags and automated building mechanism as discussed above
https://store.docker.com/images/1ae86987-df14-4741-9433-d9602a4da995?tab=description

The ones provided by @StefanScherer and myself both are natively supported within Windows Server 2016 Server (and even Azure Server 2016 /w containers ;) )

computeronix commented Dec 27, 2016

Python has a splitter between Windows and Linux so could be possible to easily implement with the proper tags and automated building mechanism as discussed above
https://store.docker.com/images/1ae86987-df14-4741-9433-d9602a4da995?tab=description

The ones provided by @StefanScherer and myself both are natively supported within Windows Server 2016 Server (and even Azure Server 2016 /w containers ;) )

@StefanScherer

This comment has been minimized.

Show comment
Hide comment
@StefanScherer

StefanScherer Feb 22, 2017

FYI, if setting up a Win2016+Docker build agent is too hard, try AppVeyor's new build agent appveyor/ci#1150 using a simple appveyor.yml like mine.

StefanScherer commented Feb 22, 2017

FYI, if setting up a Win2016+Docker build agent is too hard, try AppVeyor's new build agent appveyor/ci#1150 using a simple appveyor.yml like mine.

@StefanScherer

This comment has been minimized.

Show comment
Hide comment
@StefanScherer

StefanScherer Mar 22, 2017

Any progress having an Win 2016 server to provide official Node.js Windows Docker images?

StefanScherer commented Mar 22, 2017

Any progress having an Win 2016 server to provide official Node.js Windows Docker images?

Introduce Dockerfiles for windowsservercore
Signed-off-by: Stefan Scherer <scherer_stefan@icloud.com>
@friism

This comment has been minimized.

Show comment
Hide comment
@friism

friism Mar 22, 2017

@StefanScherer what about adding appveyor machinery to the PR?

friism commented Mar 22, 2017

@StefanScherer what about adding appveyor machinery to the PR?

@StefanScherer

This comment has been minimized.

Show comment
Hide comment
@StefanScherer

StefanScherer Mar 22, 2017

@friism Thanks, already working on a new PR that introduces both nano + core + appveyor.yml :-)

StefanScherer commented Mar 22, 2017

@friism Thanks, already working on a new PR that introduces both nano + core + appveyor.yml :-)

@StefanScherer

This comment has been minimized.

Show comment
Hide comment
@StefanScherer

StefanScherer Mar 22, 2017

Closing, see #362 for a more up to date PR with appveyor.yml

StefanScherer commented Mar 22, 2017

Closing, see #362 for a more up to date PR with appveyor.yml

@StefanScherer StefanScherer deleted the StefanScherer:introduce-windows-dockerfiles branch Jul 23, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment