Update docker image to build extension modules

[jaraco]

The current docker image doesn't support building extension modules. Attempting to build a module from source indicates:

  running build_ext
  building 'lxml.etree' extension
  error: Microsoft Visual C++ 14.0 or greater is required. Get it with "Microsoft C++ Build Tools": https://visualstudio.microsoft.com/visual-cpp-build-tools/    

I've done some investigation in bpo-43298.

[jaraco]

With the aforementioned commit, the step that installs the compiler completes rather quickly and seems to break off mid-output:

jaraco.windows feature/dockerfile-compiler $ docker build . --no-cache
Sending build context to Docker daemon  47.06MB
Step 1/15 : FROM mcr.microsoft.com/dotnet/framework/sdk:4.8-windowsservercore-ltsc2019
 ---> e8e7cb916939
Step 2/15 : ADD https://aka.ms/vs/16/release/vs_buildtools.exe vs_buildtools.exe
Downloading [==================================================>]  1.467MB/1.467MB

 ---> b38798378054
Step 3/15 : RUN ./vs_buildtools.exe --quiet --wait --norestart --nocache  --add Microsoft.VisualStudio.Component.CoreEditor  --add Microsoft.VisualStudio.Workload.CoreEditor  --add Microsoft.VisualStudio.Component.Roslyn.Compiler  --add Microsoft.Component.MSBuild  --add Microsoft.VisualStudio.Component.TextTemplating  --add Microsoft.VisualStudio.Component.VC.CoreIde  --add Microsoft.VisualStudio.Component.VC.Tools.x86.x64  --add Microsoft.VisualStudio.Component.Windows10SDK.19041  --add Microsoft.VisualStudio.Component.VC.Redist.14.Latest  --add Microsoft.VisualStudio.ComponentGroup.NativeDesktop.Core  --add Microsoft.VisualStudio.Workload.NativeDesktop
 ---> Running in 9d9ebb121023
Preparing: C:\Users\ContainerAdministrator\AppData\Local\Temp\0d316623a8f045be79f4d7689be9\vs_bootstrapper_d15\HelpFile\1028\help.html...
Preparing: C:\Users\ContainerAdministrator\AppData\Local\Temp\0d316623a8f045be79f4d7689be9\vs_bootstrapper_d15\HelpFile\2052\help.html...
Preparing: C:\Users\ContainerAdministrator\AppData\Local\Temp\0d316623a8f045be79f4d7689be9\vs_bootstrapper_d15\HelpFile\1029\help.html...
Preparing: C:\Users\ContainerAdministrator\AppData\Local\Temp\0d316623a8f045be79f4d7689be9\vs_bootstrapper_d15\HelpFile\1036\help.html...
Preparing: C:\Users\ContainerAdministrator\AppData\Local\Temp\0d316623a8f045be79f4d7689be9\vs_bootstrapper_d15\HelpFile\3082\help.html...
Preparing: C:\Users\ContainerAdministrator\AppData\Local\Temp\0d316623a8f045be79f4d7689be9\vs_bootstrapper_d15\HelpFile\1031\help.html...
Preparing: C:\Users\ContainerAdministrator\AppData\Local\Temp\0d316623a8f045be79f4d7689be9\vs_bootstrapper_d15\HelpFile\1055\help.html...
Preparing: C:\Users\ContainerAdministrator\AppData\Local\Temp\0d316623a8f045be79f4d7689be9\vs_bootstrapper_d15\HelpFile\1046\help.html...
Preparing: C:\Users\ContainerAdministrator\AppData\Local\Temp\0d316623a8f045be79f4d7689be9\vs_bootstrapper_d15\HelpFile\1042\help.html...
Preparing: C:\Users\ContainerAdministrator\AppData\Local\Temp\0d316623a8f045be79f4d7689be9\vs_bootstrapper_d15\HelpFile\1040\help.html...
Preparing: C:\Users\ContainerAdministrator\AppData\Local\Temp\0d316623a8f045be79f4d7689be9\vs_bootstrapper_d15\HelpFile\1045\help.html...
Preparing: C:\Users\ContainerAdministrator\AppData\Local\Temp\0d316623a8f045be79f4d7689be9\vs_bootstrapper_d15\HelpFile\1041\help.html...
Preparing: C:\Users\ContainerAdministrator\AppData\Local\Temp\0d316623a8f045be79f4d7689be9\vs_bootstrapper_d15\HelpFile\1049\help.html...
Preparing: C:\Users\ContainerAdministrator\AppData\Local\Temp\0d316623a8f045be79f4d7689be9\vs_bootstrapper_d15\HelpFile\1033\help.html...
Preparing: C:\Users\ContainerAdministrator\AppData\Local\Temp\0d316623a8f045be79f4d7689be9\vs_bootstrapper_d15\vs_setup_bootstrapper.exe...
Preparing: C:\Users\ContainerAdministrator\AppData\Local\Temp\0d316623a8f045be79f4d7689be9\vs_bootstrapper_d15\Microsoft.C2RSignatureReader.Interop.dll...
Preparing: C:\Users\ContainerAdministrator\AppData\Local\Temp\0d316623a8f045be79f4d7689be9\vs_bootstrapper_d15\Microsoft.C2RSignatureReader.Native.dll...
Preparing: C:\Users\ContainerAdministrator\AppData\Local\Temp\0d316623a8f045be79f4d7689be9\vs_bootstrapper_d15\Microsoft.Diagnostics.Tracing.EventSource.dll...
Preparing: C:\Users\ContainerAdministrator\AppData\Local\Temp\0d316623a8f045be79f4d7689be9\vs_bootstrapper_d15\Microsoft.VisualStudio.RemoteControl.dll...
Preparing: C:\Users\ContainerAdministrator\AppData\Local\Temp\0d316623a8f045be79f4d7689be9\vs_bootstrapper_d15\Microsoft.VisualStudio.Setup.Common.dll...
Preparing: C:\Users\ContainerAdministrator\AppData\Local\Temp\0d316623a8f045be79f4d7689be9\vs_bootstrapper_d15\Microsoft.VisualStudio.Setup.dll...
Preparing: C:\Users\ContainerAdministrator\AppData\Local\Temp\0d316623a8f045be79f4d7689be9\vs_bootstrapper_d15\Microsoft.VisualStudio.Setup.Download.dll...
Preparing: C:\Users\ContainerAdministrator\AppData\Local\Temp\0d316623a8f045be79f4d7689be9\vs_bootstrapper_d15\Microsoft.VisualStudio.Telemetry.dll...
Preparing: C:\Users\ContainerAdministrator\AppData\Local\Temp\0d316623a8f045be79f4d7689be9\vs_bootstrapper_d15\Microsoft.VisualStudio.Utilities.Internal.dll...
Preparing: C:\Users\ContainerAdministrator\AppData\Local\Temp\0d316623a8f045be79f4d7689be9\vs_bootstrapper_d15\Newtonsoft.Json.dll...
Preparing: C:\Users\ContainerAdministrator\AppData\Local\Temp\0d316623a8f045be79f4d7689be9\vs_bootstrapper_d15\it\vs_setup_bootstrapper.resources.dll...
Preparing: C:\Users\ContainerAdministrator\AppData\Local\Temp\0d316623a8f045be79f4d7689be9\vs_bootstrapper_d15\pt-BR\vs_setup_bootstrapper.resources.dll...
Preparing: C:\Users\ContainerAdministrator\AppData\Local\Temp\0d316623a8f045be79f4d7689be9\vs_bootstrapper_d15\ko\vs_setup_bootstrapper.resources.dll...
PreparinRemoving intermediate container 9d9ebb121023
 ---> 03d65afdb053
Step 4/15 : RUN powershell -c "Set-ExecutionPolicy Bypass -Scope Process -Force; [System.Net.ServicePointManager]::SecurityProtocol = [System.Net.ServicePointManager]::SecurityProtocol -bor 3072; iwr https://chocolatey.org/install.ps1 -UseBasicParsing | iex"
...

It's almost like the --wait isn't having the intended effect.

[jaraco]

I found that the command to install visual studio components isn't succeeding in installing the components. This minimal Dockerfile shows the issue:

# Install Visual Studio Build tools based on
# https://docs.microsoft.com/en-us/visualstudio/install/build-tools-container?view=vs-2019

FROM mcr.microsoft.com/dotnet/framework/sdk:4.8-windowsservercore-ltsc2019

# Download the Build Tools bootstrapper.
ADD https://aka.ms/vs/16/release/vs_buildtools.exe vs_buildtools.exe

ADD https://aka.ms/vscollect.exe collect.exe

# Install Visual Studio
RUN start-process -wait vs_buildtools -ArgumentList \"--quiet --wait --norestart --nocache modify \
 --add Microsoft.VisualStudio.Component.CoreEditor \
 --add Microsoft.VisualStudio.Workload.CoreEditor \
 --add Microsoft.VisualStudio.Component.Roslyn.Compiler \
 --add Microsoft.Component.MSBuild \
 --add Microsoft.VisualStudio.Component.TextTemplating \
 --add Microsoft.VisualStudio.Component.VC.CoreIde \
 --add Microsoft.VisualStudio.Component.VC.Tools.x86.x64 \
 --add Microsoft.VisualStudio.Component.Windows10SDK.19041 \
 --add Microsoft.VisualStudio.Component.VC.Redist.14.Latest \
 --add Microsoft.VisualStudio.ComponentGroup.NativeDesktop.Core \
 --add Microsoft.VisualStudio.Workload.NativeDesktop \
 --add Microsoft.VisualStudio.Workload.WDExpress\"

RUN ./collect.exe -zip:C:\vslogs.zip

RUN & \"${env:programfiles(x86)}\Microsoft Visual Studio\Installer\vswhere.exe\" -latest -prerelease -requiresAny -property installationPath -products *

RUN & \"${env:programfiles(x86)}\Microsoft Visual Studio\Installer\vswhere.exe\" -latest -prerelease -requiresAny -property installationPath -products * -requires Microsoft.VisualStudio.Workload.WDExpress

RUN & \"${env:programfiles(x86)}\Microsoft Visual Studio\Installer\vswhere.exe\" -latest -prerelease -requiresAny -property installationPath -products * -requires Microsoft.VisualStudio.Component.VC.Tools.x86.x64

ENTRYPOINT ["powershell.exe", "-NoLogo", "-ExecutionPolicy", "Bypass"]

It emits this:

Step 4/7 : RUN & \"${env:programfiles(x86)}\Microsoft Visual Studio\Installer\vswhere.exe\" -latest -prerelease -requiresAny -property installationPath -products *
 ---> Running in aac125f47cb5
C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools
Removing intermediate container aac125f47cb5
 ---> 698aa2809536
Step 5/7 : RUN & \"${env:programfiles(x86)}\Microsoft Visual Studio\Installer\vswhere.exe\" -latest -prerelease -requiresAny -property installationPath -products * -requires Microsoft.VisualStudio.Workload.WDExpress
 ---> Running in ed20bbbb6ee4
Removing intermediate container ed20bbbb6ee4
 ---> 630be0c1e4f7
Step 6/7 : RUN & \"${env:programfiles(x86)}\Microsoft Visual Studio\Installer\vswhere.exe\" -latest -prerelease -requiresAny -property installationPath -products * -requires Microsoft.VisualStudio.Component.VC.Tools.x86.x64
 ---> Running in b82935c89875
Removing intermediate container b82935c89875
 ---> 2aa253f0879f

Note that when either of the components are required, the build is not found.

This issue provided some insight.