Skip to content

[CI][Github] Prune windows container #141440

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged

Conversation

boomanaiden154
Copy link
Contributor

This patch partially prunes the windows container to reduce the image size, primarily to improve image pull time which is currently a pretty significant bottleneck in the new premerge due to autoscaling.

This patch removes the following:

  • An extra copy of LLVM that is not needed anymore.
  • An unneeded perl installation
  • Some extra python packages that are specific to buildbot

This overall saves about 4GB on the uncompressed image, or about 20%.

I tested this locally against the premerge pipeline and everything passes.

There are still several significant areas of opportunity, namely seeing if we can move away from the 4.8 sdk image to just the windowsservercore image (about 7GB of opportunity), and shrinking the VS installation (in total about 5GB uncompressed currently opportunity unknown).

This patch partially prunes the windows container to reduce the image
size, primarily to improve image pull time which is currently a pretty
significant bottleneck in the new premerge due to autoscaling.

This patch removes the following:
- An extra copy of LLVM that is not needed anymore.
- An unneeded perl installation
- Some extra python packages that are specific to buildbot

This overall saves about 4GB on the uncompressed image, or about 20%.
@llvmbot
Copy link
Member

llvmbot commented May 26, 2025

@llvm/pr-subscribers-github-workflow

Author: Aiden Grossman (boomanaiden154)

Changes

This patch partially prunes the windows container to reduce the image size, primarily to improve image pull time which is currently a pretty significant bottleneck in the new premerge due to autoscaling.

This patch removes the following:

  • An extra copy of LLVM that is not needed anymore.
  • An unneeded perl installation
  • Some extra python packages that are specific to buildbot

This overall saves about 4GB on the uncompressed image, or about 20%.

I tested this locally against the premerge pipeline and everything passes.

There are still several significant areas of opportunity, namely seeing if we can move away from the 4.8 sdk image to just the windowsservercore image (about 7GB of opportunity), and shrinking the VS installation (in total about 5GB uncompressed currently opportunity unknown).


Full diff: https://github.com/llvm/llvm-project/pull/141440.diff

1 Files Affected:

  • (modified) .github/workflows/containers/github-action-ci-windows/Dockerfile (+2-25)
diff --git a/.github/workflows/containers/github-action-ci-windows/Dockerfile b/.github/workflows/containers/github-action-ci-windows/Dockerfile
index 6f821b4809b7a..09cda4bb574b9 100644
--- a/.github/workflows/containers/github-action-ci-windows/Dockerfile
+++ b/.github/workflows/containers/github-action-ci-windows/Dockerfile
@@ -39,33 +39,15 @@ RUN regsvr32 /S "C:\BuildTools\DIA SDK\bin\amd64\msdia140.dll" & \
 
 # install tools as described in https://llvm.org/docs/GettingStartedVS.html
 # and a few more that were not documented...
-RUN choco install -y ninja git
+RUN choco install -y ninja git sccache
 # Pin an older version of Python; the current Python 3.10 fails when
 # doing "pip install" for the other dependencies, as it fails to find libxml
 # while compiling some package.
 RUN choco install -y python3 --version 3.9.7
 
-# ActivePerl is currently not installable via Chocolatey, see
-# http://disq.us/p/2ipditb. Install StrawberryPerl instead. Unfortunately,
-# StrawberryPerl not only installs Perl, but also a redundant C/C++ compiler
-# toolchain, and a copy of pkg-config which can cause misdetections for other
-# built products, see
-# https://github.com/StrawberryPerl/Perl-Dist-Strawberry/issues/11 for further
-# details. Remove the redundant and unnecessary parts of the StrawberryPerl
-# install.
-RUN choco install -y strawberryperl && \
-    rmdir /q /s c:\strawberry\c && \
-    del /q c:\strawberry\perl\bin\pkg-config*
-
-# libcxx requires clang(-cl) to be available
-RUN choco install -y sccache llvm
+# Testing requires psutil
 RUN pip install psutil
 
-RUN curl -LO https://github.com/mstorsjo/llvm-mingw/releases/download/20230320/llvm-mingw-20230320-ucrt-x86_64.zip && \
-    powershell Expand-Archive llvm-mingw-*-ucrt-x86_64.zip -DestinationPath . && \
-    del llvm-mingw-*-ucrt-x86_64.zip && \
-    ren llvm-mingw-20230320-ucrt-x86_64 llvm-mingw
-
 # configure Python encoding
 ENV PYTHONIOENCODING=UTF-8
 
@@ -103,11 +85,6 @@ RUN powershell -Command \
 RUN git config --system core.longpaths true & \
     git config --global core.autocrlf false
 
-# handle for debugging of files beeing locked by some processes.
-RUN choco install -y handle
-
-RUN pip3 install pywin32 buildbot-worker==2.8.4
-
 ARG RUNNER_VERSION=2.324.0
 ENV RUNNER_VERSION=$RUNNER_VERSION
 

@boomanaiden154 boomanaiden154 merged commit c02e9c8 into llvm:main May 26, 2025
13 checks passed
@boomanaiden154 boomanaiden154 deleted the prune-windows-container-parts branch May 26, 2025 02:36
sivan-shani pushed a commit to sivan-shani/llvm-project that referenced this pull request Jun 3, 2025
This patch partially prunes the windows container to reduce the image
size, primarily to improve image pull time which is currently a pretty
significant bottleneck in the new premerge due to autoscaling.

This patch removes the following:
- An extra copy of LLVM that is not needed anymore.
- An unneeded perl installation
- Some extra python packages that are specific to buildbot

This overall saves about 4GB on the uncompressed image, or about 20%.

I tested this locally against the premerge pipeline and everything
passes.

There are still several significant areas of opportunity, namely seeing
if we can move away from the 4.8 sdk image to just the
`windowsservercore` image (about 7GB of opportunity), and shrinking the
VS installation (in total about 5GB uncompressed currently opportunity
unknown).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants