devops(docker): split browser layers and use zstd compression for faster pulls#40702
devops(docker): split browser layers and use zstd compression for faster pulls#40702KRRT7 wants to merge 6 commits intomicrosoft:mainfrom
Conversation
The zstd compression output requires a buildx builder with the docker-container driver. Also removes the now-unused tag_and_push function.
Build pushes one canonical tag per arch, then tag_and_push creates additional tags via `docker buildx imagetools create` (registry-side copy, no layer re-upload).
Use `install-deps` for all system dependencies in one shared layer, then `install` (without --with-deps) per browser for binary-only layers. This eliminates duplicate apt index fetches and makes browser layers pure binary data with better zstd compression ratio.
|
Thanks for looking into it. I'm sure we have customers with older docker clients. But I'm ok with the layering change. Could you provide the numbers? Time w/ and w/o layers next to each other. |
… compatibility zstd-compressed layers require Docker 23.0+ (Feb 2023). Reverting to standard gzip keeps the parallel-layer speedup without breaking pulls for users on older Docker versions.
One-off workflow_dispatch to measure cold pull times for monolithic vs split-layer Docker images on a GitHub Actions runner.
|
Dropped the zstd change (latest commit), so no Docker version concerns anymore — this is purely the layer split. Layer structure (arm64, compressed):
Largest single layer drops from 754 MB → 321 MB. Benchmarks (Azure VM, D16s_v5, amd64, GHCR):
The split helps when the bottleneck is per-connection throughput (registry/CDN rate limits per stream — Docker downloads each layer on a separate connection). When the user's own pipe is the bottleneck, it's neutral — never worse. For CI runners with high total bandwidth but CDN-limited per-stream rates, this cuts the download portion significantly. On fast unthrottled connections, decompression dominates and both are equivalent. |
| @@ -0,0 +1,106 @@ | |||
| name: "benchmark: docker pull time" | |||
| uses: docker/setup-qemu-action@v4 | ||
| with: | ||
| platforms: arm64 | ||
| - name: Set up Docker Buildx |
| @@ -1,3 +1,4 @@ | |||
| # syntax=docker/dockerfile:1 | |||
| npm i /tmp/playwright-core.tar.gz && \ | ||
| npm exec --no -- playwright-core mark-docker-image "${DOCKER_IMAGE_NAME_TEMPLATE}" && \ | ||
| npm exec --no -- playwright-core install --with-deps && rm -rf /var/lib/apt/lists/* && \ | ||
| npm exec --no -- playwright-core install-deps chromium chromium-headless-shell firefox webkit && \ |
There was a problem hiding this comment.
Do not list browsers, install all deps.
| rm -rf /var/lib/apt/lists/* | ||
|
|
||
| # 3. Install each browser binary in its own layer for parallel pulling. | ||
| RUN cd /ms-playwright-agent && \ |
There was a problem hiding this comment.
Looks like you could benefit from splitting these too? chromium is your largest layer
| @@ -1,3 +1,4 @@ | |||
| # syntax=docker/dockerfile:1 | |||
| chmod -R 777 /ms-playwright/webkit-* | ||
|
|
||
| # 4. Install ffmpeg and clean up. | ||
| RUN cd /ms-playwright-agent && \ |
There was a problem hiding this comment.
First installed browser will do it, this layer will be empty.
| ./build.sh "--${ARCH}" "${FLAVOR}" playwright:localbuild | ||
|
|
||
| for ((i = 0; i < ${#TAGS[@]}; i++)) do | ||
| local CANONICAL_TAG="playwright.azurecr.io/public/${MCR_IMAGE_NAME}:${TAGS[0]}-${ARCH}" |
2 participants
Summary
RUNinto per-browser layers (chromium, firefox, webkit, ffmpeg) enabling Docker's parallel layer downloadingdocker build+docker pushtodocker buildx build --pushwith zstd compression (level 19) for ~25% smaller compressed layersdocker buildx imagetools create(registry-side copy, no layer re-upload)docker/setup-buildx-actionto CI (required for buildx + zstd output)Note on zstd compatibility
zstd-compressed layers require Docker 23.0+ (Feb 2023) on the client side. Older Docker versions will fail to pull. If this is a concern for your user base, the layer-splitting alone still provides a significant speedup via parallel downloads, and the zstd change could be dropped or deferred.
Fixes #40701