Skip to content

Inline Compile & CompileCLI stages into platform specific jobs#296006

Merged
joaomoreno merged 48 commits intomainfrom
joao/sapphire-ermine
Mar 1, 2026
Merged

Inline Compile & CompileCLI stages into platform specific jobs#296006
joaomoreno merged 48 commits intomainfrom
joao/sapphire-ermine

Conversation

@joaomoreno
Copy link
Member

@joaomoreno joaomoreno commented Feb 18, 2026

This PR eliminates both the Compile and the CompileCLI stage from our build, removing the overhead of transferring artifacts across jobs, which is now more than the time it takes to run the "meat" of those stages, now that they are faster (Compile now uses esbuild, CompileCLI now uses sccache).

We now have flat jobs, they can all start anew:

Product build:

image

CI build:

image

Also fixes #296465

joaomoreno and others added 11 commits February 17, 2026 12:48
Remove the standalone Compile stage from the ADO pipeline. Each
platform job (Windows, Linux, macOS, Alpine, Web) now compiles
TypeScript itself instead of downloading a shared Compilation artifact.

- Add VSCODE_RUN_CHECKS parameter to Linux jobs for hygiene/lint/CG
- Add VSCODE_RUN_COMPILE_EXTRAS parameter to macOS jobs for telemetry
  extraction and sourcemap upload
- Remove VSCODE_COMPILE_ONLY parameter entirely
- Delete product-compile.yml (no longer referenced)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The macOS Universal app merge requires both x64 and arm64 builds to have
identical file sets. Telemetry extraction was only running on arm64 (via
VSCODE_RUN_COMPILE_EXTRAS), causing the universal merge to fail due to
missing telemetry-core.json and telemetry-extensions.json in the x64 build.

Move telemetry extraction outside the VSCODE_RUN_COMPILE_EXTRAS gate so
it runs on all macOS builds. Sourcemap upload remains gated.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
All client builds (Linux, Windows, macOS) need telemetry-core.json and
telemetry-extensions.json. Previously only macOS arm64 ran extract-telemetry.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Linux arm64 and armhf are server-only builds, no need for telemetry files.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The bash extract-telemetry.sh script fails on Windows because the Unix
bin shim gets interpreted as Node.js code. Use a native PowerShell
implementation that calls the extractor JS entry point directly.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The telemetry extractor may skip emitting declarations-resolved.json
when no events are found. Handle this case with a warning instead of
failing the build.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
When each platform compiles independently, the build date embedded in
cli.js (via INSERT_PRODUCT_CONFIGURATION) differs between machines
because each uses new Date().toISOString(). This causes the macOS
Universal app merge to fail since cli.js SHA differs between x64/arm64.

Fix: use the git commit date (git log -1 --format=%cI HEAD) instead
of the current wall-clock time. This ensures all independent builds
on different machines produce identical timestamps.

Updated in:
- build/lib/date.ts: writeISODate() uses git commit date
- build/next/index.ts: bundle(), transpile, and readISODate fallback
  all use git commit date

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings February 18, 2026 13:49
@joaomoreno joaomoreno self-assigned this Feb 18, 2026
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR aims to make multi-machine/platform builds deterministic (notably for macOS Universal) by eliminating wall-clock build timestamps and by inlining the former shared “Compile” job into each platform-specific pipeline job.

Changes:

  • Switch build timestamp generation to use the git commit date (with wall-clock fallback) and reuse out-build/date where available.
  • Inline compilation into platform pipelines and remove the shared compilation artifact/job, updating stage wiring accordingly.
  • Adjust macOS universal merge behavior and some hygiene filters to accommodate the new pipeline outputs.

Reviewed changes

Copilot reviewed 17 out of 17 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
build/next/index.ts Uses git commit date for build timestamps; normalizes Windows paths in extension host bundle detection; reuses out-build/date when bundling.
build/lib/date.ts Updates writeISODate() to use git commit date for deterministic timestamps.
build/filters.ts Excludes additional paths/files from indentation hygiene.
build/darwin/create-universal-app.ts Skips universal-merge comparison for telemetry JSON files.
build/azure-pipelines/win32/steps/product-build-win32-compile.yml Removes downloading compile artifact; compiles in-place; validates outputs; generates telemetry lists.
build/azure-pipelines/web/product-build-web.yml Removes downloading compile artifact; compiles in-place; validates outputs.
build/azure-pipelines/product-compile.yml Removes the shared Compile job (deleted).
build/azure-pipelines/product-build.yml Removes Compile stage and VSCODE_COMPILE_ONLY; updates stage dependencies/flags for new per-platform compilation.
build/azure-pipelines/product-build-macos.yml Removes Compile stage dependency; enables compile extras in macOS CI.
build/azure-pipelines/linux/steps/product-build-linux-compile.yml Inlines compilation; optionally runs full checks; validates outputs; generates telemetry on x64.
build/azure-pipelines/linux/product-build-linux.yml Plumbs VSCODE_RUN_CHECKS into compile steps.
build/azure-pipelines/linux/product-build-linux-ci.yml Enables VSCODE_RUN_CHECKS for Linux CI job variant.
build/azure-pipelines/darwin/steps/product-build-darwin-compile.yml Inlines compilation; validates outputs; generates telemetry; optionally uploads sourcemaps via “compile extras”.
build/azure-pipelines/darwin/product-build-darwin.yml Plumbs VSCODE_RUN_COMPILE_EXTRAS into compile steps.
build/azure-pipelines/darwin/product-build-darwin-ci.yml Enables VSCODE_RUN_COMPILE_EXTRAS for macOS CI job variant.
build/azure-pipelines/common/publish.ts Updates stage tracking to reflect removal of the Compile stage.
build/azure-pipelines/alpine/product-build-alpine.yml Removes downloading compile artifact; compiles in-place; validates outputs.

joaomoreno and others added 10 commits February 18, 2026 15:20
core-ci (esbuild path) already includes cleanExtensionsBuildTask,
compileNonNativeExtensionsBuildTask, and compileExtensionMediaBuildTask.
Running extensions-ci in parallel caused a race condition where
core-ci's rimraf of .build/extensions clashed with extensions-ci
writing to the same directory.

Also removes dead code:
- extensions-ci and extensions-ci-pr task definitions (fully subsumed)
- core-ci-old task (useEsbuildTranspile is always true)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…` function by exporting a single helper function from `build/lib/date.ts` and importing it in `build/next/index.ts` to ensure consistent behavior and improve code maintainability.
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fyi @osortega

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fyi @aeschli

- Remove centralized CompileCLI stage
- Move CLI jobs into Windows, Linux, macOS stages as independent jobs
- CLI jobs now compile, publish unsigned mid-job, sign, and publish signed
- Platform compile jobs use deemon + waitForArtifacts for async CLI download
- Delete separate CLI sign jobs (now merged into CLI compile jobs)
- Remove CompileCLI from publish.ts stage tracking
@joaomoreno joaomoreno changed the title Inline Compile job into platform specific jobs Inline Compile & CompileCLI stages into platform specific jobs Feb 21, 2026
@joaomoreno joaomoreno added this to the March 2026 milestone Feb 27, 2026
@joaomoreno joaomoreno added feature-request Request for new features or functionality engineering VS Code - Build / issue tracking / etc. labels Feb 27, 2026
@joaomoreno
Copy link
Member Author

joaomoreno commented Feb 27, 2026

@joaomoreno joaomoreno marked this pull request as ready for review February 28, 2026 19:35
@joaomoreno joaomoreno enabled auto-merge (squash) February 28, 2026 20:50
@joaomoreno joaomoreno merged commit d2f1f7c into main Mar 1, 2026
91 checks passed
@joaomoreno joaomoreno deleted the joao/sapphire-ermine branch March 1, 2026 07:48
DonJayamanne pushed a commit that referenced this pull request Mar 2, 2026
* chore: ralph loop checkpoint - 2026-02-17 12:48

* fix shell script

* Inline compile stage into platform jobs

Remove the standalone Compile stage from the ADO pipeline. Each
platform job (Windows, Linux, macOS, Alpine, Web) now compiles
TypeScript itself instead of downloading a shared Compilation artifact.

- Add VSCODE_RUN_CHECKS parameter to Linux jobs for hygiene/lint/CG
- Add VSCODE_RUN_COMPILE_EXTRAS parameter to macOS jobs for telemetry
  extraction and sourcemap upload
- Remove VSCODE_COMPILE_ONLY parameter entirely
- Delete product-compile.yml (no longer referenced)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Always run telemetry extraction on macOS builds

The macOS Universal app merge requires both x64 and arm64 builds to have
identical file sets. Telemetry extraction was only running on arm64 (via
VSCODE_RUN_COMPILE_EXTRAS), causing the universal merge to fail due to
missing telemetry-core.json and telemetry-extensions.json in the x64 build.

Move telemetry extraction outside the VSCODE_RUN_COMPILE_EXTRAS gate so
it runs on all macOS builds. Sourcemap upload remains gated.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Run telemetry extraction on all client builds

All client builds (Linux, Windows, macOS) need telemetry-core.json and
telemetry-extensions.json. Previously only macOS arm64 ran extract-telemetry.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Only run telemetry extraction on Linux x64 (client build)

Linux arm64 and armhf are server-only builds, no need for telemetry files.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Remove .ralph scaffolding files

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Fix telemetry extraction on Windows with native PowerShell

The bash extract-telemetry.sh script fails on Windows because the Unix
bin shim gets interpreted as Node.js code. Use a native PowerShell
implementation that calls the extractor JS entry point directly.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Handle missing telemetry files gracefully on Windows

The telemetry extractor may skip emitting declarations-resolved.json
when no events are found. Handle this case with a warning instead of
failing the build.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Use deterministic build date from git commit timestamp

When each platform compiles independently, the build date embedded in
cli.js (via INSERT_PRODUCT_CONFIGURATION) differs between machines
because each uses new Date().toISOString(). This causes the macOS
Universal app merge to fail since cli.js SHA differs between x64/arm64.

Fix: use the git commit date (git log -1 --format=%cI HEAD) instead
of the current wall-clock time. This ensures all independent builds
on different machines produce identical timestamps.

Updated in:
- build/lib/date.ts: writeISODate() uses git commit date
- build/next/index.ts: bundle(), transpile, and readISODate fallback
  all use git commit date

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Remove redundant extensions-ci task from CI pipelines

core-ci (esbuild path) already includes cleanExtensionsBuildTask,
compileNonNativeExtensionsBuildTask, and compileExtensionMediaBuildTask.
Running extensions-ci in parallel caused a race condition where
core-ci's rimraf of .build/extensions clashed with extensions-ci
writing to the same directory.

Also removes dead code:
- extensions-ci and extensions-ci-pr task definitions (fully subsumed)
- core-ci-old task (useEsbuildTranspile is always true)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* remove VSCODE_RUN_COMPILE_EXTRAS

* address PR feedback regarding code duplication of `getGitCommitDate()` function by exporting a single helper function from `build/lib/date.ts` and importing it in `build/next/index.ts` to ensure consistent behavior and improve code maintainability.

* update readISODate function to return git commit date instead of current date

* add telemetry sorting script and integrate into build process for consistent output

* refactor telemetry extraction process: replace shell script with TypeScript implementation

* update skill

* update telemetry-extractor dependency to version 1.19.0

* fix build

* fix more duplicate telemetry definition issues

* cleanup

* refactor: consolidate validation checks into quality checks and remove obsolete tasks

* bust the cache

* undo cache bust

* fix expression

* fix

* fix: update Azure storage account name in quality checks

* fix: initialize stages set with 'Quality'

* fix: add VSCODE_BUILD_TYPE parameter with options for Product and CI builds

* fix: update Azure Pipeline CLI to use parameters instead of variables for queueing builds

* fix: update VSCODE_BUILD_TYPE parameter values for clarity

* fix: update default value for VSCODE_BUILD_TYPE parameter to 'Product'

* leaner

* even leaner

* only npm ci in build

* 💄

* run entire npm ci

* fix

* fix

* fix it

* Inline CompileCLI into platform stages

- Remove centralized CompileCLI stage
- Move CLI jobs into Windows, Linux, macOS stages as independent jobs
- CLI jobs now compile, publish unsigned mid-job, sign, and publish signed
- Platform compile jobs use deemon + waitForArtifacts for async CLI download
- Delete separate CLI sign jobs (now merged into CLI compile jobs)
- Remove CompileCLI from publish.ts stage tracking

* fix: macOS CLI signing - use proper directory structure for ESRP

* fix: add BUILDS_API_URL to Windows and Linux job templates

* fix: label Linux CLI jobs clearly

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: João Moreno <22350+joaomoreno@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

engineering VS Code - Build / issue tracking / etc. feature-request Request for new features or functionality

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Telemetry tooling fails to report inconsistent telemetry events

3 participants