Skip to content

ci: switch PR workflows back to 1ES self-hosted runners with JobId#313128

Merged
Yoyokrazy merged 2 commits intomainfrom
milively/relight-1es-runners
Apr 29, 2026
Merged

ci: switch PR workflows back to 1ES self-hosted runners with JobId#313128
Yoyokrazy merged 2 commits intomainfrom
milively/relight-1es-runners

Conversation

@Yoyokrazy
Copy link
Copy Markdown
Collaborator

Summary

Re-applies #311975 (reverted in #312033). Switches PR workflow Ubuntu/Windows jobs back to the 1ES self-hosted pools and adds a per-run+attempt JobId label to scope each agent to a specific GitHub Actions run, preventing the intermittent 1ES cancellations seen previously.

This time, also wires the pr.yml compile job's GITHUB_TOKEN back to secrets.VSCODE_OSS (with the historical fork-aware fallback). On 1ES pools, the shared egress IPs hit the anonymous 60/hr `api.github.com` rate limit during cross-repo release fetches (`vscode-js-debug`, `vscode-js-debug-companion`, `vscode-js-profile-visualizer`, …), producing 403 fan-out across PRs — that was the actual cause of the failures that triggered the previous revert, not 1ES itself.

Session Context

Investigation of the previous revert (#312033):

  • Root cause was GitHub API rate limiting, not auth/secret rotation. Failed runs during the ci: switch PR workflows back to 1ES self-hosted runners with JobId #311975 window logged `status code: 403 (you may be rate limited)` against `api.github.com/repos/microsoft/vscode-js-*`. `secrets.GITHUB_TOKEN` is repo-scoped and can't auth cross-repo, so `build/lib/fetch.ts` falls back to anonymous; on 1ES pools concurrent jobs share egress IPs and blow through the 60/hr anonymous limit immediately. GH-hosted runners didn't see this because each runner gets a fresh IP from a huge pool.
  • `VSCODE_OSS` already exists as a repo secret (last rotated 2026-03-26) and is already used elsewhere in CI for the same cross-repo fetches — see `pr-node-modules.yml` (6 spots). No new secret needed.
  • Fork-aware conditional preserved. Matches the pattern that existed in `pr.yml` before Engineering - more github action cleanup #255987 cleanup: `head.repo.full_name == github.repository && secrets.VSCODE_OSS || secrets.GITHUB_TOKEN`. Forks can't read org/repo secrets, so they fall back to the runner-issued token.
  • JobId label format unchanged from ci: switch PR workflows back to 1ES self-hosted runners with JobId #311975: `JobId=-${{ github.run_id }}-${{ github.run_number }}-${{ github.run_attempt }}`. Reusable workflows include `inputs.job_name` to disambiguate.

Pools (unchanged):

  • `1es-vscode-oss-ubuntu-22.04-x64`
  • `1es-vscode-oss-windows-2022-x64`

Changes

  • `pr.yml`: `compile`, `copilot-check-test-cache`, `copilot-check-telemetry`, `copilot-linux-tests`, `copilot-windows-tests` → 1ES + JobId. `compile` job's `GITHUB_TOKEN` switched to `VSCODE_OSS` (fork-conditional) on the `Install dependencies` and `Compile & Hygiene` steps.
  • `pr-linux-cli-test.yml`, `pr-win32-test.yml`: reusable test workflows back on 1ES + JobId.
  • `pr-node-modules.yml`: `compile`, `linux` back on 1ES + JobId; `windows` (already 1ES) gains JobId.

Note: `pr-linux-test.yml` is intentionally left on `ubuntu-24.04` — that move happened after #311975 to address a fontconfig SIGSEGV.

Closes follow-up to #312033.

Re-applies #311975 (reverted in #312033). Adds per-run+attempt JobId
labels to scope 1ES agents to specific GitHub Actions runs and prevent
intermittent runner cancellations.

Also switches the pr.yml compile job's GITHUB_TOKEN from the
ephemeral repo-scoped runner token to secrets.VSCODE_OSS so cross-repo
GitHub API release fetches (vscode-js-debug, vscode-js-debug-companion,
vscode-js-profile-visualizer, etc.) authenticate properly. On 1ES pools
the shared egress IPs hit the anonymous 60/hr api.github.com rate limit
and produced 403 fan-out across PRs last time.
Match the historical pattern from before #255987 — fork PRs can't
access secrets.VSCODE_OSS, so use the conditional to pick GITHUB_TOKEN
for forks.
Copilot AI review requested due to automatic review settings April 28, 2026 22:34
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Switches the PR GitHub Actions workflows back to 1ES self-hosted runner pools and scopes each job to a unique agent via a per-run/attempt JobId label, while also restoring authenticated GitHub API access for the PR compile path to avoid anonymous rate limiting on shared 1ES egress IPs.

Changes:

  • Update runs-on for PR and reusable test workflows to target 1ES pools and add a unique JobId=... label per job.
  • In pr.yml’s compile job, route GITHUB_TOKEN to secrets.VSCODE_OSS for same-repo PRs (with fork fallback to secrets.GITHUB_TOKEN) on the dependency install and compile/hygiene steps.
  • Apply the same 1ES + JobId runner selection to the node-modules workflow jobs.
Show a summary per file
File Description
.github/workflows/pr.yml Move selected PR jobs to 1ES runners with JobId; use VSCODE_OSS for authenticated GitHub API access in compile-related steps with fork-aware fallback.
.github/workflows/pr-win32-test.yml Run reusable Windows PR tests on 1ES with a per-invocation JobId including inputs.job_name for disambiguation.
.github/workflows/pr-linux-cli-test.yml Run reusable Linux CLI tests on 1ES with a per-invocation JobId including inputs.job_name.
.github/workflows/pr-node-modules.yml Run node-modules CI jobs on 1ES with per-job JobId labels (including adding it to the existing 1ES Windows job).

Copilot's findings

  • Files reviewed: 4/4 changed files
  • Comments generated: 0

@rzhao271 rzhao271 added this to the 1.119.0 milestone Apr 29, 2026
@Yoyokrazy Yoyokrazy marked this pull request as ready for review April 29, 2026 17:51
@Yoyokrazy Yoyokrazy merged commit 7a833e1 into main Apr 29, 2026
30 checks passed
@Yoyokrazy Yoyokrazy deleted the milively/relight-1es-runners branch April 29, 2026 17:51
Yoyokrazy added a commit that referenced this pull request Apr 29, 2026
Follow-up to #313128. The VSCODE_OSS fallback isn't needed for the
api.github.com calls in core-ci — secrets.GITHUB_TOKEN already
authenticates those reads with permissions: contents: read (added in
#304929), so we don't hit the anonymous rate limit on 1ES.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants