Skip to content

fix: harden Linux E2E scriptless hack and skip it in VHD-build pipelines#8794

Merged
r2k1 merged 1 commit into
mainfrom
r2k1-miniature-meme
Jun 29, 2026
Merged

fix: harden Linux E2E scriptless hack and skip it in VHD-build pipelines#8794
r2k1 merged 1 commit into
mainfrom
r2k1-miniature-meme

Conversation

@r2k1

@r2k1 r2k1 commented Jun 29, 2026

Copy link
Copy Markdown
Contributor

What this PR does / why we need it:

The scriptless E2E "hack" compiles aks-node-controller, uploads it to blob storage, and curls it onto the test node at provision time. It was failing in pipelines. Three fixes:

  1. Don't run the hack in pipelines that build the VHD. The Linux PR/release and Windows VHD-build pipelines just built a VHD that already contains the exact binary under test — re-downloading a freshly-compiled copy over the node's outbound path was an unnecessary failure point. The Linux PR builder and Windows template already set DISABLE_SCRIPTLESS_COMPILATION: true; this adds it to the release Linux builder, which was the remaining gap. Standalone E2E pipelines (e2e.yaml, e2e-gpu.yaml, etc.) test published VHDs and intentionally keep the hack to exercise the PR's source.

  2. Bump the SAS link expiry 1h → 6h. The signed download link is cached and reused across the whole suite; a 1-hour expiry lapsed mid-run on longer suites.

  3. Harden the curl download with --retry-connrefused. Note: deliberately not using --retry-all-errors — it requires curl ≥7.71.0, but the Ubuntu 20.04 FIPS VHDs ship curl 7.68.0 (and run scriptless_nbc subtests via the hack), so that flag would hard-fail provisioning with "unknown option". --retry-connrefused (curl 7.52.0+) is safe on every target VHD.

All changes are confined to e2e/ and pipeline YAML — no product provisioning scripts touched.

Which issue(s) this PR fixes:

Fixes #

The scriptless E2E "hack" compiles aks-node-controller, uploads it to blob
storage, and curls it onto the test node at provision time. Two issues made
it fail in pipelines:

- The VHD-build pipelines (Linux PR/release, Windows) ran the hack to download
  a freshly-compiled binary even though the VHD they just built already
  contains that exact binary. Disable it there via
  DISABLE_SCRIPTLESS_COMPILATION so they test the baked-in binary. The release
  Linux pipeline was the one still missing the flag.
- The SAS link is cached and reused across the whole suite; a 1h expiry
  lapsed mid-run. Bump to 6h.
- Harden the curl download with --retry-connrefused (curl 7.52+, safe on all
  target VHDs including Ubuntu 20.04's curl 7.68).

Standalone E2E pipelines (e2e.yaml, e2e-gpu.yaml, etc.) test published VHDs
and intentionally keep the hack to exercise the PR's source.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR hardens the Linux E2E “scriptless” aks-node-controller compilation/download workaround and avoids running it in VHD-build release pipelines where the VHD already contains the binary under test, reducing flaky pipeline failures without touching production provisioning scripts.

Changes:

  • Adds --retry-connrefused to curl invocations used to download the hack binary during E2E provisioning.
  • Extends the blob SAS/user-delegation credential expiry from 1 hour to 6 hours to prevent mid-suite expiration.
  • Sets DISABLE_SCRIPTLESS_COMPILATION: true in the Linux VHD-builder release pipeline E2E stage to skip the compile+download hack.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.

File Description
e2e/vmss.go Makes the hack binary download more resilient by retrying connection-refused errors.
e2e/config/azure.go Extends signed-link expiration to cover longer E2E suites.
.pipelines/.vsts-vhd-builder-release.yaml Disables scriptless compilation in release VHD-build E2E stage to avoid unnecessary download failures.

@r2k1 r2k1 enabled auto-merge (squash) June 29, 2026 23:03
@r2k1 r2k1 merged commit 0e63f0d into main Jun 29, 2026
21 of 24 checks passed
@r2k1 r2k1 deleted the r2k1-miniature-meme branch June 29, 2026 23:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants