reliably use flox in CI by cullenmcdermott · Pull Request #2 · NVIDIA/aicr

cullenmcdermott · 2026-01-31T00:04:28Z

Summary

Standardizes all CI workflows on using Flox the source tool versions. Ensures we are using the same versions across CI and local dev

Motivation / Context

This is about standardization and ease of use for contributors.

Type of Change

Bug fix (non-breaking change that fixes an issue)
New feature (non-breaking change that adds functionality)
Breaking change (fix or feature that would cause existing functionality to change)
Documentation update
Refactoring (no functional changes)
Build/CI/tooling

Component(s) Affected

CLI (cmd/eidos, pkg/cli)
API server (cmd/eidosd, pkg/api, pkg/server)
Recipe engine / data (pkg/recipe)
Bundlers (pkg/bundler, pkg/component/*)
Collectors / snapshotter (pkg/collector, pkg/snapshotter)
Validator (pkg/validator)
Core libraries (pkg/errors, pkg/k8s)
Docs/examples (docs/, examples/)
Other: ____________

Implementation Notes

Testing

# Commands run (prefer `make qualify` for non-trivial changes)
make qualify

Risk Assessment

Low — Isolated change, well-tested, easy to revert
Medium — Touches multiple components or has broader impact
High — Breaking change, affects critical paths, or complex rollout

Rollout notes:

Checklist

Tests pass locally (make test with -race)
Linter passes (make lint)
I did not skip/disable tests to make CI green
I added/updated tests for new functionality
I updated docs if user-facing behavior changed
Changes follow existing patterns in the codebase
Commits are signed off (git commit -s) — DCO info

Copilot

Pull request overview

This PR migrates CI workflows from custom GitHub Actions with manually managed tool versions to Flox-based builds, simplifying dependency management and improving reliability.

Changes:

Replaced custom GitHub Actions (go-ci, go-build-release, load-versions, etc.) with Flox-based tooling
Added new Flox actions (setup-flox, flox-run) to manage development environment
Consolidated tool versions in .flox/env/manifest.toml instead of scattered YAML files

Reviewed changes

Copilot reviewed 12 out of 13 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
`.github/workflows/on-tag.yaml`	Replaced version loading and custom actions with Flox-based build/test/release steps
`.github/workflows/on-push.yaml`	Split monolithic test job into parallel test/lint/security-scan jobs using Flox
`.github/workflows/e2e-test.yaml`	Migrated E2E tests to run all commands via Flox instead of manual tool installation
`.github/actions/setup-flox/action.yml`	New action to install and configure Flox with Nix cache
`.github/actions/flox-run/action.yml`	New action to run commands inside Flox environment
`.github/actions/attest-image-from-tag/action.yml`	Updated to use Flox for crane instead of manual installation
`.flox/env/manifest.toml`	Added E2E testing tools (crane, ctlptl, helm, kind, kubectl, syft, tilt) to Flox environment
`.github/actions/setup-build-tools/action.yml`	Removed (replaced by Flox)
`.github/actions/load-versions/action.yml`	Removed (replaced by Flox)
`.github/actions/install-e2e-tools/action.yml`	Removed (replaced by Flox)
`.github/actions/go-ci/action.yml`	Removed (replaced by Flox)
`.github/actions/go-build-release/action.yml`	Removed (replaced by Flox)

Copilot

Pull request overview

Copilot reviewed 13 out of 14 changed files in this pull request and generated 1 comment.

Copilot

Pull request overview

Copilot reviewed 14 out of 15 changed files in this pull request and generated 1 comment.

Copilot

Pull request overview

Copilot reviewed 13 out of 14 changed files in this pull request and generated no new comments.

…dos into cullen/add-flox-in-ci

Copilot

Pull request overview

Copilot reviewed 14 out of 15 changed files in this pull request and generated no new comments.

mchmarny

I have some concerns about the divergence between local development and CI here. By introducing new tooling for PR qualification that differs from our standard make commands, we’re creating a 'works on my machine' gap for anyone not using Flox. To keep the barrier to entry low for new Go contributors, I think we should ensure our primary qualification path remains idiomatic and accessible without requiring additional toolchains.

We certainly can provide the Flow way as an alternative without making it must-have, right?

cullenmcdermott · 2026-01-31T14:51:03Z

I have some concerns about the divergence between local development and CI here. By introducing new tooling for PR qualification that differs from our standard make commands, we’re creating a 'works on my machine' gap for anyone not using Flox. To keep the barrier to entry low for new Go contributors, I think we should ensure our primary qualification path remains idiomatic and accessible without requiring additional toolchains.

We certainly can provide the Flow way as an alternative without making it must-have, right?

I think it would be beneficial for us to make flox the happy path. It provides much more consistency and reliability than any home grown tool we will make. Can keep it as a side thing if that's the consensus from the group but I have not had a team have a bad experience leveraging Flox to provide all of our tools. Its a single tool that can encapsulate everything else, this will really simplify onboarding and ensuring everyone has the right pieces installed at the right versions. I would like for us to at least give it a try and if after some time its not working well for us we can back it out.

@mchmarny

values.yaml sets gc.enable: true and registry.yaml configures GC nodeScheduling paths, but the health check only validated Master and Worker. Adds an assertion on the node-feature-discovery-gc Deployment so a broken GC is caught by the conformance run. Addresses @mchmarny review comment NVIDIA#2 on NVIDIA#518. Signed-off-by: Carlos Eduardo Arango Gutierrez <eduardoa@nvidia.com>

@mchmarny

values.yaml sets gc.enable: true and registry.yaml configures GC nodeScheduling paths, but the health check only validated Master and Worker. Adds an assertion on the node-feature-discovery-gc Deployment so a broken GC is caught by the conformance run. Addresses @mchmarny review comment NVIDIA#2 on NVIDIA#518. Signed-off-by: Carlos Eduardo Arango Gutierrez <eduardoa@nvidia.com>

@mchmarny

values.yaml sets gc.enable: true and registry.yaml configures GC nodeScheduling paths, but the health check only validated Master and Worker. Adds an assertion on the node-feature-discovery-gc Deployment so a broken GC is caught by the conformance run. Addresses @mchmarny review comment NVIDIA#2 on NVIDIA#518. Signed-off-by: Carlos Eduardo Arango Gutierrez <eduardoa@nvidia.com>

@mchmarny

values.yaml sets gc.enable: true and registry.yaml configures GC nodeScheduling paths, but the health check only validated Master and Worker. Adds an assertion on the node-feature-discovery-gc Deployment so a broken GC is caught by the conformance run. Addresses @mchmarny review comment NVIDIA#2 on NVIDIA#518. Signed-off-by: Carlos Eduardo Arango Gutierrez <eduardoa@nvidia.com>

@mchmarny

values.yaml sets gc.enable: true and registry.yaml configures GC nodeScheduling paths, but the health check only validated Master and Worker. Adds an assertion on the node-feature-discovery-gc Deployment so a broken GC is caught by the conformance run. Addresses @mchmarny review comment NVIDIA#2 on NVIDIA#518. Signed-off-by: Carlos Eduardo Arango Gutierrez <eduardoa@nvidia.com>

The `nvsentinel` registry entry declared: defaultRepository: https://helm.ngc.nvidia.com/nvidia defaultChart: nvidia/nvsentinel But the chart isn't published to the HTTPS NGC index — only to the OCI registry at `oci://ghcr.io/nvidia/nvsentinel`. The defaults are silently ignored today: every nvsentinel-using overlay sets its own `source: oci://ghcr.io/nvidia` + chart `nvsentinel`, so the broken HTTPS default never resolves. But anyone relying on the registry defaults (e.g. via `aicr bundle` without explicit overlay overrides on this entry) would hit the dead path. Update the defaults to match what every overlay already uses: defaultRepository: oci://ghcr.io/nvidia defaultChart: nvsentinel Same shape as the kai-scheduler entry post-NVIDIA#720 (OCI registry path in `defaultRepository`, bare chart name in `defaultChart`). Verified locally: $ helm pull oci://ghcr.io/nvidia/nvsentinel --version v1.3.0 Pulled. $ aicr bundle -r recipe.yaml -o /tmp/bundle ... generates upstream.env with CHART='oci://ghcr.io/nvidia/nvsentinel' REPO='' VERSION='v1.3.0' Note: other NGC HTTPS entries in the registry (gpu-operator, network-operator, nodewright-operator, nvidia-dra-driver-gpu) are unchanged — those charts are genuinely served by the HTTPS NGC index. nvsentinel is special because it ships only via OCI. Refs: NVIDIA#698 (Phase 1 follow-up NVIDIA#2)

github-actions · 2026-05-04T06:59:16Z

This pull request has been automatically locked since it has been closed for 90 days with no further activity. Please open a new pull request for related changes.

Copilot AI review requested due to automatic review settings January 31, 2026 00:04

Copilot AI reviewed Jan 31, 2026

View reviewed changes

Comment thread .github/workflows/on-tag.yaml

Comment thread .github/workflows/on-push.yaml Outdated

cullenmcdermott commented Jan 31, 2026

View reviewed changes

Comment thread .github/workflows/on-push.yaml

Copilot AI review requested due to automatic review settings January 31, 2026 00:13

Copilot AI reviewed Jan 31, 2026

View reviewed changes

Comment thread .github/actions/flox-run/action.yml

Copilot AI review requested due to automatic review settings January 31, 2026 00:43

Copilot AI reviewed Jan 31, 2026

View reviewed changes

Comment thread .github/workflows/on-tag.yaml

cullenmcdermott added 4 commits January 30, 2026 17:46

reliably use flox in CI

7bc74f6

remove comment

528a184

allow webfetch from github and address copilot review

20a592f

Don't use blocked actions

938cbea

cullenmcdermott force-pushed the cullen/add-flox-in-ci branch from 19abdc4 to 938cbea Compare January 31, 2026 00:47

Merge branch 'main' into cullen/add-flox-in-ci

5454df4

Copilot AI review requested due to automatic review settings January 31, 2026 00:48

Copilot AI reviewed Jan 31, 2026

View reviewed changes

cullenmcdermott added 2 commits January 30, 2026 18:01

small fixes

b77aa54

Merge branch 'cullen/add-flox-in-ci' of github.com:cullenmcdermott/ei…

87cd119

…dos into cullen/add-flox-in-ci

cullenmcdermott requested a review from Copilot January 31, 2026 01:03

Copilot AI reviewed Jan 31, 2026

View reviewed changes

cache flox and nix-magic-cache

a701ae3

mchmarny self-requested a review January 31, 2026 11:50

mchmarny requested changes Jan 31, 2026

View reviewed changes

cullenmcdermott closed this Feb 2, 2026

dims mentioned this pull request Feb 22, 2026

feat(validator): add Go-based CNCF AI conformance checks #180

Merged

8 tasks

This was referenced Mar 9, 2026

feat(validation): container-per-validator execution engine #290

Merged

fix(validator): add retry for ai-service-metrics Prometheus query #393

Merged

yuanchen8911 mentioned this pull request Apr 20, 2026

enhance(validator): add targeted post-deployment GPU readiness checks #611

Merged

25 tasks

lockwobr mentioned this pull request Apr 24, 2026

API /v1/bundle double-wraps INVALID_REQUEST as INTERNAL 500 (retryable: true) #679

Closed

yuanchen8911 mentioned this pull request Apr 30, 2026

fix(recipes): correct nvsentinel registry default to OCI source #725

Merged

10 tasks

github-actions Bot locked as resolved and limited conversation to collaborators May 4, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

reliably use flox in CI#2

reliably use flox in CI#2
cullenmcdermott wants to merge 8 commits into
NVIDIA:mainfrom
cullenmcdermott:cullen/add-flox-in-ci

cullenmcdermott commented Jan 31, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI left a comment

Uh oh!

mchmarny left a comment •

edited

Loading

Uh oh!

cullenmcdermott commented Jan 31, 2026

Uh oh!

github-actions Bot commented May 4, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

cullenmcdermott commented Jan 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Motivation / Context

Type of Change

Component(s) Affected

Implementation Notes

Testing

Risk Assessment

Checklist

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

mchmarny left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

cullenmcdermott commented Jan 31, 2026

Uh oh!

github-actions Bot commented May 4, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

cullenmcdermott commented Jan 31, 2026 •

edited

Loading

mchmarny left a comment •

edited

Loading