Skip to content

fix(ci): a staged-off component's chart is never rendered, so no rendered-children check validates itΒ #3498

Description

@devantler

πŸ€– Generated by the Agentic Engineer

Evidence

Raised by Codex on #3476 while reviewing the staged-off first revision of the
data-product-controller. The broad form of the finding is not correct, and the narrow form is β€”
both measured with ksail workload validate against the PR head.

Staged-off manifests ARE schema-validated. Both ksail configs note that workload validate
schema-checks unreferenced files standalone, and that is real rather than cosmetic:

  • baseline: all 9 k8s/bases/apps/data-product-controller/*.yaml files report βœ” … validated, and
    fleetdm (also staged off) contributes 30 further validated lines;
  • ablation A β€” an unknown field (bogusFieldThatCannotExist) in a staged-off manifest was not
    caught (exit 0). That is kubeconform running non-strict, and it applies to every manifest in the
    repository, not just staged-off ones;
  • ablation B β€” a type violation on a known field (spec.selector as a string instead of a
    LabelSelector) in the same staged-off file was caught: βœ— bases/apps/data-product-controller failed, exit 1.

So schema validation genuinely reaches a staged-off component. Ablation A is recorded because it is
the reason ablation B was needed β€” the first attempt did not fire, and a control that does not fire
proves nothing.

The gap

What does not reach it is the Helm render, and therefore every check that operates on rendered
chart children.

Discriminator: point the staged-off HelmRelease's chartRef.name at an OCIRepository that does not
exist. chartRef.name stays a valid string, so standalone schema validation is unaffected β€” only an
actual render would have to resolve it. ksail workload validate still reported
βœ” bases/apps/data-product-controller validated and exited 0.

This matters beyond schema coverage. The production CEL authorization rules are evaluated over
rendered children, and #3497 is about strengthening exactly that surface for
authorization-scope: isolated-chart charts. A staged-off component is invisible to both β€” so the
fix landing in #3497 would also not run for it. The "land it disabled, validate, then enable" sequence
therefore buys deploy safety (no workloads, no public route) but not render-level validation
evidence, and those are different claims.

Limitation of this measurement

The enabled-app control was not run: I did not break an enabled app's chartRef to confirm that
local ksail workload validate resolves chart references at all. Without it, the exit-0 result above
is consistent with either "staged-off components are not rendered" or "local validation never renders
any chart". The repository's own CI comments state that rendering happens (the prod validate step
carries timeout-minutes: 10 because it "reaches every upstream chart registry"), but that is
documentation, not measurement. Run that control before relying on the attribution.

Acceptance criteria

  1. A validation-only path renders a staged-off component's chart and evaluates the production CEL
    authorization rules over its children, while keeping it out of both deploy overlays.
  2. The enabled-app control above is run and its result recorded here, so the attribution is measured
    rather than inferred.
  3. A test proves the path has teeth: a deliberate violation in a staged-off component's rendered
    children fails validation, and the ablation is shown firing for the reason it names.
  4. Closes before the data-product-controller is activated, alongside fix(ci): isolated-chart exemption is not verified for rendered children's namespacesΒ #3497.

Part of the #3476 rollout.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions