Summary
Three launch-eve bugs (#226, #227, #229) all shipped past CI because no test runs CLI/controller-emitted Kubernetes objects through real apiserver validation.
What we missed
| PR |
Bug |
Why CI missed it |
| #226 |
ToolPolicy.spec.appliesTo.sandboxName (field doesn't exist) |
Tests asserted JS object shape against itself; no CRD-OpenAPI schema check |
| #227 |
InferencePolicy.spec.modelPreference.primary.endpoint (field doesn't exist; strict decoder rejects) |
Same as above; buildInferencePolicy not run through any apiserver |
| #229 |
Mirror ConfigMap label value contained / (K8s rejects) |
E2E covers ToolPolicy creation + ClawSandbox creation in isolation, but the e2e ClawSandbox has no toolPolicyRef so mirror_configmap is never invoked |
Fixes
1. CRD-OpenAPI validator in cli/test (~50 LOC)
Load deploy/helm/azureclaw/templates/crd-*.yaml, extract OpenAPI schemas, run every build* function in cli/src/refs.ts and cli/src/migrate/* through ajv. Catches #226 and #227 at PR time without needing a cluster.
2. Wire toolPolicyRef into the e2e ClawSandbox
In tests/e2e/run.sh:153 (test_create_sandbox):
- Apply a ToolPolicy alongside the InferencePolicy
- Add
toolPolicyRef: { name: e2e-test-toolpolicy } to the ClawSandbox spec
- After namespace appears, wait for the mirrored ConfigMap
toolpolicy-e2e-test-toolpolicy-profile in azureclaw-e2e-test ns
This exercises mirror_configmap and mirror_secret against a real apiserver — would have caught #229.
3. Fix the 3 deferred audit findings (separate PR)
From the earlier session audit:
from_kagent.ts:463 — allowedEndpoints[*].port missing (default to 443)
toolpolicy.ts:43-47 — CLI doesn't fail-fast on empty sandboxMatchLabels (kubectl rejects via CEL)
convert.ts:515-525 + migrate.ts:116-120 — spec.upstreamCompatibility field doesn't exist in ClawSandbox CRD (overlay/translate/observe modes silently broken)
The validator from (1) catches all three automatically.
Priority
Post-launch. None of these block today's release; they are CI hardening to prevent the same class of regression.
Summary
Three launch-eve bugs (#226, #227, #229) all shipped past CI because no test runs CLI/controller-emitted Kubernetes objects through real apiserver validation.
What we missed
ToolPolicy.spec.appliesTo.sandboxName(field doesn't exist)InferencePolicy.spec.modelPreference.primary.endpoint(field doesn't exist; strict decoder rejects)buildInferencePolicynot run through any apiserver/(K8s rejects)ClawSandboxhas notoolPolicyRefsomirror_configmapis never invokedFixes
1. CRD-OpenAPI validator in
cli/test(~50 LOC)Load
deploy/helm/azureclaw/templates/crd-*.yaml, extract OpenAPI schemas, run everybuild*function incli/src/refs.tsandcli/src/migrate/*through ajv. Catches #226 and #227 at PR time without needing a cluster.2. Wire
toolPolicyRefinto the e2e ClawSandboxIn
tests/e2e/run.sh:153(test_create_sandbox):toolPolicyRef: { name: e2e-test-toolpolicy }to the ClawSandbox spectoolpolicy-e2e-test-toolpolicy-profileinazureclaw-e2e-testnsThis exercises
mirror_configmapandmirror_secretagainst a real apiserver — would have caught #229.3. Fix the 3 deferred audit findings (separate PR)
From the earlier session audit:
from_kagent.ts:463—allowedEndpoints[*].portmissing (default to 443)toolpolicy.ts:43-47— CLI doesn't fail-fast on emptysandboxMatchLabels(kubectl rejects via CEL)convert.ts:515-525+migrate.ts:116-120—spec.upstreamCompatibilityfield doesn't exist in ClawSandbox CRD (overlay/translate/observe modes silently broken)The validator from (1) catches all three automatically.
Priority
Post-launch. None of these block today's release; they are CI hardening to prevent the same class of regression.