Manually rebuild eso chart - #22
Merged
Merged
Conversation
Signed-off-by: Nick Lathe <nick.lathe@code.org>
carl-codeorg
approved these changes
Aug 7, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
TL;DR
There's a fair amount of Claude generated PR description below that is helpful, but difficult to follow. Here's the TL;DR:
apps/infra/standard-envtypes/chart/charts/eso-per-env-0.1.0.tgzis what Argo (via Helm) uses, however this is stale and hasn't been rebuilt to include the previous PR changes (it also includes a couple other PR changes, as it was 3 commits behind since being last rebuilt)Why
PR #21 added a second
dataFromentry to the per-namespaceExternalSecretsothat
CfnStack/<env>/*secrets sync intocdo-external-secrets. It merged, Argosynced it, and nothing happened.
The evidence:
kubectl get application standard-envtypes -n argocd -o jsonpath='{.status.sync.status}{.status.health.status}'Synced Healthy
kubectl get application standard-envtypes -n argocd -o jsonpath='{.status.sync.revisions}'["4a22fdf...","4a22fdf..."] # the merge commit of #21
git show 4a22fdf:apps/infra/charts/eso-per-env/templates/_envtype.tpl | grep -c CfnStackthe change is in the source at that revision
kubectl get externalsecret cdo-external-secrets -n staging -o jsonpath='{.spec.dataFrom[*].find.path}'staging/cdo/ # ...but only one path is live
Synced to the correct commit, correct source at that commit, wrong rendered output.
The cause is
apps/infra/standard-envtypes/chart/charts/eso-per-envtype-0.1.0.tgz.Helm renders subcharts from the
charts/directory, not from thefile://sourcepath, so the committed archive is what actually produces manifests. argocd-repo-server
does run
helm dependency buildbefore templating — but that command only fetches orpackages dependencies that are missing. The archive was already present at the
version pinned in
Chart.lock, so helm left it untouched and rendered a copy of_envtype.tpllast packaged in April.Every layer behaved correctly and the net result was that source edits to
_envtype.tplsilently did nothing.What
Repackage the vendored subchart from source:
cd apps/infra/standard-envtypes/chart && rm -f charts/*.tgz && helm dependency buildOne binary file changes.
Chart.lockis untouched — the subchart version isunchanged at 0.1.0, so this is a repackage, not a version bump.
Reviewing a binary diff
The archive is tracked via Git LFS (
*.tgz filter=lfs), so the diff is unreadable.To see what's inside:
tar xzOf apps/infra/standard-envtypes/chart/charts/eso-per-envtype-0.1.0.tgzeso-per-envtype/templates/_envtype.tpl | diff - apps/infra/charts/eso-per-env/templates/_envtype.tplThat should now be empty. Before this PR it differed by three commits' worth of edits.
What else ships with this
The archive was stale by three commits, not one, so rebuilding also lands the two
that preceded #21:
02b7896— addedcreationPolicy/deletionPolicy/conversionStrategy/decodingStrategyto the manifests1021fba— removed them againNet effect: those four fields disappear from the desired state. All four are ESO
CRD defaults, so the API server repopulates them on the live object and there is no
behavioral change — but Argo may report drift on them at first sync if the existing
managed-fields ignores don't cover them. Cosmetic if it happens.
Verification
After sync, allow one 5m ESO
refreshInterval, then:kubectl get externalsecret cdo-external-secrets -n staging -o jsonpath='{.spec.dataFrom[*].find.path}'expect: staging/cdo/ CfnStack/staging/
kubectl get secret cdo-external-secrets -n staging -o jsonpath='{.data}' | jq -r 'keys[]' | grep db_endpointexpect: db_endpoint_proxy_reader, db_endpoint_proxy_reporting, etc.
kubectl rollout restart deploy/cdo-active-job-worker -n stagingThe restart is required —
envFromvalues are injected once at container start.Confirmed locally: rendering the chart directory as-is with the old archive produced
zero
CfnStackreferences; with the rebuilt archive it produces the expected twodataFrompaths for staging, test, levelbuilder, and production, with the adhocClusterExternalSecretunchanged.Follow-up
This repo is in a half-vendored state: the archive is tracked so it wins, but nothing
regenerates it, so template edits are silent no-ops. Three commits died this way before
anyone noticed.
eso-per-envtypeis a library chart with one template file, one consumer, in the samerepo, referenced by
file://. Vendoring buys nothing and costs a lockfile, an LFSbinary, and this failure mode. Next PR moves
_envtype.tplintoapps/infra/standard-envtypes/chart/templates/and deletes the subchart, thedependency,
Chart.lock, andcharts/— which should render byte-identically.Until that lands, anyone editing
_envtype.tplmust rebuild the archive in the samecommit.
🤖 Generated with Claude Code