fix(k8s): stop hardcoding the CloudWatch logging annotations - #224
Merged
andres-nullplatform merged 1 commit intoAug 13, 2026
Merged
Conversation
andres-nullplatform
force-pushed
the
feat/configurable-cloudwatch-annotations
branch
from
August 13, 2026 21:20
b009479 to
4f290f4
Compare
Closes #223. Five annotations were literals in both deployment templates: the CloudWatch gate plus the log group, retention, stream name pattern and region. Every pod in every account therefore shipped to CloudWatch, under `<namespace>.<application>`, kept for exactly 7 days — and none of it could change without editing this repository. They are removed rather than made configurable here. All five are the logs controller's inputs, and whoever owns an account's logging setup already stamps annotations through k8s_modifiers.deployment.annotations; splitting them across two layers would mean two sources for one decision. Keeping them in one place is also what allows the gate and the naming to disagree in only one direction: a pod that nothing annotates ships nowhere, which is the right outcome for an account with no logging provider. The templates still render `annotations:` and the modifier merges, so an override supplies the same keys with the same values it used to get from here — verified against a real render. Also drops a bug: scheduled_task hardcoded `region: us-east-1` rather than deriving it from the context like the k8s template did, so every scheduled task outside that region named the wrong CloudWatch region. BREAKING: a cluster relying on these templates to stamp `nullplatform.logs.cloudwatch: 'true'` stops shipping application logs until an override injects the gate.
andres-nullplatform
force-pushed
the
feat/configurable-cloudwatch-annotations
branch
from
August 13, 2026 21:27
4f290f4 to
07dee78
Compare
fedemaleh
approved these changes
Aug 13, 2026
andres-nullplatform
deleted the
feat/configurable-cloudwatch-annotations
branch
August 13, 2026 21:31
andres-nullplatform
restored the
feat/configurable-cloudwatch-annotations
branch
August 13, 2026 21:37
andres-nullplatform
deleted the
feat/configurable-cloudwatch-annotations
branch
August 13, 2026 21:41
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #223.
Why
Five annotations were literals in both deployment templates:
So every pod in every account shipped to CloudWatch, under
<namespace>.<application>, kept for exactly 7 days. None of it could change without a change to this repository — and the gate in particular decided routing for workloads whose account may not even use CloudWatch.What
All five are removed.
annotations:and thek8s_modifiersmerges stay, so the same keys arrive from wherever an account's logging setup is owned.They are removed rather than made configurable here on purpose. All five are inputs to the same consumer, the logs controller, and an override already stamps annotations through
k8s_modifiers.deployment.annotations. Splitting them across two layers would mean two sources for one decision, and the two could then disagree — a gate pointing at one annotation family while the naming points at another produces logs that ship to the wrong place, or nowhere, with no error anywhere.Keeping them in one place also makes the failure direction the safe one: a pod that nothing annotates ships nowhere.
Also fixes the scheduled_task region
scheduled_task/deployment/templates/deployment.yaml.tplhardcodedregion: us-east-1instead of deriving it from the context like the k8s template did. Every scheduled task outside us-east-1 was naming the wrong CloudWatch region. Removing the block drops that too.A cluster that relied on these templates to stamp
nullplatform.logs.cloudwatch: 'true'stops shipping application logs once its scopes are redeployed, until something injects the gate. The logs controller's rules are keyed on that annotation; an unannotated pod matches nothing and its records are dropped with no error.Accounts that stamp their own gates through an override are unaffected. If a gradual path is wanted, the place for it is that override defaulting to CloudWatch when it finds no logging provider, rather than reinstating a literal here.
Verification
bats k8s/deployment/tests/*.bats scheduled_task/deployment/tests/*.bats→ 314 passing, 0 failing.Deployment/CronJobwith every container intact.annotations:renders as null, which the API server treats as absent.k8s_modifiers.deployment.annotationscarrying the gate and the naming keys: they come through unchanged, at the right indentation, in both templates.setup_logs_provider): the pod ends up with the fournullplatform.logs.cloudwatch.*values identical to what these templates used to produce, plus the gates the override resolves.