Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 19 additions & 4 deletions client/templates/requests-proxy-deployment.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -48,13 +48,28 @@ spec:
capabilities:
drop: ["ALL"]
readOnlyRootFilesystem: true
{{- /* Nil-guard the whole resources.requestsProxy chain. A
helm upgrade --reuse-values from a release predating this
field (any 1.3.x ≤ 1.3.5) won't have it in stored values,
and the direct path .Values.resources.requestsProxy... on
a nil parent dict crashes with "nil pointer evaluating
interface{}.requests". Default-through-dict idiom lets
missing values fall through to the inline literals.
The trailing dash on `{{-` strips the newline AFTER
readOnlyRootFilesystem; we deliberately do NOT use `-}}`
on the last `:=` so the newline before `resources:` is
preserved (an earlier version of this guard ate the
newline and rendered `readOnlyRootFilesystem: trueresources:`). */ -}}
{{- $rp := default dict (default dict .Values.resources).requestsProxy }}
{{- $rpReq := default dict $rp.requests }}
{{- $rpLim := default dict $rp.limits }}
resources:
requests:
cpu: 100m
memory: 256Mi
cpu: {{ $rpReq.cpu | default "100m" | quote }}
memory: {{ $rpReq.memory | default "256Mi" | quote }}
limits:
cpu: 1000m
memory: 512Mi
cpu: {{ $rpLim.cpu | default "1000m" | quote }}
memory: {{ $rpLim.memory | default "512Mi" | quote }}
env:
- name: MYSQL_HOST
value: "mysql-client"
Expand Down
10 changes: 10 additions & 0 deletions client/values.schema.json
Original file line number Diff line number Diff line change
Expand Up @@ -293,6 +293,16 @@
}
}
},
"ingestor": {
"type": "object",
"properties": {
"digest": {
"type": "string",
"pattern": "^(sha256:[a-f0-9]{64})?$",
"description": "Optional canonical ghcr.io/tracebloc/ingestor digest (sha256:<64 hex>). Surfaced into jobs-manager as the INGESTOR_IMAGE_DIGEST env so the customer-facing tracebloc/ingestor subchart can pick it up without per-install overrides. Empty disables the default and forces customers to --set image.digest on each ingestor subchart install."
}
}
},
"mysqlClient": {
"type": "object",
"properties": {
Expand Down
11 changes: 11 additions & 0 deletions client/values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -249,6 +249,17 @@ resources:
limits:
cpu: "500m"
memory: "512Mi"
# requests-proxy serves the Service Bus / backend communication path and
# is mostly idle (a few req/min). 100m/256Mi requests with headroom for
# the occasional burst; revisit if heap profiling shows growth. Schema
# at values.schema.json#resources.requestsProxy validates overrides.
requestsProxy:
requests:
cpu: "100m"
memory: "256Mi"
limits:
cpu: "1000m"
memory: "512Mi"

# -- PriorityClass for the data-plane (mysql).
# Cluster-scoped resource. Created with helm.sh/resource-policy: keep so a
Expand Down
43 changes: 36 additions & 7 deletions ingestor/templates/_helpers.tpl
Original file line number Diff line number Diff line change
Expand Up @@ -31,21 +31,50 @@ Naming:
{{- end -}}

{{- /*
Resolved idempotency key. Defaults to "<release>-<unix-epoch>" so each
install is a fresh run — including reinstalls under the same release
name, where Helm restarts revisions at 1 and a revision-derived key
would collide with the previous attempt and trip jobs-manager's
"already used with a different image_digest or table" guard. Explicit
override is honored verbatim; set it to a stable UUID only when you
want at-most-once semantics across reinstalls.
Resolved idempotency key.

Default behavior:
- First helm install: stamp a fresh "<release>-<unix-epoch>" key.
- helm upgrade of the same release: REUSE the existing key by looking
up the post-install hook ConfigMap from the previous render. This
preserves replay semantics — jobs-manager sees the same key on
upgrade and returns 200 (replay) rather than spawning a new run.
- helm install after uninstall: lookup misses (ConfigMap was deleted
on uninstall), so we fall through to a fresh now-based key. No
collision with the previous run because the epoch differs.

Earlier versions defaulted to `now | unixEpoch` on every render. That
worked for installs but accidentally created a NEW key on
`helm upgrade --reuse-values` (Helm preserves the stored value `""`,
not the previously-rendered key, so the template re-evaluates `now`).
The result: customers running `helm upgrade` thinking it was a no-op
got duplicate ingestion runs. Bugbot caught it on PR #137. See #139.

Helm template (no cluster connection) returns empty for lookup, so
local previews always re-stamp with a fresh key — matches the
in-cluster install path the first time around.

Explicit override is honored verbatim; set `idempotencyKey` to a
stable UUID when you want strict at-most-once semantics across
uninstall/reinstall cycles.
*/ -}}
{{- define "ingestor.idempotencyKey" -}}
{{- if .Values.idempotencyKey -}}
{{ .Values.idempotencyKey }}
{{- else -}}
{{- $existing := lookup "v1" "ConfigMap" .Release.Namespace (include "ingestor.configMapName" .) -}}
{{- /* The ConfigMap key is literally "body.json" (a single key with a dot in
its name, not a nested path), so use `index` rather than dot-access.
The fromJson call then parses the JSON body and we read its
idempotency_key field. Guards against missing data map (e.g. an
in-flight create) by defaulting through `dict`. */ -}}
{{- if and $existing (hasKey ($existing.data | default dict) "body.json") -}}
{{- (fromJson (index $existing.data "body.json")).idempotency_key -}}
{{- else -}}
{{ printf "%s-%s" .Release.Name (now | unixEpoch) }}
{{- end -}}
{{- end -}}
{{- end -}}

{{- define "ingestor.labels" -}}
app.kubernetes.io/name: ingestor
Expand Down
Loading