Skip to content

Commit 35f0210

Browse files
devantlerclaude
andcommitted
fix(openbao): bump audit PVC to 10Gi + document fail-closed mode
OpenBao's file audit backend does not rotate, and OpenBao fails CLOSED on audit-write errors (every API request blocks once the volume is full). The chart default of 1Gi would silently degrade to a fully sealed cluster after a few months at this cluster's request volume. Changes: - auditStorage.size: 1Gi -> 10Gi. 10Gi gives multi-year headroom for this cluster's traffic (~700 KB/day from current ESO + vault-snapshot use). Variable override matches the dataStorage idiom so fork operators can tune per-cluster. - Inline comment documents: * the failure mode (fail-closed, blocks API); * the rotation strategy until the observability stack ships the audit stream off-PVC (a manual SIGHUP rotate from the openbao pod); * the metric to monitor while we're still file-backed. This is a tactical sizing/documentation fix. Proper rotation + shipping happens in the observability rollout (per the observability-production-ready memory) -- promtail will consume audit.log and the PVC sizing becomes irrelevant. Tracked as a follow-up to this PR. Validation: $ ksail workload validate → 256 files validated $ ksail --config ksail.prod.yaml workload validate → 256 files validated Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1 parent 685bbfe commit 35f0210

1 file changed

Lines changed: 15 additions & 1 deletion

File tree

k8s/bases/infrastructure/controllers/openbao/helm-release.yaml

Lines changed: 15 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -63,7 +63,21 @@ spec:
6363
size: ${openbao_storage_size:=1Gi}
6464
auditStorage:
6565
enabled: true
66-
size: 1Gi
66+
# 10Gi (up from the chart default 1Gi). The file audit backend
67+
# does not rotate, and OpenBao FAILS CLOSED on audit-write
68+
# errors (every API request blocks once the volume is full).
69+
# 10Gi gives multi-year headroom for this cluster's request
70+
# volume (~700 KB/day from current ESO + vault-snapshot
71+
# traffic). Once the observability stack lands, promtail will
72+
# ship the stream off-PVC and the size will become irrelevant.
73+
# Until then, monitor 'kubelet_volume_stats_available_bytes'
74+
# on the openbao-audit-* PVC and rotate manually via
75+
# kubectl -n openbao exec openbao-0 -- sh -c \
76+
# 'mv /openbao/audit/audit.log /openbao/audit/audit.log.$(date -u +%Y%m%dT%H%M%SZ) && \
77+
# kill -HUP 1'
78+
# (OpenBao reopens its audit FD on SIGHUP, so the move +
79+
# signal pattern is safe.)
80+
size: ${openbao_audit_storage_size:=10Gi}
6781
standalone:
6882
enabled: true
6983
config: |

0 commit comments

Comments
 (0)