Skip to content

[encryption]: Add payload encryption observability metrics - #95

Merged
pseudomuto merged 1 commit into
mainfrom
encryption-metrics
Jul 27, 2026
Merged

[encryption]: Add payload encryption observability metrics#95
pseudomuto merged 1 commit into
mainfrom
encryption-metrics

Conversation

@pseudomuto

@pseudomuto pseudomuto commented Jul 26, 2026

Copy link
Copy Markdown
Collaborator

The payload encryption subsystem emitted no telemetry, leaving operators a bit blind.

This adds some Prometheus metrics under the "encryption" subsystem:

  • encryption_dek_ops_total (operation, result, namespace) and
    encryption_dek_ops_duration_secs (operation, namespace) for payload seal/open
  • encryption_kek_ops_total (provider, operation, result) and encryption_kek_ops_duration_secs (provider, operation) for KEK wrap/unwrap
  • encryption_dek_cache_hits_total / encryption_dek_cache_misses_total /
    encryption_dek_cache_size

To keep pkg/crypto free of both Prometheus and internal packages, the Vault emits cache events via a small Observer interface whose methods take a struct, allowing fields to be added later without breaking implementers.

internal/kms implements it with a Prometheus-backed Reporter and meters KEK wrap/unwrap through a crypto.KEK decorator that resolves the provider label from the URI scheme. DEK payload operations are recorded by the proxy encryption interceptor, the one place the namespace is known.

The namespace label on dek_ops is a deliberate choice to allow per-namespace attribution of encryption work; the remaining metrics avoid high-cardinality labels to keep series counts bounded.

@pseudomuto
pseudomuto requested a review from Copilot July 26, 2026 17:37
@pseudomuto
pseudomuto requested a review from a team as a code owner July 26, 2026 17:37
@codecov-commenter

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 93.61702% with 9 lines in your changes missing coverage. Please review.

📢 Thoughts on this report? Let us know!

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds Prometheus observability for the payload encryption subsystem (DEK encrypt/decrypt, KEK wrap/unwrap, and DEK cache behavior) while keeping pkg/crypto decoupled from Prometheus by introducing a small observer interface.

Changes:

  • Introduce crypto.Observer + WithObserver and emit DEK cache hit/miss events from Vault.Open.
  • Add Prometheus reporters for proxy-side DEK payload ops and KMS-side KEK ops + cache metrics; wire them via fx.
  • Extend internal/metrics.Factory with NewGauge and add unit tests for the new metrics behavior.

Reviewed changes

Copilot reviewed 17 out of 17 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
pkg/crypto/vault.go Adds observer support and emits cache hit/miss events from Open.
pkg/crypto/observer.go Defines Observer + CacheEvent and a default no-op implementation.
pkg/crypto/observer_test.go Tests observer behavior for hit/miss, disabled cache, and default no-op.
internal/proxy/reporter.go Adds Prometheus reporter for DEK payload operation counters/histograms.
internal/proxy/fx.go Wires encryption reporter into the proxy encryption interceptor via fx.
internal/proxy/fx_test.go Provides a metrics factory in fx tests to satisfy new dependency.
internal/proxy/encryption.go Records DEK encrypt/decrypt timing/results in the interceptor.
internal/proxy/encryption_test.go Adds tests validating DEK op metrics and pass-through behavior.
internal/metrics/factory.go Adds NewGauge helper for namespaced/subsystemed gauges.
internal/metrics/factory_test.go Tests gauge naming/registration via the metrics factory.
internal/kms/reporter.go Adds Prometheus reporter for KEK ops + DEK cache hit/miss/size metrics.
internal/kms/reporter_test.go Tests KEK op and cache event metric emission.
internal/kms/metered.go Adds metered KEK decorator and provider label mapping by URI scheme.
internal/kms/metered_test.go Tests metered KEK wrap/unwrap recording and scheme mapping.
internal/kms/fx.go Wires KMS reporter and wraps KEKs for KEK telemetry; attaches observer to vault.
internal/kms/fx_test.go Updates fx tests for new reporter/factory dependencies.
e2e/harness_test.go Provides a metrics factory in the e2e harness wiring.
Comments suppressed due to low confidence (1)

internal/proxy/encryption.go:132

  • decryptPayloads calls r.DEKOp unconditionally; passing a nil Reporter will panic once an encrypted payload is encountered. Guard the metric emit so nil means "no metrics" instead of a runtime panic.
			start := time.Now()
			pt, err := v.Open(ctx, &crypto.Message{
				Ciphertext: p.Data,
				KeyMaterial: &crypto.DEKMaterial{
					KEKID:        string(p.Metadata[metadataEncryptionKeyID]),
					EncryptedDEK: string(p.Metadata[metadataEncryptionDEK]),
				},
			})
			r.DEKOp("decrypt", resultLabel(err), ns, time.Since(start).Seconds())
			if err != nil {

Comment thread pkg/crypto/vault.go
Comment thread pkg/crypto/observer_test.go
Comment thread internal/proxy/encryption.go
Comment thread internal/proxy/reporter.go
@pseudomuto
pseudomuto force-pushed the encryption-metrics branch from 10c17a3 to d25e247 Compare July 27, 2026 11:08
The payload encryption subsystem emitted no telemetry, leaving operators
blind to KMS pressure, DEK cache effectiveness, and encryption error rates.

This adds seven Prometheus metrics under the "encryption" subsystem:

  - encryption_dek_ops_total (operation, result, namespace) and
    encryption_dek_ops_duration_secs (operation, namespace) for payload
    seal/open
  - encryption_kek_ops_total (provider, operation, result) and
    encryption_kek_ops_duration_secs (provider, operation) for KEK
    wrap/unwrap
  - encryption_dek_cache_hits_total / encryption_dek_cache_misses_total /
    encryption_dek_cache_size

To keep pkg/crypto free of both Prometheus and internal packages, the Vault
emits cache events through a small Observer interface, whose methods take a
struct so fields can be added later without breaking implementers.
internal/kms implements it with a Prometheus-backed Reporter and meters KEK
wrap/unwrap through a crypto.KEK decorator that resolves the provider label
from the URI scheme. DEK payload operations are recorded by the proxy
encryption interceptor, the one place the namespace is known. Both reporters
are built once and share the "encryption" subsystem with disjoint metric
names, so there is no duplicate-registration risk.

Metric collection is behavior-preserving: the KEK decorator returns the
wrapped call's result and error verbatim, and the interceptor's timing is
purely additive to the existing seal/open path.

The namespace label on dek_ops is a deliberate choice to allow per-namespace
attribution of encryption work; the remaining metrics avoid high-cardinality
labels to keep series counts bounded.
@pseudomuto
pseudomuto force-pushed the encryption-metrics branch from d25e247 to 9319f0a Compare July 27, 2026 11:10
@pseudomuto
pseudomuto merged commit 2167828 into main Jul 27, 2026
6 checks passed
@pseudomuto
pseudomuto deleted the encryption-metrics branch July 27, 2026 11:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants