Skip to content

Distributor signing keys: empty entry in -distributor.sign-write-requests-keys silently downgrades V01 stream auth to forgeable #7579

@friedrichg

Description

@friedrichg

Describe the bug

SecretStringSliceCSV.Set() in pkg/util/flagext/secretstringslicecsv.go:23 uses strings.Split(s, ","), which preserves empty entries. A stray or trailing comma in -distributor.sign-write-requests-keys (e.g. newkey,, or a fat-fingered newkey, ,oldkey) leaves a "" in the key list.

The ingester's NewStreamSigningServerInterceptor(keys...) then accepts the stream when subtle.ConstantTimeCompare(sig, computeStreamHMAC("", orgID)) == 1. Because orgID here is the worker name (ingester-<addr>-stream-push-worker-<N> — documented in pkg/ingester/ingester.go), it isn't a secret. So an empty key collapses HMAC verification to HMAC-SHA256("", <guessable>), i.e. forgeable with no secret.

The distributor still signs with keys[0], so legitimate traffic flows and the misconfiguration is invisible. Net effect: a single dangling comma silently downgrades V01 stream authentication (PR #7475) to no-auth, re-opening the tenant-impersonation surface that V01 closed.

To Reproduce

  1. Start Cortex with -distributor.sign-write-requests=true -distributor.sign-write-requests-keys=newkey, (note the trailing comma).
  2. From a client that knows the worker-name format, sign a PushStream request with HMAC-SHA256("", "ingester-<addr>-stream-push-worker-0").
  3. The ingester accepts the forged signature.

Expected behavior

SecretStringSliceCSV.Set() should TrimSpace each entry and drop or reject empty entries. If trimming leaves zero valid keys, return an error from Set(). Equivalently, the ingester's stream interceptor should refuse to construct over an empty-string key.

Environment:

Additional Context

Metadata

Metadata

Assignees

No one assigned

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions