Skip to content

[bug] Collector mode ignores clustering.tls: leader→member API dispatch cannot use TLS client certs (mTLS impossible, private-CA server certs fail) #891

Description

@vuthinh0301

Summary
In collector mode (gnmic collect), the cluster leader's HTTP clients that call other members' REST APIs (target assignment, unassignment, target deletion) are plain &http.Client{} instances that never load the clustering.tls configuration. As a result:

A private-CA server certificate on api-server.tls breaks target dispatch with x509: certificate signed by unknown authority.
mTLS (api-server.tls.client-auth: require-verify) is impossible: the dispatch client presents no client certificate, so the leader gets remote error: tls: bad certificate calling members — including itself.
This works correctly in subscribe mode, where clustering.tls was wired up by #544 (v0.39.0, for #529). Collector mode (added in v0.43.0) has a separate cluster implementation under pkg/collector/managers/cluster/ that never got the same treatment — the code still carries // TODO: markers at the relevant client.

Version
main @ 4c6c11a (2026-07-09), also reproduced on v0.46.0.
Reproduce
3-instance gnmic collect cluster + Consul locker. Enable TLS on the API with a private CA:

api-server:
tls:
ca-file: /certs/ca.pem # implies client-auth: require-verify (mTLS)
cert-file: /certs/server.pem
key-file: /certs/server-key.pem
clustering:
cluster-name: c1
instance-name: ${INSTANCE}
service-address: ${ADDR}
tls: # documented on the HA page, but ignored here
ca-file: /certs/ca.pem
cert-file: /certs/client.pem
key-file: /certs/client-key.pem
locker: { type: consul, address: consul:8500 }
Add a target via the API. The leader elects fine, then fails to dispatch:

level=INFO msg="assigning target" target=X assignee=gnmic-0
http: TLS handshake error from : remote error: tls: bad certificate
level=ERROR msg="failed to push assignments" error="Post "https://gnmic-0...:7890/api/v1/assignments\": tls: failed to verify certificate: x509: certificate signed by unknown authority"
The target is never assigned; no telemetry. (Server-only TLS with a publicly trusted cert and no client-auth would connect, but private-CA and mTLS — the normal on-prem setup — cannot.)

Root cause (main @ 4c6c11a)
The clustering TLS config is fully parsed and even env-expanded, but never used to build the dispatch clients:

Parsed: pkg/config/clustering.go:37 — TLS *types.TLSConfig, populated at pkg/config/clustering.go:57-61.
Env-expanded (proof it's processed in collector mode): pkg/collector/env/env.go:18-21 expands clusteringConfig.TLS.{CaFile,CertFile,KeyFile}.
Never consumed — the two outbound inter-member clients are hardcoded plain clients:
pkg/collector/managers/cluster/cluster_manager.go:69 — apiClient: &http.Client{Timeout: 10 * time.Second}, // TODO:
pkg/collector/managers/cluster/assigner.go:48 — client: &http.Client{Timeout: 10 * time.Second}
grep -rn 'Clustering.TLS' pkg/collector/ returns nothing — the collector package never reads it.

Meanwhile members register __protocol=https whenever api-server.tls != nil (cluster_manager.go:240-244), so the assigner correctly builds https:// URLs (assigner.go:61,105) — it just dials them with a TLS-unaware client.

Expected behavior
Collector-mode dispatch clients should honor clustering.tls exactly as subscribe mode does, so private-CA server certs verify and client certs are presented for mTLS.

Suggested fix
Mirror the reference implementation App.createAPIClient() at pkg/app/clustering.go:704-737 (which builds the client via utils.NewTLSConfig(CaFile, CertFile, KeyFile, "", SkipVerify, false)), and inject the resulting *http.Client into both NewClusterManager (replacing the // TODO: client at cluster_manager.go:69) and NewAssigner (assigner.go:44-52). Both already have access to the clustering config (ClusterManager.clusteringConfig), so the client can be constructed once from clusteringConfig.TLS and shared.

Impact / workaround
On-prem clusters (private CA, mTLS policy) cannot secure the collector API end-to-end today. Current workarounds: (a) api-server.tls with cert/key only — no ca-file, no client-auth (encryption + server-auth only; caution: setting ca-file silently defaults client-auth to require-verify and breaks the cluster), plus SSL_CERT_FILE=<ca.pem> in the process environment so the dispatch client trusts the internal CA, and network-layer authn (firewall / k8s NetworkPolicy / service-mesh mTLS); or (b) use subscribe mode (where clustering.tls works) with a target loader instead of collector mode.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions