Repository navigation
Client certificates for the Management and Signal connections, with PKCS#11 support #7356
csiqueirasilva
started this conversation in
Ideas & Feature Requests
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Before posting
Product area
Client / Agent, CLI, Mobile app, Identity provider / SSO, Self-hosting / Deployment
Problem or use case
As the operator of a self-hosted deployment, I need a physical cryptographic
device to be present on a machine at the moment it connects to our private
WireGuard network, not at enrolment only but on every connection. This comes
from my employer's security requirements: network access must be bound to
something that cannot be copied, so that losing access is a physical event (the
device is handed back, or reported lost) rather than a forensic one. The device
is a smartcard-class token whose private key is marked non-extractable, which is
the entire reason it was procured.
NetBird cannot express that today.
The peer's WireGuard private key is stored in plaintext in the profile file
(
PrivateKey string, writtenout as JSON), and once a peer is registered, Management asks for nothing else on
reconnect. Copying that file to another machine yields a working peer: same key,
same identity, same access to whatever the peer's groups allow. The copy is not
visible as an anomaly, and revoking it requires noticing it first. So the token
stops mattering as soon as enrolment finishes: whatever was required to register
the peer is not required again on any reconnect, and, for a peer registered with
a setup key, not required again at all.
The threat model is narrow and ordinary: anyone who can read files as the user, as
root, from a backup, or from a disk image obtains a durable network identity
rather than a session.
Why the existing mTLS support does not cover it. PR #2188 ("[client] Mtls
support", merged 2024-08-13) added a client certificate to the PKCE token exchange
against the IdP. Its six changed files are
client/internal/auth/pkce_flow.go,client/internal/auth/oauth.go,client/internal/pkce_auth.go,client/internal/config.goand the two mobile login shims: the certificatereaches the IdP's token endpoint and nowhere else.
That secures enrolment, and after it the peer reconnects on its WireGuard key
alone. A peer registered with a setup key never repeats the SSO exchange, since
login expiration is only switched on for peers
added by a user; a peer registered through
SSO repeats it when its login expires, which by default is
once a day. So a copied profile
connects, and at best the certificate is asked for again once a day rather than
on every connection. The key there must also come from
tls.LoadX509KeyPair,i.e. from a file, which for this requirement is a second blocker, since a
copyable key file next to the WireGuard key does not amount to "a device must be
present".
Why #5364 is not this. Issue #5364 ("mTLS Auth for Proxy Services", open) is
adjacent but different: it proposes mTLS on the reverse-proxy / Agent Network
edge, with a managed internal CA, per-user issuance and revocation, and identity
header injection for applications that cannot do OIDC. It is about authenticating
clients to published services, not about the peer's own control-plane connections.
One of its comments asks for the proxy to read identity off the peer certificate,
which is the closest anyone gets there. If an existing ticket does cover the
control plane and I missed it, point me at it and I will move this there.
Issue #4278 ("mTLS certificates broken", closed) is relevant only as evidence that
the current mTLS path breaks quietly: the multi-profile refactor stopped sending
the certificate to the IdP, and the original contributor of the feature is the one
who noticed.
Proposed solution
Three changes in this repository, plus one unrelated build fix in the mobile
repository. Part 1 stands alone. Parts 2 and 3 each build on it and are unrelated
to each other, so any of the three can be accepted, rejected, or reshaped without
deciding the rest.
1. Present client certificates on the Management and Signal connections. This
is what makes the requirement expressible at all: if the control plane demands a
certificate on every connection, a copied profile is inert.
CreateConnectiontakes the certificatesexplicitly instead of reading them from anywhere global; both gRPC clients gain
WithClientCertificates([]tls.Certificate)in the option pattern they alreadyuse (management,
signal);
every call site passes the profile's certificates, which means the engine's Management and Signal connections, both clients in
client/internal/auth, the daemon's Management client, and theURL prober. All
candidates are offered and the server's
CertificateRequestdecides which one ispresented, the same selection a browser performs. Empty by default, in which case
the TLS config is what it is today. No proto change and no server-side change:
gating is done by whatever terminates TLS in front of Management and Signal. This
part works with ordinary file-based certificates and is useful to deployments with
no hardware requirement at all.
2. PKCS#11 in the client, so the key can stay in the device. The private key is
never read:
tls.Certificate.PrivateKeyis acrypto.Signerthat delegates to thedevice, so only the signature crosses the PKCS#11 boundary. Nothing that can go
stale is written into the profile. The driver is discovered by probing the usual
install locations for the common vendor modules, the token is enumerated at login,
the certificate is discovered on the token, and the chain is built by walking up
through the CA certificates found there. Only the driver path that worked is
persisted, and if it later disappears from disk, discovery runs again rather than
failing. Two flags:
netbird login --pkcs11to discover,--pkcs11-module <path>for an unusual install.
3. The same certificate on Android, from a file or from hardware. On mobile
there is no command line and no module to enumerate, so the certificate is
configured explicitly through
Preferences.SetClientCertificate(certPath, keyPath)and
ClearClientCertificate(), plus aClientCertificateSignerinterface the appimplements. That interface is the shape this package already uses for
TunAdapterandConnectionListener, and itexists so that a key held by the platform keystore or a security key never has to
become a file.
Plus, unrelated: fix
-androidapiinbuild-android-lib.sh(netbirdio/android-client). Nothing to do with mTLS and the smallest item of the
four.
gomobile binddefaults to Android API 16 and NDK 27 accepts 21..35, so thebind fails outright with "unsupported API version 16". The fix is one line on
the
gomobile bindinvocation,-androidapi "${NB_ANDROID_API:-24}", default 24 and overridable; build with 26 tomatch
minSdkVersion.The linked branch carries more than that line: it also has the
onStateChangedoverrides the app needs to build against a newer netbird, andthis fork's own certificate wiring. Only the
-androidapiline is proposed here.Alternatives or workarounds considered
as described above. This is the workaround currently in use, and it is what
prompted the request.
dependency, and it is genuinely the right answer when the identity belongs to
the machine, as WireGuard already does with its own key. It does not
meet a requirement phrased as "the person, with the device in hand". Part 1
above delivers exactly this and is proposed separately for that reason.
client change, but the endpoints the client dials come from Management and are
global, so pointing them at localhost affects every client.
NB_PKCS11_PINin the daemon environment only. Means writing the PIN wherethe service manager can read it, which removes the second factor the device
exists to provide.
UnlockTokenRPC instead of a field onLoginRequest. Cleanerin shape, but the PIN still crosses the same socket, so it did not seem to buy
anything. Happy to be told otherwise.
Community impact and priority
operations team. I cannot speak for anyone else's scale.
control plane ever offered client-certificate gating; nothing here assumes it.
running a patched client, which is not a position to stay in.
path regressed unnoticed), PR [client] Mtls support #2188 (the IdP-only implementation).
Examples from other tools or products
OpenSSH takes this approach with
PKCS11Provider: the key stays on the device and the client asks it to sign, with no file anywhere. Browsers do the same selection this proposal relies on: the server names the CAs it accepts in itsCertificateRequestand the client offers what matches, which is why the design here offers all candidates rather than deciding in advance.Security, privacy, and compatibility considerations
Nothing changes for anyone who does not configure a certificate:
CreateConnectionpasses the slice straight intotls.Config.Certificates;with none configured it is empty, and the
tls.Configis byte for byte what itis today.
and config reads no longer touch the token when no PIN is available. Before
that guard, every daemon startup, status query and profile listing spent a token
round trip and logged a failure. The certificate is loaded on the login path
only.
unlocked certificate touches only a profile that takes its certificate from a
token, and never overwrites one already loaded. Each of these has a test
asserting exactly that.
LoginRequestand onestring on
GetConfigResponse. Older peers ignore them.ClientCertPath/ClientCertKeyPathpath is untouchedand remains the default; PKCS#11 is mutually exclusive with it.
CGO_ENABLED=0 GOOS=android GOARCH=arm64 go build ./client/android/succeeds:the PKCS#11 implementation sits behind a cgo build tag with a stub that returns
a reason, because the Android library is built with
CGO_ENABLED=0 gomobile bindand the vendormodule is a C library loaded at runtime.
Two security points worth arguing about rather than assuming:
LoginRequest, because the daemon is a system service with no terminal, and onWindows runs in a session that cannot prompt at all.
GetConfigResponsereportswhich PKCS#11 module the active profile uses so the CLI knows to ask before
logging in rather than after a failure. The PIN is held in memory for that login
and never serialized: writing it into the profile beside the WireGuard key would
collapse the device's two factors into one.
C_Login, and the token open is retried only on the two failures that precedelogin, never past it, because these devices lock themselves after a handful of
wrong attempts.
Implementation ideas
There is a working implementation in a public fork. It is not offered as
merge-ready and nobody is being asked to review it as it stands. It exists so the
questions below are concrete rather than hypothetical.
Each of the three parts above is a separate branch, sized to be reviewable on its
own. Parts 2 and 3 are stacked on part 1, so their links compare against it rather
than against
main.client-cert-management-signal(compare against upstream main).
10 files, +236/-28, of which 174 lines are a test that runs a real mutually
authenticated handshake.
client-pkcs11-client-certificate(compare against part 1).
19 files, +1458/-17, of which 407 lines are tests and 72 changed lines
(+63/-9) are regenerated proto output. This one is past the
size CONTRIBUTING asks for
and I could not find an honest way to halve it: driver discovery, the PIN prompt and the token loading
are one feature, and a branch missing any of them is a config field that nothing
sets. If it should be split anyway, say where and I will cut it there.
android-client-certificate(compare against part 1).
7 files, +499/-2, of which 236 lines are tests.
csiqueirasilva/android-client, branch
feat/pkcs11-client-cert(compare against upstream main).
The single branch these were cut from is still there as
feat/pkcs11-client-certif the development order is of any use. It is not proposed for review.
On each of the three branches,
go build ./client/...,go build ./shared/...,CGO_ENABLED=0 GOOS=android GOARCH=arm64 go build ./client/android/,gofmt -lonthe touched files,
golangci-lint run --new-from-rev=origin/main, andgo test -tags devcerton the touched packages are all clean.Where this has actually been run
All of it against the same physical device, a SafeNet eToken 5110 SC.
Linux, ordinary machine (Ubuntu host). The PKCE code exchange completes
against an IdP token endpoint that requires a client certificate, with the peer
registering bound to the user, where an unpatched client is refused. Starting from
a profile with no PKCS#11 block at all, the driver is found and named, the token is
found, the client certificate is discovered alongside the intermediate CA sharing
the token, the chain comes out at two, and the mutually authenticated request
passes the gate. On a machine with both SafeNet and OpenSC installed, OpenSC was
correctly passed over for seeing no token; a shorter probe retry took discovery
from 6.7s to 2.4s. The CLI prompt path was exercised with the daemon started with
no PIN in its environment.
Linux, inside an unprivileged Docker container. The host is Ubuntu and the
containerised client runs on Debian 12, with the vendor PKCS#11 library and the
pcscdsocket bind-mounted in from the host and no added privileges. One detailworth passing on to anyone repeating it: the image had to be rebuilt on a glibc
base, because the official client image is
Alpine/musl and the vendor module is glibc-only.
Windows 11, on a laptop, with the token plugged into that machine. The browser
presented the token's certificate at
/oauth2/authand the daemon presented it at/oauth2/token, both confirmed in the server log. An earlier build then failedimmediately afterwards:
403onmanagement.ManagementService/GetServerKey, withthe server recording that connection as carrying no certificate. That was the bug
described above: the certificate was being treated as part of logging in rather
than as part of the transport, so it was lost on the first config reload after
login. With that fixed, the peer enrolled by SSO and stayed connected.
Wine was used earlier, before Windows hardware was available, only to check that
the vendor DLL loaded and that the PIN handling behaved; it is not the basis for
the Windows result above.
In production, on one small self-hosted installation, where the TLS terminator
in front of the control plane requires a client certificate on
/management.ManagementService/*and
/signalexchange.SignalExchange/*. A peerwhose device is not plugged in does not connect, which is the behaviour the
requirement asks for.
What has not been proven
I would rather say this than imply otherwise:
it has never been run. On Android,
ClientCertificateSignerhas no Javaimplementation at all, only Go-side tests where a fake signer holding a real EC
key completes an actual TLS handshake, and a refusing signer fails the handshake
rather than proceeding without a certificate.
certificates themselves; that is left to the TLS terminator, which may or may not
be the design you would want.
Are you willing to help?
Yes, I can submit a PR if the approach is accepted.
Additional context
The open questions, which are the reason for posting rather than opening a PR:
Does the split into three make sense? Specifically, is part 1, client
certificates on Management and Signal with file-based keys and no new
dependency, worth considering on its own, independently of whether PKCS#11 ever
lands in the client? The work as a whole is well past the
PR size CONTRIBUTING asks for, so some split is
needed regardless; the question is whether this is the right one, and whether
part 2 needs cutting further.
How should the PIN be collected? The current shape puts a secret on the
daemon IPC, which is not a comfortable place for one, and the alternatives I
found are listed above. If there is a preferred pattern for daemon-side secret
prompting, I will build to it.
Would a PKCS#11 dependency behind a cgo build tag be acceptable?
crypto11/miekg/pkcs11binds the native module and needs cgo, and thenon-cgo build gets a stub that returns a reason. That matters more than I
first thought.
.goreleaser.yamlbuilds thenetbirdandnetbird-staticclient binaries withCGO_ENABLED=0; onlynetbird-mgmt,netbird-serverandnetbird-idp-migrateuse cgo. So as itstands the released client would ship the stub, and PKCS#11 would exist only
for people building from source. The
CGO_ENABLED=1ingolang-test-linux.ymlis a build check, not the release path.
That makes this your call rather than mine: would you consider cgo for the
client build, is a
purego-style dynamic load without cgo preferable, orshould hardware-key support stay out of the official binaries altogether? It
is also two new modules in
go.mod. I would rather build to the shape youwant than guess at it.
Is
Preferences.SetClientCertificatethe API you want on Android? Itfollows the existing preferences shape, but it is a public SDK surface and an
app would need a screen for it. If the certificate should come only from the
platform keystore via
ClientCertificateSigner, and never from file paths,that is a smaller surface and the path-based half can be dropped.
Does the discovery behaviour fit how the client should behave generally?
Enumerating drivers and tokens at login, prompting when there is more than one,
persisting only what worked, is a deliberate departure from writing a path
into a config file, and it touches CLI behaviour, so it seemed like something to
agree on before writing more of it.
The
-androidapibuild fix is independent of all of this and can go tonetbirdio/android-clienton its own whenever that is useful.All reactions