Repository navigation
NetBird access policy and group membership from reviewed files, with a read-only dashboard: how we did it #8622
dlf-dds
started this conversation in
Show and tell
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
goat is an open-source data fabric from the U.S. Army (USAREUR-AF) whose mesh is self-hosted NetBird: code, docs.
On our networks the NetBird dashboard is for looking. Access policies and peer group membership are declared in files in git, reviewed as pull requests, and applied through the management API by a service user. We call the tooling warrant. This post describes the pieces, because most of them are ordinary NetBird features arranged in a particular way, and two of them needed something NetBird does not offer.
Why
Every platform admin had the NetBird
adminrole and could change any policy or group from the dashboard. Over three months on one network, people made hundreds of such edits by hand, with no review. Access rules were already generated from files; group membership was not, and it was the change that escaped every check we had.Access policies: generated, and read back
ops/warrant/port-sets.yamlholds eight named sets of protocol and ports. Before this, our policy file carried 118 rules with ports and only 13 distinct combinations of protocol and ports, each repeated as literals.generate.pywrites the fully expanded policy file that the deployment applies through the API. CI fails if the committed file differs by one byte from what the generator produces, so nobody edits the output.reconcile.pyreads the live policies and sorts each into generated, orphaned (ours, but no longer declared) or undeclared (made by hand). It changes nothing. Deciding what to do with a hand-made policy is left to a person, and--adoptprints it as a rule intent for whoever made it to fill in and submit.We identify our own policies by a tag in the policy's
descriptionfield, because that is the only free-text field the API gives us. A policy made in the dashboard has an empty description.Group membership: declared, and converged
Each network has a directory of small YAML files, split by who owns them (infrastructure hosts, service hosts, people's devices, holds, one file per organization), each with its own reviewer. A file holds only exceptions to a device's default groups: grant a group, keep a group absent, set the whole list, or hold the device in quarantine. Every entry has an owner, a reason and an expiry date.
A converger runs on the management host every five minutes as a NetBird service user:
reportmode it prints the difference between the declaration and the live groups and changes nothing. A network starts here.enforcemode it makes the live groups match. A membership edit made by hand is put back at the next run.Quarantine is one entry, and release is reverting that commit.
One-time events are not in git: minting an enrollment key, removing a peer, granting a role. A pull request cannot hand a one-time key to a person. Those keep their own audited commands.
The dashboard: two things were needed
NetBird authorizes by the role of the token, so a second hostname alone changes nothing, and a role change alone leaves the few people who must be able to write doing it from the everyday address.
auditor. A separate small group maps toadmin.GET,HEADandOPTIONSunder/apito the management server. Everything else is refused at the proxy. A second dashboard, behind its own sign-in limited to the small admin group, uses the full API. Where warrant owns membership, that second dashboard's route also answers 403 toPOST,PUT,PATCHandDELETEon/api/groupsand/api/groups/*, so a group edit is refused when it is made, not accepted and reverted five minutes later.No NetBird code changed for any of this. The route is in our Traefik template (
mgmt-dynamic.yml.j2).Design:
mesh-change-control.md. Decisions: ADR 1129, ADR 1189. Operator guide:warrant-runbook.md. A runnable example with made-up groups is inops/warrant/demo/.Cost
A membership change now takes a pull request, a deploy of the file and up to five minutes. People who had
adminto do their work now haveauditor, and those holders can read every peer.Not finished
auto_groupswith group propagation on.reportmode and is switched toenforceby a reviewed change.Questions
managed_bystring set through the API and shown in the dashboard) be acceptable upstream? We use the policy description for this today, and groups have no equivalent.The tooling is Apache-2.0 and takes a
--policy-dir, so it runs against your own policy files without ours.All reactions