Releases: theta42/theta-directory
Releases · theta42/theta-directory
Release list
v2.16.0
- feat: model events are standardized, so storage backend no longer decides what can be live. The ORM announced changes for models it managed; everything else was silent, which meant LDAP groups and users, Redis-backed notifications and PATs could not update a page no matter what the view did.
utils/model_eventsgives them the identical contract —model:<Name>:<action>with{model, action, pk, data}— so a subscriber cannot tell which backend a model uses.datagoes throughtoJSON()(strippingisPrivatefields), and a delete never carries a body, enforced in the emitter rather than trusted to each call site. ORM and bespoke models share one filtered bus, so "does this model have a read gate?" is answered in exactly one place. - feat: LDAP groups and users announce their changes.
Groupcovers add, remove and member/owner changes;Usercovers add, update and remove. The users and profile views consume them, so a user added or a group membership changed by another admin now shows up without a refresh — including on someone's own profile, where a stale page is most misleading since it is showing them their own access. - security: read gates for every model whose data the UI renders —
ResourceEdge,Group,User,Notification,ApiToken,MeshClient,MeshSite,MeshExitGrant,AccessRequest,Agent. Several are row-level, not merely model-level: a user receives their ownUserrecord, notifications, PATs and mesh clients and nobody else's, while a directory admin receives all of them except PATs — which have no admin path because the REST route has none either. - security:
Userpayloads stripuserPasswordexplicitly. It is present on a record read withattributes: ['*','+']as the admin bind, and stayed off the wire only becauseuser_parse()sets it toundefinedinside anifbranch — an incidental protection in an unrelated function, not somewhere to hang a credential. - note: the
Groupgate mirrors its currently ungated REST route (routes/group.js:8) rather than being tightened past it. A socket stricter than the endpoint feeding the same page shows a list that silently stops updating; tighten the route and the gate follows automatically.
v2.15.0
- feat: the Directory and Discovery Plugins views update themselves. A resource added, renamed or removed by another admin (or by a discovery plugin run) now appears without a refresh. The Directory table is derived — hierarchy from the edge list, hostName from the parent, agent status from another service, indentation and carets recomputed by
renderTable()— so a change re-derives the view rather than patching one row, which would leave a new child at the wrong depth with no caret on its parent.loadResources()re-reads the search, sort and secrets-only controls, so the operator's filter survives. Debounced, since one edit emits several events and an import emits a burst. - fix: the socket never connected at all.
authIOcalledAuth.checkToken(tok)with a bare string where{token}is expected, then calledtoken.getUser()— butcheckTokenreturns the User itself and has no such method. Every handshake failed withtoken.getUser is not a function, so no client in this app has ever had a working socket. - fix: nothing published.
@simpleworkjs/ormhas apubsubhook that emitsmodel:<Name>:<action>on save/delete, and it was never wired. It is now, throughutils/socket_pubsub.ormBus— a filter that forwards only models with a socket read gate. The ORM publishes for everything it loads, includingAuthToken,OtpTokenandPasswordResetToken, written on every login and password reset; those must never reach a browser and would be constant bus churn besides. - security:
READERSis the single source of truth for what goes live. A model listed there both publishes (LIVE_MODELSis derived from it) and is authorized there, so the two cannot drift — publishing something ungated would leak, gating something that never publishes would be dead code.Resource,ResourceGroupandPluginInstanceare gated to the same admin groups that guard their REST routes, resolved transitively from LDAP asutils/permission.byGroupdoes and cached briefly per socket. - note:
views/plugins.ejsis dead code —/pluginsredirects to/directoryand that UI was ported intodirectory.ejs. The Discovery Plugins subscription lives with the code that actually runs.
v2.14.0
- security: the socket bridge rebroadcast whatever a client published.
socket.on('P2PSub')took any topic and payload an authenticated client emitted and fanned it out to every other connected client. No app code has ever calledapp.publish(), so nothing legitimate used it. Events now flow server → client only. - security: the outbound side was an unconditional
app.io.emitof every event on the bus, with its full record, to every authenticated socket. Nothing here publishes model events yet, so it carried no traffic — but it would have started leaking the moment anything did. It is replaced byutils/socket_pubsub.js, a per-socket read gate whoseREADERStable is empty by design: nothing is broadcast until a model opts in together with the check that decides who may see its events. - fix: a delete event carries a
nullbody, and the client tagged it unconditionally — throwing and killing the socket handler. - feat: the shared UI shell loads
app.sync.jsandapp.filter.jsfrom@simpleworkjs/frontend, so views here can adopt live updates and filtering. - chore:
authIOrecords the session's groups on the socket, which is what a read gate resolves rights from.
v2.13.0
v2.13.0
Security
PUT /api/mesh/selfis admin-gated. It writes this site's WireGuard identity, exit posture and LAN/DNS mapping — previously any authenticated user could clobber the gateway public key (breaking every tunnel to this site cluster-wide), flipexitOpen, or remap DNS/LAN. The legitimate gateway authenticates with a PAT minted as the directory admin, sorequireAdminlets it through while keeping everyone else out.GET /api/mesh/peersand/api/mesh/site-clientsare admin-gated. Both expose the full network map and device registry, and exist for the gateway to consume.GET /api/mesh/rosterscrubs WireGuard keys/endpoints for non-admins. The mesh page still shows site metadata, but no longer hands a regular user full network enumeration (gatewayPublishedboolean instead of the key).
Fixed
- The Multi-Site modal's gateway-mesh count no longer 404s. It called
GET /api/mesh/gateways, an endpoint that died with the mesh-v2 rewrite. The count is now computed locally from theMeshSiteroster (the directory IS the registry in v2). - The LDIF import confirm is no longer a native
confirm()— it uses the app's promise-basedapp.messages.confirm(...)modal.
Other
- Synced the served
theta-agent/install.shwith theta-agent v2.5.0 (join-key merge + Linux tray arch).
v2.12.0 — the site network
v2.12.0
The directory is now where the WireGuard cluster is configured. Sites, devices,
LAN mapping and internet exits all live here; each site's gateway reads the
roster and configures itself. Requires theta-gateway v3.0.0.
- feat: the site network. A site that joins the directory is on the network
— no separate mesh to set up.siteIdIS the site'sldapServerId, already
allocated once cluster-wide by the master at join time (under a lock, after
two spokes were once handed the same one), so the mesh needs no allocator of
its own. That one number drives LDAP replication, the gateway's
172.24.0.<id>address, and the site's10.<id>.0.0/16. - feat:
/networkpage, visible to any signed-in user because enrolling
your own devices is not an admin task. Sites tab shows each site's addresses
and whether its gateway has actually published a key — joined-but-not-started
is a real state that used to look identical to healthy. Devices tab enrols
hardware and picks an exit. Exit Access tab is admin-only. - feat: devices with optional key custody. Supply a public key and the
private half never reaches the server; omit it and one is generated, rendered
into a config once, and forgotten — never stored, never logged, not
recoverable. Addresses come from the site's own pool, lowest-free so a
removed device returns its address rather than a counter marching upward. - feat: per-device internet exits. Two independent things: a site marked
exitOpenis saying it is willing to carry traffic; a grant says a user may
use it. Revoking a grant immediately drops affected devices back to local
breakout rather than leaving them routed somewhere they may no longer go.
Exit choice is a routing rule on the gateway, so switching exits produces a
byte-identical device config and needs no reconnect. - feat: DNS is pushed as the mapped address, not the physical one. A device
handed192.168.1.1only resolves while sitting on that LAN; over the tunnel
what is routed is the shadow range. The site form shows the translation as
you type and warns when an address is outside both mapped LANs, where it
cannot be mapped and devices would get no resolver at all. - feat: MTU clamped to 1380 in device configs. Mesh-then-exit is WireGuard
inside WireGuard, and a device sized for one hop blackholes large packets on
the second — the classic "SSH works, HTTPS hangs". - feat: devices running theta-agent are configured over the agent's existing
websocket instead of a human copying a file. The push deliberately carries no
private key: the agent holds its own. - feat: the hub — the site carrying
10.0.0.0/8as a catch-all — is chosen in
the UI rather than implied by which site holds the master directory, since
the natural hub is a cheap always-up VPS. - change!:
utils/mesh_route.jsstops being a workaround. It existed because
WireGuard lived inside the gateway container's namespace, so the only path to
a peer was a userspace relay on a port derived from the site index. A peer
site's directory is now simply10.<siteId>.0.2:3001, and the derived-port
contract the two repos had to keep in sync is gone. An address in the retired
scheme deliberately resolves to nothing rather than silently naming a
different site. The public-endpoint fallback stays, so a deployment whose
containers have no route into the mesh still replicates over the internet. - fix: the roster now reaches every site. It is written at each site (a
gateway publishes to its own directory) but replication only flows
master -> spoke, so two halves were missing: a spoke's public key never left
the spoke, and a spoke never learned any other site existed. Either alone
means the mesh works only at whichever site happens to be the master. Spokes
now forward their gateway details overPOST /api/site/spokes— the channel
they already have, with the credential they already hold — and the master
carries the whole roster in its export. Roster edits push a resync
immediately instead of waiting for an unrelated catalog change. A site's own
row is never overwritten by an incoming export, since the local copy is
always at least as fresh. - fix: exit interfaces need their own key. A gateway's exit interface and
its mesh interface presented the same public key to the same remote, which
keeps one endpoint and one session per peer key — so the remote's endpoint
flapped between the two and they invalidated each other's session. Verified
against wireguard-go: with one key on two interfaces the remote settled on
whichever handshook last while both kept re-handshaking; with separate keys
it holds two stable peers. Gateways now publish a secondgatewayExitPublicKey
andGET /api/mesh/peerstells an exit site which gateways to accept under
it, allowed only the device addresses actually using that exit. - docs: The Site Network.
v2.11.0 — import an existing LDAP directory
v2.11.0 - 2026-08-11
Added
- Import users and groups from an existing LDAP directory. New wizard at Users → Import LDIF (
/users/import, admin-only) takes aslapcat/ldapsearchexport and migrates accounts through the app's own model layer -- so every imported account arrives with aUserVerificationrow, a personal group, cache invalidation and service-account membership, exactly as if it had been created here. Two guarantees drive the design:uidNumber/gidNumberare preserved verbatim (they are what every file on every host is owned by -- reallocating them turns a migration into a filesystem-wide chown), anduserPasswordis carried across as the stored hash, never re-hashed, so people keep the password they already have. See docs/ldif-import.md.utils/ldif.js-- a standalone RFC 2849 parser: line folding, base64, attribute options, CRLF. It refusesattr:< urlvalues (a file-disclosure primitive handed to whoever uploads the dump) and rejects change records rather than misreading them as content.utils/ldif_import.js-- schema-agnostic profiling, planning and applying. The layout of the source directory is detected and then editable, so a FreeIPA or AD export is a mapping change rather than a code change. Membership is resolved per entry, because real directories mixgroupOfNames/memberwithposixGroup/memberUidand a single file-wide choice silently drops whichever kind loses.- Review happens before anything is written: every account shows what it will bring across, what it will not, and why it cannot be imported. Blocked rows (no
uidNumber, a duplicate inside the file, a collision with an account already here) are skipped no matter what the client sends. - Groups are never created by an import. A group name from another directory carries no meaning in this one, where access is a projection of the resource graph (docs/groups.md). Each source group either has its members merged into a group that already exists, or is dropped; many-to-one is supported for collapsing a sprawling old directory.
- Carried across where present: password hash, uid/gid numbers, name, email, phone, shell, home directory, description, location, date of birth, every SSH key, sudo rules, and the account's disabled state. Anything else is listed on the row as not migrated rather than dropped quietly.
- Onboarding is a per-run choice (treat ToS as accepted / email as verified) so a cutover need not re-ask 30 people. Legacy MD5 passwords still always force a change at first login, and no welcome email is sent to anyone.
- Staging lives in Redis under a one-hour expiry and is destroyed on apply or abandon; the parsed dump is never written to disk and password hashes are never included in an API response.
Changed
User.add(data, options)takes an options argument forpreserveIds,preserveHashandsuppressWelcome. These are a second argument rather than fields ondatadeliberately: every route callsUser.add(req.body), so auidNumberhonoured whenever present would let anyone who can create a user claim uid 0, and auserPasswordthat skipped hashing whenever it looked hashed would let them plant a known hash. Normal user creation is unchanged.- An account may share its primary group with another account. Each account normally gets a personal group of its own, but nothing in POSIX requires that, and real directories do share -- the dump this was built against has two accounts whose primary gid is a third account's group. Under
preserveIds, a group already holding the gid is referenced rather than duplicated. sudoHost/sudoCommand/sudoUserare honoured when supplied. New accounts still get this directory'sALL/ALLdefault; a migrated account keeps the rule it already had, rather than being silently granted more than the old directory gave it.User.addno longer runs its "email already in use" lookup when the account has no email address.
Fixed
- The docs allowlist gained
ldif-import, so the wizard's help link resolves in-app.
v2.10.0
v2.10.0 - 2026-08-11
- GET /install-agent/authorize: the theta-agent installer's join-key button opens this page; a logged-in site admin gets a join key and is redirected to the installer's loopback callback (loopback-validated).
- The Directory's Windows install command now downloads the installer from its own /resources/theta-agent/ (matching the suite's pinned version) instead of GitHub.
v2.9.0 — multi-site join, replication and mesh fixes
Fixed
- The multi-site join's LDAP import had never worked. Three stacked defects, each hidden because the only report was an
ldap.notestring no test asserted on:ldapAddArgs()omittedargv[0], so the caller ranexecFile('-c', ...)and every join/resync died withspawn -c ENOENT; rawslapcatoutput carries operational attributes (structuralObjectClass,entryUUID,memberOf, …) thatldapaddrejects outright; andldapadd -cexits non-zero on the benign "Already exists" every spoke produces, so even a clean import reported failure. A spoke adopted the resource catalog and signing key but not one user or group. Fixed withstripOperationalAttrs()+summarizeLdapAddResult(); the e2e now seeds a master-only user and requires it to appear in the spoke's own slapd. - Catalog updates and deletions never reached a spoke.
importDirectory()calledResource.update(id, data)andResourceEdge.delete(id)— statics@simpleworkjs/ormhas never had (update/delete are instance methods). Both threw into a swallowingcatch, so renames, metadata edits, edge removals and resource deletions were silently dropped; only new resources ever appeared. The injected test stubs implemented them as statics, which is why the suite stayed green. Stubs now mirror the real ORM shape,tests/orm_method_guard.test.jscoversupdate/delete(and is now actually innpm test— it never ran in CI), and a contract test pins the stubs to the real base model. importDirectory()is now converging rather than destructive. It deleted every edge before recreating them, so a crash mid-import left a truncated graph. Creates land first, extras are removed afterward, and edge endpoints are remapped master-id → slug → local-id (a spoke's own bootstrap rows have local ids, and slugs likeopenrestyare shared across sites, so raw ids pointed at rows that didn't exist). Resource deletions now propagate too, but only for rows this path adopted (__replicatedFromprovenance) — never a spoke's own locally-bootstrapped catalog.- Promotion orphaned every site beyond the second. The promoted node was a spoke, so its own
SiteSpokeregistry was empty andreplicateToSpokes('master-promoted')fanned out to nobody, while sibling spokes kept following the demoted master.POST /api/site/demotenow hands its registry (including each spoke'spushToken) to the incoming master, which adopts it and re-points each sibling via the newPOST /api/site/master-changed. Two-site clusters always worked, which is why this was invisible. - Concurrent spoke registrations were assigned the same LDAP ServerID.
nextFreeLdapServerId()+SiteSpoke.create()is a read-then-write; two simultaneous joins both read the same used-set and both got id 2. Duplicate ServerIDs don't fail loudly — they break MMR, since ServerID is how syncrepl tells originators apart. Serialized with a newutils/mutex.js— and, because that lock is process-local and would protect nothing if this app were ever run as two processes against one database, enforced for real by a unique index (see below). unique: trueon a model field never reached the database. Two independent gaps:@simpleworkjs/ormonly forwardsuniquefor string fields (IntegerField.toSequelize()drops it), and the ORM callssequelize.sync()with no options, which creates missing tables but never alters existing ones — so evenSiteSpoke.endpoint's long-declared constraint does not exist on any already-deployed site.models/index.js's newensureUniqueIndexes()adds the indexes explicitly, on the same add-only, fail-soft terms as the existinghealSchema(). A database written before this can already hold duplicate ServerIDs, which would makeaddIndexfail, sorepairDuplicateServerIds()runs first: the oldest registration keeps the id, the rest move to free ones (safe unattended — a spoke re-reads its ServerID from the master on every reconcile, and the duplicate state was already broken).- The mutex could deadlock silently. Re-entering the same lock — directly, or by an outbound call that comes back into a route taking it — waited forever on a promise that could never settle, with the symptom surfacing somewhere else entirely (during promotion it appeared on the calling node as
re-point failed: This operation was aborted, its own fetch timeout, while the callee sat waiting on itself). Same-context re-entry now throws immediately viaAsyncLocalStorage; cross-node re-entry hits an acquisition timeout that names the current holder.withLockalso no longer passes the previous holder's resolution value into the next callback, and drops its bookkeeping entry once a lock is idle so data-derived lock names cannot leak. - A base-DN mismatch between sites now fails the join up front.
CFG_DOMAINmust be identical cluster-wide (MMR replicas cannot diverge on base DN), but a mismatch used to half-succeed: catalog and signing key adopted, LDAP silently rejected entry-by-entry with "no global superior knowledge". - The mesh carried no service traffic. Consumers dialled
172.24.<idx>.1:3001directly — an address that exists only inside the peer gateway's network namespace, unreachable from thesso-manager/theta-proxycontainers told to use it, with nothing listening on:3001there anyway. No-inbound relay routes and mesh-preferred resync pushes both resolved to a dead target.utils/mesh_route.jsnow routes via the LOCAL gateway's per-peer forwarding port, derived from the mesh index (seetheta-gateway'sservices/mesh_forwarder.js). - LDAP tunnel relay sockets were unbounded. No idle timeout, no connect deadline, no per-agent ceiling, so an agent that vanished without a clean close stranded its relay sockets indefinitely; and
cleanup(agentId)tore down a reconnecting agent's new sockets along with the old ones. Now bounded on all three axes and scoped per WebSocket. - The Multi-Site modal's confirmations did nothing.
app.messages.confirm()renders into a.actionMessageelement and its promise never settles without one — the modal had none, so both the spoke-remove and the pre-existing promote confirmation silently no-op'd on click. Added the region; per-row removal uses an inline row-scoped confirm instead (a single shared banner desyncs across rows).
Added
- LDAP replication config is applied live — no
setup.shre-run, on any node.slapdnow runs from thecn=configdynamic backend (converted at container start from the generatedslapd.confseed), soolcServerID/olcSyncreplare modifiable while it serves.utils/ldap_runtime_config.jsconverges the running config on the cluster's view (read, diff, apply only what differs);utils/ldap_reconcile.jstriggers it on spoke registration/removal, join, resync, master-changed, promotion, boot, and a periodic sweep. Previously every site's peer list went stale each time any site joined, fixable only by an operator re-runningsetup.sheverywhere. Drift detection is retained as a fault indicator. - Registered Spokes is actionable:
DELETE /api/site/spokes/:id,POST /api/site/spokes/resync(awaited, so "Sync now" reports real reachability), andGET /api/site/spokes. POST /api/site/reregister— the recovery path when a spoke and its master disagree about the push token (a removed/recreated registry row, or a join made withoutselfUrl).POST /joincannot fix it: it refuses once a node is a spoke.- Three-site e2e (
docker-compose.multisite-e2e.yml) covering the promotion handoff, live LDAP replication config, LDAP tree replication, and resource update/delete convergence — all previously uncovered.
v2.8.0
v2.8.0 - 2026-08-11
Fixed
- Promotion no longer orphans the demoted old master's LDAP replication. Neither
/site-promotenor/demotetouchedSiteSpoke-- the demoted old master got a fresh join key but noSiteSpokeentry on the new master (noldapServerId, invisible to the peer list), and structurally could never self-heal via/join(refuses re-join for a node that's already a spoke)./demotenow registers itself with the new master immediately, deriving its own endpoint fromstack.selfUrl/stack.ssoHost./site-promote's response also now surfaces that the promoted node's own OpenLDAP ServerID needs asetup.shre-run to actually apply. - The Directory's site slug and the multi-site replication identity are unified. These were two unrelated values that happened to share a name -- a deployment could show a real site name in the Directory catalog and the literal
site-defaultfallback on the Multi-Site modal for the same node.POST /resourcesnow syncssite_config'ssiteSlugto match the moment this node's own site Resource is first created, for a still-default master only.
Added
- LDAP replication status + per-spoke detail on the Multi-Site modal. New
utils/ldap_replication.js'scurrentSlapdServerId()reads the actual running ServerID from this node's ownslapd.conf-- distinct from what the API currently advertises, which can genuinely disagree right after a promotion or a new spoke joining.GET /directory-admin/site-statusnow surfaces both plus astaleflag and a full spokes list (endpoint, assignedldapServerId, relay path), not just an aggregate count.
v2.7.0
v2.7.0 - 2026-08-11
Added
- OpenLDAP N-way multi-master replication auto-config.
SiteSpoke.ldapServerIdis now auto-assigned at registration (next free from 2 upward, 1 reserved for the master -- same pattern as jump-host's mesh index), and each site's LDAP URL is derived from its already-known HTTP(S) endpoint rather than a separately-configured field. Newutils/ldap_replication.js,GET /api/site/ldap-peers(spoke-facing, Bearer site join key) andGET /directory-admin/ldap-replication-config(master-local). Operators no longer hand-maintainLDAP_SERVER_ID/LDAP_REPLICATION_HOSTSfor atheta-suite-joined cluster (seetheta-suite'sbootstrap/site-ldap-register.js). Verified against real running containers (docker-compose.multisite-e2e.yml).