Skip to content

Releases: theta42/theta-directory

v2.16.0

Choose a tag to compare

@wmantly wmantly released this 12 Aug 16:14
114bd7d
  • feat: model events are standardized, so storage backend no longer decides what can be live. The ORM announced changes for models it managed; everything else was silent, which meant LDAP groups and users, Redis-backed notifications and PATs could not update a page no matter what the view did. utils/model_events gives them the identical contract — model:<Name>:<action> with {model, action, pk, data} — so a subscriber cannot tell which backend a model uses. data goes through toJSON() (stripping isPrivate fields), and a delete never carries a body, enforced in the emitter rather than trusted to each call site. ORM and bespoke models share one filtered bus, so "does this model have a read gate?" is answered in exactly one place.
  • feat: LDAP groups and users announce their changes. Group covers add, remove and member/owner changes; User covers add, update and remove. The users and profile views consume them, so a user added or a group membership changed by another admin now shows up without a refresh — including on someone's own profile, where a stale page is most misleading since it is showing them their own access.
  • security: read gates for every model whose data the UI renders — ResourceEdge, Group, User, Notification, ApiToken, MeshClient, MeshSite, MeshExitGrant, AccessRequest, Agent. Several are row-level, not merely model-level: a user receives their own User record, notifications, PATs and mesh clients and nobody else's, while a directory admin receives all of them except PATs — which have no admin path because the REST route has none either.
  • security: User payloads strip userPassword explicitly. It is present on a record read with attributes: ['*','+'] as the admin bind, and stayed off the wire only because user_parse() sets it to undefined inside an if branch — an incidental protection in an unrelated function, not somewhere to hang a credential.
  • note: the Group gate mirrors its currently ungated REST route (routes/group.js:8) rather than being tightened past it. A socket stricter than the endpoint feeding the same page shows a list that silently stops updating; tighten the route and the gate follows automatically.

v2.15.0

Choose a tag to compare

@wmantly wmantly released this 12 Aug 15:42
d02f3ba
  • feat: the Directory and Discovery Plugins views update themselves. A resource added, renamed or removed by another admin (or by a discovery plugin run) now appears without a refresh. The Directory table is derived — hierarchy from the edge list, hostName from the parent, agent status from another service, indentation and carets recomputed by renderTable() — so a change re-derives the view rather than patching one row, which would leave a new child at the wrong depth with no caret on its parent. loadResources() re-reads the search, sort and secrets-only controls, so the operator's filter survives. Debounced, since one edit emits several events and an import emits a burst.
  • fix: the socket never connected at all. authIO called Auth.checkToken(tok) with a bare string where {token} is expected, then called token.getUser() — but checkToken returns the User itself and has no such method. Every handshake failed with token.getUser is not a function, so no client in this app has ever had a working socket.
  • fix: nothing published. @simpleworkjs/orm has a pubsub hook that emits model:<Name>:<action> on save/delete, and it was never wired. It is now, through utils/socket_pubsub.ormBus — a filter that forwards only models with a socket read gate. The ORM publishes for everything it loads, including AuthToken, OtpToken and PasswordResetToken, written on every login and password reset; those must never reach a browser and would be constant bus churn besides.
  • security: READERS is the single source of truth for what goes live. A model listed there both publishes (LIVE_MODELS is derived from it) and is authorized there, so the two cannot drift — publishing something ungated would leak, gating something that never publishes would be dead code. Resource, ResourceGroup and PluginInstance are gated to the same admin groups that guard their REST routes, resolved transitively from LDAP as utils/permission.byGroup does and cached briefly per socket.
  • note: views/plugins.ejs is dead code — /plugins redirects to /directory and that UI was ported into directory.ejs. The Discovery Plugins subscription lives with the code that actually runs.

v2.14.0

Choose a tag to compare

@wmantly wmantly released this 12 Aug 07:27
a4bf2b6
  • security: the socket bridge rebroadcast whatever a client published. socket.on('P2PSub') took any topic and payload an authenticated client emitted and fanned it out to every other connected client. No app code has ever called app.publish(), so nothing legitimate used it. Events now flow server → client only.
  • security: the outbound side was an unconditional app.io.emit of every event on the bus, with its full record, to every authenticated socket. Nothing here publishes model events yet, so it carried no traffic — but it would have started leaking the moment anything did. It is replaced by utils/socket_pubsub.js, a per-socket read gate whose READERS table is empty by design: nothing is broadcast until a model opts in together with the check that decides who may see its events.
  • fix: a delete event carries a null body, and the client tagged it unconditionally — throwing and killing the socket handler.
  • feat: the shared UI shell loads app.sync.js and app.filter.js from @simpleworkjs/frontend, so views here can adopt live updates and filtering.
  • chore: authIO records the session's groups on the socket, which is what a read gate resolves rights from.

v2.13.0

Choose a tag to compare

@wmantly wmantly released this 12 Aug 05:40
be9cec1

v2.13.0

Security

  • PUT /api/mesh/self is admin-gated. It writes this site's WireGuard identity, exit posture and LAN/DNS mapping — previously any authenticated user could clobber the gateway public key (breaking every tunnel to this site cluster-wide), flip exitOpen, or remap DNS/LAN. The legitimate gateway authenticates with a PAT minted as the directory admin, so requireAdmin lets it through while keeping everyone else out.
  • GET /api/mesh/peers and /api/mesh/site-clients are admin-gated. Both expose the full network map and device registry, and exist for the gateway to consume.
  • GET /api/mesh/roster scrubs WireGuard keys/endpoints for non-admins. The mesh page still shows site metadata, but no longer hands a regular user full network enumeration (gatewayPublished boolean instead of the key).

Fixed

  • The Multi-Site modal's gateway-mesh count no longer 404s. It called GET /api/mesh/gateways, an endpoint that died with the mesh-v2 rewrite. The count is now computed locally from the MeshSite roster (the directory IS the registry in v2).
  • The LDIF import confirm is no longer a native confirm() — it uses the app's promise-based app.messages.confirm(...) modal.

Other

  • Synced the served theta-agent/install.sh with theta-agent v2.5.0 (join-key merge + Linux tray arch).

v2.12.0 — the site network

Choose a tag to compare

@wmantly wmantly released this 12 Aug 02:07
c423ae4

v2.12.0

The directory is now where the WireGuard cluster is configured. Sites, devices,
LAN mapping and internet exits all live here; each site's gateway reads the
roster and configures itself. Requires theta-gateway v3.0.0.

  • feat: the site network. A site that joins the directory is on the network
    — no separate mesh to set up. siteId IS the site's ldapServerId, already
    allocated once cluster-wide by the master at join time (under a lock, after
    two spokes were once handed the same one), so the mesh needs no allocator of
    its own. That one number drives LDAP replication, the gateway's
    172.24.0.<id> address, and the site's 10.<id>.0.0/16.
  • feat: /network page, visible to any signed-in user because enrolling
    your own devices is not an admin task. Sites tab shows each site's addresses
    and whether its gateway has actually published a key — joined-but-not-started
    is a real state that used to look identical to healthy. Devices tab enrols
    hardware and picks an exit. Exit Access tab is admin-only.
  • feat: devices with optional key custody. Supply a public key and the
    private half never reaches the server; omit it and one is generated, rendered
    into a config once, and forgotten — never stored, never logged, not
    recoverable. Addresses come from the site's own pool, lowest-free so a
    removed device returns its address rather than a counter marching upward.
  • feat: per-device internet exits. Two independent things: a site marked
    exitOpen is saying it is willing to carry traffic; a grant says a user may
    use it. Revoking a grant immediately drops affected devices back to local
    breakout rather than leaving them routed somewhere they may no longer go.
    Exit choice is a routing rule on the gateway, so switching exits produces a
    byte-identical device config and needs no reconnect.
  • feat: DNS is pushed as the mapped address, not the physical one. A device
    handed 192.168.1.1 only resolves while sitting on that LAN; over the tunnel
    what is routed is the shadow range. The site form shows the translation as
    you type and warns when an address is outside both mapped LANs, where it
    cannot be mapped and devices would get no resolver at all.
  • feat: MTU clamped to 1380 in device configs. Mesh-then-exit is WireGuard
    inside WireGuard, and a device sized for one hop blackholes large packets on
    the second — the classic "SSH works, HTTPS hangs".
  • feat: devices running theta-agent are configured over the agent's existing
    websocket instead of a human copying a file. The push deliberately carries no
    private key: the agent holds its own.
  • feat: the hub — the site carrying 10.0.0.0/8 as a catch-all — is chosen in
    the UI rather than implied by which site holds the master directory, since
    the natural hub is a cheap always-up VPS.
  • change!: utils/mesh_route.js stops being a workaround. It existed because
    WireGuard lived inside the gateway container's namespace, so the only path to
    a peer was a userspace relay on a port derived from the site index. A peer
    site's directory is now simply 10.<siteId>.0.2:3001, and the derived-port
    contract the two repos had to keep in sync is gone. An address in the retired
    scheme deliberately resolves to nothing rather than silently naming a
    different site. The public-endpoint fallback stays, so a deployment whose
    containers have no route into the mesh still replicates over the internet.
  • fix: the roster now reaches every site. It is written at each site (a
    gateway publishes to its own directory) but replication only flows
    master -> spoke, so two halves were missing: a spoke's public key never left
    the spoke, and a spoke never learned any other site existed. Either alone
    means the mesh works only at whichever site happens to be the master. Spokes
    now forward their gateway details over POST /api/site/spokes — the channel
    they already have, with the credential they already hold — and the master
    carries the whole roster in its export. Roster edits push a resync
    immediately instead of waiting for an unrelated catalog change. A site's own
    row is never overwritten by an incoming export, since the local copy is
    always at least as fresh.
  • fix: exit interfaces need their own key. A gateway's exit interface and
    its mesh interface presented the same public key to the same remote, which
    keeps one endpoint and one session per peer key — so the remote's endpoint
    flapped between the two and they invalidated each other's session. Verified
    against wireguard-go: with one key on two interfaces the remote settled on
    whichever handshook last while both kept re-handshaking; with separate keys
    it holds two stable peers. Gateways now publish a second gatewayExitPublicKey
    and GET /api/mesh/peers tells an exit site which gateways to accept under
    it, allowed only the device addresses actually using that exit.
  • docs: The Site Network.

v2.11.0 — import an existing LDAP directory

Choose a tag to compare

@wmantly wmantly released this 11 Aug 22:10
3622050

v2.11.0 - 2026-08-11

Added

  • Import users and groups from an existing LDAP directory. New wizard at Users → Import LDIF (/users/import, admin-only) takes a slapcat/ldapsearch export and migrates accounts through the app's own model layer -- so every imported account arrives with a UserVerification row, a personal group, cache invalidation and service-account membership, exactly as if it had been created here. Two guarantees drive the design: uidNumber/gidNumber are preserved verbatim (they are what every file on every host is owned by -- reallocating them turns a migration into a filesystem-wide chown), and userPassword is carried across as the stored hash, never re-hashed, so people keep the password they already have. See docs/ldif-import.md.
    • utils/ldif.js -- a standalone RFC 2849 parser: line folding, base64, attribute options, CRLF. It refuses attr:< url values (a file-disclosure primitive handed to whoever uploads the dump) and rejects change records rather than misreading them as content.
    • utils/ldif_import.js -- schema-agnostic profiling, planning and applying. The layout of the source directory is detected and then editable, so a FreeIPA or AD export is a mapping change rather than a code change. Membership is resolved per entry, because real directories mix groupOfNames/member with posixGroup/memberUid and a single file-wide choice silently drops whichever kind loses.
    • Review happens before anything is written: every account shows what it will bring across, what it will not, and why it cannot be imported. Blocked rows (no uidNumber, a duplicate inside the file, a collision with an account already here) are skipped no matter what the client sends.
    • Groups are never created by an import. A group name from another directory carries no meaning in this one, where access is a projection of the resource graph (docs/groups.md). Each source group either has its members merged into a group that already exists, or is dropped; many-to-one is supported for collapsing a sprawling old directory.
    • Carried across where present: password hash, uid/gid numbers, name, email, phone, shell, home directory, description, location, date of birth, every SSH key, sudo rules, and the account's disabled state. Anything else is listed on the row as not migrated rather than dropped quietly.
    • Onboarding is a per-run choice (treat ToS as accepted / email as verified) so a cutover need not re-ask 30 people. Legacy MD5 passwords still always force a change at first login, and no welcome email is sent to anyone.
    • Staging lives in Redis under a one-hour expiry and is destroyed on apply or abandon; the parsed dump is never written to disk and password hashes are never included in an API response.

Changed

  • User.add(data, options) takes an options argument for preserveIds, preserveHash and suppressWelcome. These are a second argument rather than fields on data deliberately: every route calls User.add(req.body), so a uidNumber honoured whenever present would let anyone who can create a user claim uid 0, and a userPassword that skipped hashing whenever it looked hashed would let them plant a known hash. Normal user creation is unchanged.
  • An account may share its primary group with another account. Each account normally gets a personal group of its own, but nothing in POSIX requires that, and real directories do share -- the dump this was built against has two accounts whose primary gid is a third account's group. Under preserveIds, a group already holding the gid is referenced rather than duplicated.
  • sudoHost/sudoCommand/sudoUser are honoured when supplied. New accounts still get this directory's ALL/ALL default; a migrated account keeps the rule it already had, rather than being silently granted more than the old directory gave it.
  • User.add no longer runs its "email already in use" lookup when the account has no email address.

Fixed

  • The docs allowlist gained ldif-import, so the wizard's help link resolves in-app.

v2.10.0

Choose a tag to compare

@wmantly wmantly released this 11 Aug 16:15
0065120

v2.10.0 - 2026-08-11

  • GET /install-agent/authorize: the theta-agent installer's join-key button opens this page; a logged-in site admin gets a join key and is redirected to the installer's loopback callback (loopback-validated).
  • The Directory's Windows install command now downloads the installer from its own /resources/theta-agent/ (matching the suite's pinned version) instead of GitHub.

v2.9.0 — multi-site join, replication and mesh fixes

Choose a tag to compare

@wmantly wmantly released this 11 Aug 15:42
9bfd58d

Fixed

  • The multi-site join's LDAP import had never worked. Three stacked defects, each hidden because the only report was an ldap.note string no test asserted on: ldapAddArgs() omitted argv[0], so the caller ran execFile('-c', ...) and every join/resync died with spawn -c ENOENT; raw slapcat output carries operational attributes (structuralObjectClass, entryUUID, memberOf, …) that ldapadd rejects outright; and ldapadd -c exits non-zero on the benign "Already exists" every spoke produces, so even a clean import reported failure. A spoke adopted the resource catalog and signing key but not one user or group. Fixed with stripOperationalAttrs() + summarizeLdapAddResult(); the e2e now seeds a master-only user and requires it to appear in the spoke's own slapd.
  • Catalog updates and deletions never reached a spoke. importDirectory() called Resource.update(id, data) and ResourceEdge.delete(id) — statics @simpleworkjs/orm has never had (update/delete are instance methods). Both threw into a swallowing catch, so renames, metadata edits, edge removals and resource deletions were silently dropped; only new resources ever appeared. The injected test stubs implemented them as statics, which is why the suite stayed green. Stubs now mirror the real ORM shape, tests/orm_method_guard.test.js covers update/delete (and is now actually in npm test — it never ran in CI), and a contract test pins the stubs to the real base model.
  • importDirectory() is now converging rather than destructive. It deleted every edge before recreating them, so a crash mid-import left a truncated graph. Creates land first, extras are removed afterward, and edge endpoints are remapped master-id → slug → local-id (a spoke's own bootstrap rows have local ids, and slugs like openresty are shared across sites, so raw ids pointed at rows that didn't exist). Resource deletions now propagate too, but only for rows this path adopted (__replicatedFrom provenance) — never a spoke's own locally-bootstrapped catalog.
  • Promotion orphaned every site beyond the second. The promoted node was a spoke, so its own SiteSpoke registry was empty and replicateToSpokes('master-promoted') fanned out to nobody, while sibling spokes kept following the demoted master. POST /api/site/demote now hands its registry (including each spoke's pushToken) to the incoming master, which adopts it and re-points each sibling via the new POST /api/site/master-changed. Two-site clusters always worked, which is why this was invisible.
  • Concurrent spoke registrations were assigned the same LDAP ServerID. nextFreeLdapServerId() + SiteSpoke.create() is a read-then-write; two simultaneous joins both read the same used-set and both got id 2. Duplicate ServerIDs don't fail loudly — they break MMR, since ServerID is how syncrepl tells originators apart. Serialized with a new utils/mutex.js — and, because that lock is process-local and would protect nothing if this app were ever run as two processes against one database, enforced for real by a unique index (see below).
  • unique: true on a model field never reached the database. Two independent gaps: @simpleworkjs/orm only forwards unique for string fields (IntegerField.toSequelize() drops it), and the ORM calls sequelize.sync() with no options, which creates missing tables but never alters existing ones — so even SiteSpoke.endpoint's long-declared constraint does not exist on any already-deployed site. models/index.js's new ensureUniqueIndexes() adds the indexes explicitly, on the same add-only, fail-soft terms as the existing healSchema(). A database written before this can already hold duplicate ServerIDs, which would make addIndex fail, so repairDuplicateServerIds() runs first: the oldest registration keeps the id, the rest move to free ones (safe unattended — a spoke re-reads its ServerID from the master on every reconcile, and the duplicate state was already broken).
  • The mutex could deadlock silently. Re-entering the same lock — directly, or by an outbound call that comes back into a route taking it — waited forever on a promise that could never settle, with the symptom surfacing somewhere else entirely (during promotion it appeared on the calling node as re-point failed: This operation was aborted, its own fetch timeout, while the callee sat waiting on itself). Same-context re-entry now throws immediately via AsyncLocalStorage; cross-node re-entry hits an acquisition timeout that names the current holder. withLock also no longer passes the previous holder's resolution value into the next callback, and drops its bookkeeping entry once a lock is idle so data-derived lock names cannot leak.
  • A base-DN mismatch between sites now fails the join up front. CFG_DOMAIN must be identical cluster-wide (MMR replicas cannot diverge on base DN), but a mismatch used to half-succeed: catalog and signing key adopted, LDAP silently rejected entry-by-entry with "no global superior knowledge".
  • The mesh carried no service traffic. Consumers dialled 172.24.<idx>.1:3001 directly — an address that exists only inside the peer gateway's network namespace, unreachable from the sso-manager/theta-proxy containers told to use it, with nothing listening on :3001 there anyway. No-inbound relay routes and mesh-preferred resync pushes both resolved to a dead target. utils/mesh_route.js now routes via the LOCAL gateway's per-peer forwarding port, derived from the mesh index (see theta-gateway's services/mesh_forwarder.js).
  • LDAP tunnel relay sockets were unbounded. No idle timeout, no connect deadline, no per-agent ceiling, so an agent that vanished without a clean close stranded its relay sockets indefinitely; and cleanup(agentId) tore down a reconnecting agent's new sockets along with the old ones. Now bounded on all three axes and scoped per WebSocket.
  • The Multi-Site modal's confirmations did nothing. app.messages.confirm() renders into a .actionMessage element and its promise never settles without one — the modal had none, so both the spoke-remove and the pre-existing promote confirmation silently no-op'd on click. Added the region; per-row removal uses an inline row-scoped confirm instead (a single shared banner desyncs across rows).

Added

  • LDAP replication config is applied live — no setup.sh re-run, on any node. slapd now runs from the cn=config dynamic backend (converted at container start from the generated slapd.conf seed), so olcServerID/olcSyncrepl are modifiable while it serves. utils/ldap_runtime_config.js converges the running config on the cluster's view (read, diff, apply only what differs); utils/ldap_reconcile.js triggers it on spoke registration/removal, join, resync, master-changed, promotion, boot, and a periodic sweep. Previously every site's peer list went stale each time any site joined, fixable only by an operator re-running setup.sh everywhere. Drift detection is retained as a fault indicator.
  • Registered Spokes is actionable: DELETE /api/site/spokes/:id, POST /api/site/spokes/resync (awaited, so "Sync now" reports real reachability), and GET /api/site/spokes.
  • POST /api/site/reregister — the recovery path when a spoke and its master disagree about the push token (a removed/recreated registry row, or a join made without selfUrl). POST /join cannot fix it: it refuses once a node is a spoke.
  • Three-site e2e (docker-compose.multisite-e2e.yml) covering the promotion handoff, live LDAP replication config, LDAP tree replication, and resource update/delete convergence — all previously uncovered.

v2.8.0

Choose a tag to compare

@wmantly wmantly released this 11 Aug 03:59
7efd3fd

v2.8.0 - 2026-08-11

Fixed

  • Promotion no longer orphans the demoted old master's LDAP replication. Neither /site-promote nor /demote touched SiteSpoke -- the demoted old master got a fresh join key but no SiteSpoke entry on the new master (no ldapServerId, invisible to the peer list), and structurally could never self-heal via /join (refuses re-join for a node that's already a spoke). /demote now registers itself with the new master immediately, deriving its own endpoint from stack.selfUrl/stack.ssoHost. /site-promote's response also now surfaces that the promoted node's own OpenLDAP ServerID needs a setup.sh re-run to actually apply.
  • The Directory's site slug and the multi-site replication identity are unified. These were two unrelated values that happened to share a name -- a deployment could show a real site name in the Directory catalog and the literal site-default fallback on the Multi-Site modal for the same node. POST /resources now syncs site_config's siteSlug to match the moment this node's own site Resource is first created, for a still-default master only.

Added

  • LDAP replication status + per-spoke detail on the Multi-Site modal. New utils/ldap_replication.js's currentSlapdServerId() reads the actual running ServerID from this node's own slapd.conf -- distinct from what the API currently advertises, which can genuinely disagree right after a promotion or a new spoke joining. GET /directory-admin/site-status now surfaces both plus a stale flag and a full spokes list (endpoint, assigned ldapServerId, relay path), not just an aggregate count.

v2.7.0

Choose a tag to compare

@wmantly wmantly released this 11 Aug 03:07
eef7852

v2.7.0 - 2026-08-11

Added

  • OpenLDAP N-way multi-master replication auto-config. SiteSpoke.ldapServerId is now auto-assigned at registration (next free from 2 upward, 1 reserved for the master -- same pattern as jump-host's mesh index), and each site's LDAP URL is derived from its already-known HTTP(S) endpoint rather than a separately-configured field. New utils/ldap_replication.js, GET /api/site/ldap-peers (spoke-facing, Bearer site join key) and GET /directory-admin/ldap-replication-config (master-local). Operators no longer hand-maintain LDAP_SERVER_ID/LDAP_REPLICATION_HOSTS for a theta-suite-joined cluster (see theta-suite's bootstrap/site-ldap-register.js). Verified against real running containers (docker-compose.multisite-e2e.yml).