Directory Sync Developer Guide

Directory sync — Developer Guide

How importing people from a directory is built: the scope model, the safety rules, why preview runs the same code, and the traps that produce a wrong answer looking exactly like a working feature.

The plain-language version is Importing people from Active Directory or LDAP. The design conversation — the questions, the reasoning, and the places the first answer was wrong — is Directory sync — the design. To change the feature, see Extending directory sync.

Sign-in is a separate story with separate code: LDAP — Developer Guide.

Asked for by email, quoted in full on the design page. Engine in 9f6f2b-era slice 2; field mapping b2f18f41; preview detail 652c645c; OU browser 93501943; renamed-branch detection 3dfe8728.

---

1. 📁 The files involved

🟢 The engine

File	Role
includes/directory_sync.php	Everything. Scope arithmetic, paged fetch, matching, apply, managers, deactivation, the brake, the run log
includes/ldap.php	Reused, not duplicated: ldapOpen(), ldapBindService(), ldapStringifyId()
includes/users.php	USER_PERSON_FIELDS, USER_DIRECTORY_OWNED, userManagerIsSafe()
scripts/directory_sync.php	CLI: --provider=N, --all, --preview, --quiet

🟠 The APIs

File	Role
api/system/run_directory_sync.php	Runs one provider, live or preview. Returns the whole run row
api/system/get_directory_sync_log.php	?provider_id= recent runs · ?run_id= one run, person by person
api/system/browse_directory_ous.php	The OU tree plus a head count per branch
api/system/test_directory_mapping.php	Reads ONE person and applies the mapping from the posted form values
api/system/save_sso_provider.php	Writes the provider, including sync_ou_includes / _excludes via ssoDnLines()

⚪ The screens

File	Role
system/sso/provider.php	The five-tab provider page: connection, sign-in, importing, field mapping, history. The OU tree and the run modal are here
system/sso/index.php	The provider list. OIDC still uses its modal; LDAP providers link out to the page above

🔵 Storage

Table	Role
auth_providers	sync_* columns, and the ldap_attr_* mapping shared with sign-in
users	The person fields — see Extending directory sync §1
user_sso_identities	subject = the stringified objectGUID. The identity anchor
directory_sync_runs	One row per run, preview included
directory_sync_entries	One row per person per run: action, detail

🟣 Tests

File	Needs the fixture?	Covers
tests/directory-sync-scopes.php	No	Scope arithmetic. 19 assertions, runs anywhere
tests/directory-sync.sh	Yes	End to end against Samba AD. 26 assertions, exits 0 with SKIP when the container is absent
tests/people-fields.sh	Yes	The person fields and the users screen
docker/ldap-test/seed-ad-people.sh	—	The fixture: ~30 people, 8 OUs, awkward on purpose

---

2. It is not a second LDAP client

directory_sync.php requires ldap.php and reuses its connection and bind. The difference is the question being asked:

	Sign-in (ldap.php)	Import (directory_sync.php)
Question	"Is this one person real, and what are they allowed to be?"	"List everyone"
Scope	One entry, found by substituting into a filter	Whole subtrees, paged
Failure	This person cannot log in	Potentially everybody is marked as having left

That last row is why this file is disproportionately defensive for its size.

---

3. The scope model

Two lists, not one. sync_ou_includes holds ticked branches; sync_ou_excludes holds carve-outs within them. One DN per line, and dsyncScopes() is the only thing that reads them.

A branch means the whole branch, now and in future. So what is stored is the exceptions, not the members. Storing every member OU would freeze the selection at today's shape of the directory, and nothing would ever say it had gone stale.

dsyncScopes($provider) -> ['includes' => [...], 'excludes' => [...]]

It does three things:

1. Falls back. Empty includes → sync_base_dn → ldap_base_dn.
2. Lower-cases everything. A DN is case-insensitive; string comparison is not, and every test in this file is a string comparison.
3. Prunes overlaps. Ticking a parent and its child drops the child, so one search covers it.

🔴 The upgrade path is the whole reason for rule 1. Both columns NULL is the only state an install predating the OU browser can be in, and it must import exactly who it imported yesterday. Without the fallback, upgrading imports nobody — and then the only thing between that and every person in the company being marked as left is the sanity brake. Asserted explicitly in tests/directory-sync-scopes.php.

🔴 The comma is load-bearing

function dsyncDnIsUnder(string $dn, string $ancestor): bool
{
    return $dn === $ancestor || str_ends_with($dn, ',' . $ancestor);
}

Drop the , and OU=Sales,DC=x tests true against OU=WholesaleSales,DC=x. A carve-out then silently swallows an unrelated department — a wrong answer that looks exactly like a working feature, which is the worst kind. Asserted in both directions.

Carve-outs are applied in PHP, not in the filter

dsyncDnIsExcluded() runs over each entry after it comes back. This is not laziness: "not under this subtree" is a statement about position in the tree, and an LDAP filter tests attributes. It cannot be expressed as a filter at all.

Results are keyed by DN

dsyncFetchPeople() accumulates into $byDn. Overlapping ticks are already pruned, but referrals and aliases can still hand the same entry back twice — and a duplicate is not merely wasted work, it is counted twice by the brake. An inflated baseline is the dangerous half.

---

4. The safety rules

Four independent things stop this feature destroying a customer's user list. They are independent on purpose: each one covers a failure the others cannot see.

4.1 The sanity brake — syncBrakeTripped()

Runs before anything is written. If a run finds sync_brake_percent fewer people than sync_last_count, it stops with status stopped and changes nothing.

if ($pct <= 0 || $last === null || $last === 0) return null;   // no baseline yet
if ($seen >= $last) return null;                                // grew, or level

⚠️ A preview never becomes the baseline. It changed nothing, so it proves nothing about the state of the world. Only a successful live run writes sync_last_count.

🔴 The brake only guards against a DROP. People being added — which is what a vanished carve-out causes — is completely invisible to it. That is why §4.4 exists.

4.2 N consecutive misses — dsyncHandleMissing()

Missing once is noise; missing sync_deactivate_after runs is a fact. sync_missed_count increments, and only on reaching the threshold is somebody marked as left. 0 disables it entirely.

Nobody is ever deleted. Assets, tickets and handover documents all hang off user_id.

4.3 Preview runs the identical code path

There is no separate preview implementation to drift from the real one. directorySyncRun($conn, $provider, 'preview') walks exactly the same code with writes suppressed at seven points — if (!$preview) around each UPDATE/INSERT, and dsyncHandleMissing() returning early.

⚠️ A new write must be added inside a $preview guard. A write that escapes one makes preview lie, and preview lying is worse than having no preview: it is trusted precisely because it is supposed to change nothing.

The run log is written either way. That is deliberate — what somebody was shown before they pressed the button is worth being able to check afterwards.

4.4 Renamed branches — dsyncMissingScopes()

A ticked branch is stored as a DN, and a DN changes when somebody reorganises an OU. Both directions failed silently before this existed:

Gone	Behaviour	Why
An include	throw — the run fails, changes nothing, names the branch	Importing the branches that do still exist is exactly how everybody in the missing one drifts into being marked as left. Fail-fast beats a partial import
An exclude	Runs, logs an error entry, sets the run message	Unwanted people arriving is additive and undoable. Refusing would stop everybody's import over the smaller problem

Skipped entirely when sync_ou_includes is empty — installs on a typed base DN have always behaved this way, and warning on every run of a working setup only teaches people to ignore warnings.

The brake message prepends rather than replaces ($message = $brake . "\n\n" . $message). A vanished branch is very often why the count dropped, and reporting the effect while discarding the cause leaves somebody staring at "far fewer people than last time" with the explanation already thrown away.

---

5. Matching a directory person to a FreeITSM one

dsyncFindExisting() tries three things, in descending order of reliability:

Order	By	Survives
1	user_sso_identities.subject (the GUID)	Renames and OU moves
2	users.directory_username	Records linked before GUIDs were captured
3	users.email	Somebody who already existed here

🔴 Step 3 is tenancy-scoped and must stay that way. Two companies can legitimately share an address — a contractor, a shared admin@ — and matching across them merges two customers' people. That is a data leak dressed as a convenience.

An email match is a conflict, not a silent adoption: sync_on_conflict decides between adopt (link them, which stops their portal password working) and flag (change nothing, log it).

NULL is not ''

'email' => $email !== '' ? $email : null,

users.email carries a UNIQUE index. MySQL tolerates many NULLs and exactly one '' — so storing '' for a mailbox-less person means the second one is rejected as a duplicate. The fixture has three no-mail accounts specifically to keep this honest.

---

6. Managers are a second pass

manager holds a DN, not a name, and the person it points at may be somebody this very run just created. So dsyncResolveManagers() runs after everybody exists, mapping DN → user_id from $dnToUserId.

userManagerIsSafe() (in includes/users.php) walks the chain before writing, bounded, rejecting cycles. The database cannot express "no cycles" in a self-referencing FK, so this is the only thing standing between an import and an infinite loop in the org chart.

---

7. The run log

Two tables, deliberately: counts alone are not something anybody can act on.

directory_sync_runs      one row per run   -> seen/created/updated/adopted/deactivated/conflict/error, status, message
directory_sync_entries   one row per person -> action, detail

action is one of create · update · adopt · unchanged · deactivate · skip · conflict · error, ordered for display by FIELD(action, 'error','conflict','deactivate','adopt','create','update','skip','unchanged') — attention first, the 400 rows saying nothing last.

detail holds English prose, written at run time, not translation keys. The log is written once and read later; a key would need a locale to render in and the run has no user. dsyncFieldLabel() keeps column names out of it.

📌 These entries existed and were served by the API from the day the engine shipped. Nothing displayed them for two slices. Preview produced four numbers and asked to be trusted. If you add something to the log, check something renders it.

---

8. The traps

8.1 ldap_get_entries() lower-cases attribute names

Every key comes back lower-cased regardless of what you asked for. dsyncValue() handles it; do not index $entry['sAMAccountName'] directly.

8.2 AD caps at 1000 entries, silently

Without LDAP_CONTROL_PAGEDRESULTS you get 1000 people and no error. On the 1001st-person directory that reads as "everybody else has left" — which is precisely the input the brake exists to catch, and you would rather not be relying on it. DSYNC_PAGE_SIZE = 500.

8.3 userAccountControl bit 2 is AD-only

DSYNC_UAC_DISABLED = 2. A directory without the attribute reports 0, which reads as enabled — the right default, since we should never invent a reason to deactivate somebody.

8.4 objectGUID is binary

ldapStringifyId() (from ldap.php) converts it. Printing it raw produces mojibake; test_directory_mapping.php reports (binary) rather than emitting it.

8.5 The bind password is encrypted at rest

Every entry point must call decryptValue($provider['ldap_bind_password']) before ldapBindService(). Miss it and the bind fails as "Invalid credentials", which reads like a wrong password rather than an un-decrypted one. Cost an afternoon once.

8.6 A DN component can contain an escaped comma

Splitting a DN naively cuts such a name in half and orphans everything beneath it. Use preg_split('/(?<!\\\\),/', $dn, 2) — see ouBrowseParent().

---

9. Testing

php tests/directory-sync-scopes.php     # no fixture needed
bash tests/directory-sync.sh            # needs freeitsm-samba-ad

Bring the fixture up:

docker start freeitsm-samba-ad
bash docker/ldap-test/seed-ad-company.sh && bash docker/ldap-test/seed-ad-people.sh

🔴 A count is not a test of a carve-out. "35 people instead of 37" is satisfied exactly as well by excluding the wrong two people. The assertions name who must be present and who must not. The first attempt at that control asserted on contractor names that had been invented rather than looked up, passed, and proved nothing — the real ones are z.contractor and w.vendor.

⚠️ Pair every "cannot do X" with a positive control. tests/directory-sync.sh proves the brake stops a run and that the same run proceeds with the brake off; that real branches produce no warning and that missing ones do. Without the control, "no warning" is indistinguishable from a check that never ran.

The fixture is awkward on purpose — a tidy directory would let us build a sync that only works on tidy directories. Contractors that must not sync, two people with the same display name, an email that is another person's username, three accounts with no mail, one person disabled in place and another disabled and moved, a three-deep manager chain, IT reporting to Finance, and non-ASCII names.

---

10. Known gaps

• Scheduling. The CLI exists; nothing drives it. The only unbuilt part of the feature.
• OU identity across renames. Detected (§4.4), not self-healed. Storing each OU's GUID would make a rename a non-event rather than a message.
• Custom attributes. The design page proposes a key/value side table; not built.
• Analysts. Import creates users only. JIT is probably the right answer for analysts — bulk-creating them is a security event, not a convenience.
• ms and uk translations of the sso namespace.

---

See also

• Importing people from a directory — the user-facing guide
• Directory sync — the design — why it works this way
• Extending directory sync — adding a field, supporting another directory
• LDAP — Developer Guide — sign-in, which shares the connection
• Setting up LDAP with Docker — the fixtures