Skip to content

CardDAV Write Back Internals

Jump to bottom
Ed Mozley edited this page Sep 17, 2026 · 4 revisions

CardDAV write-back — internals

How FreeITSM sends a change made here back to a contact card, without damaging the card. Asked for in #133, where the argument for it is that a phone number changes during the call and correcting it again in a second application is exactly the double entry a service desk tool ought to remove.

This page covers the write half only. Reading — connecting, PROPFIND, REPORT, and turning a vCard into a person — is on the developer guide; the import lifecycle is on CardDAV import internals; the administrator's view is on CardDAV contact sync.

1. Why this is a separate file, and a separate page

includes/carddav_write.php exists so that a sentence stays true:

The import issues PROPFIND and REPORT and nothing else.

That is the honest answer when an operator asks whether connecting FreeITSM can damage their address book, and it is worth more as a fact about the code than as a promise about intentions. PUT lives in a file the import path never loads. Nothing about switching write-back on changes what an import does.

The page is separate for the same reason the file is: reading a vCard is a small idea, and editing one safely is not.

2. The one rule everything else follows

🔴 Never generate a vCard. Edit the bytes the server sent.

The obvious implementation takes the person fields, renders a card, and PUTs it. That silently destroys everything FreeITSM does not model:

On the card FreeITSM models it?
PHOTO no
BDAY no
NOTE no
ADR street, region, postcode, country only the town
X-* properties written by the operator's CRM no
CATEGORIES read for scope, never stored
item1.X-ABLabel and other Apple group labels no

A regenerating implementation looks like it worked. The damage is discovered months later, and there is nothing on our side to restore from.

So cardDavApplyPersonEdits() rewrites only the line whose value changed and passes every other line through untouched, understood or not.

🔑 This also disposes of the vCard 3.0 versus 4.0 question, which otherwise needs a policy nobody wants to own. The card keeps whatever VERSION it already declares, because nothing rewrites it.

3. The pieces

Function Job
cardDavLineSpans() split into logical lines remembering byte offsets, the counterpart to cardDavUnfold() which throws them away
cardDavLineProperty() the property name of a line, group prefix stripped
cardDavReplaceLineValue() swap a value, keeping the name, the group prefix and every parameter
cardDavInsertLine() add a property before END:VCARD
cardDavRemoveLine() delete a line and its continuations
cardDavSetComponent() change one field inside a semicolon-separated value
cardDavFoldLine() fold at 75 octets, UTF-8 safe
cardDavEscapeText() / cardDavUnescapeText() the vCard TEXT escaping pair (these two live in carddav.php, because reading needs them too)
cardDavApplyPersonEdits() the whole surgical edit, field by field
cardDavGetCard() / cardDavPutCard() one card in, one card out
cardDavCanWrite() ask the server what this account may do
cardDavPushPersonChanges() the orchestrator, including the conflict rule

4. What is writable, and what is deliberately not

Exactly five, matching USER_CARDDAV_OWNED in includes/users.php:

Field Where it lives on the card
job_title TITLE, the whole value
phone TEL without a CELL type
mobile TEL;TYPE=CELL
department ORG, component 1 of 2
office ADR, component 3 of 7 (the locality)

employee_id and manager_id are never sent. A vCard has nowhere to keep a payroll number, and almost nothing in the wild writes RELATED;TYPE=manager.

⚠️ This is also why they became editable. Before this work, USER_DIRECTORY_OWNED greyed out all seven person fields on any managed record — so an imported contact had their employee ID and manager locked against edits on behalf of an import that would never populate them. userDirectoryOwnedFields($protocol) now answers per protocol: LDAP and OIDC keep all seven, CardDAV owns five.

Structured values are edited per component

ORG:Acme Ltd;Finance and ADR:;;12 High St;Norwich;;NR1 1AA;UK each hold several fields in one line. Writing the whole value back would erase the company name and the entire street address respectively.

🔑 An emptied component is blanked in place, never removed. Deleting the ADR line because somebody cleared the office would take the postcode with it. A whole-value field like TITLE is removed when cleared, because there is nothing else on that line.

⚠️ cardDavSetComponent() pads with empty components when the stored value is shorter than the index being written. A card carrying only ADR:;;12 High St has no town slot at all, and appending one needs the separators in between or the town lands in street.

5. 🔴 Read and write share one selector

A card routinely carries three or four TEL lines. "The mobile" is not a property name — it is the result of a selection rule: first CELL, else first WORK, else the first untyped one.

If the writer re-implemented that rule and got it even slightly different, editing somebody's mobile would overwrite their desk number and leave the mobile untouched, with no error anywhere.

So cdsyncPhoneLines() was split out of cdsyncPhones() and returns line indexes. The reader takes the values at those indexes; cardDavLocateField() writes to the same ones. One selector, used by both ends, makes the mismatch unrepresentable rather than merely unlikely.

6. Escaping, and the trap in it

RFC 6350 §3.4: inside a text value, \\, \,, \; and \n stand for a backslash, comma, semicolon and line break.

🔴 A single left-to-right pass, not chained str_replace calls.

Chaining gets \\, wrong in both directions. Unescaping \\ first leaves \,, which the next pass then reads as an escaped comma — so a value that legitimately ended in a backslash before a comma comes out having silently lost the separator.

⚠️ Two of the four import mappers never unescaped at all. cdsyncOrg() and cdsyncLocality() handled \; and \,; TITLE and TEL did not, so a job title of Head\, Sales imported with the backslash in it. Harmless while the sync was one-way — and compounding the moment a value can be written back, since it would be re-escaped to Head\\, Sales and the card would gain a backslash on every run.

The property that matters is not "escapes correctly once" but round-trips without drift: a value read and written five times must come back identical. That is what the tests assert.

7. Folding, and why it is UTF-8 aware

The 75-character limit is defined in octets, not characters. A naive substr($s, 0, 75) splits a multi-byte character across the fold, and the two halves either side rejoin into a byte sequence that is not valid UTF-8.

This is not an edge case for FreeITSM: there are real users running Polish and Danish installs, and ordinary names and job titles hit it.

cardDavFoldLine() walks back off a continuation byte (10xxxxxx), at most three bytes, so a character is never cut in two. Continuations carry a leading space, which counts toward the 75.

⚠️ Testing this needs a case that would actually have broken. A test that folds a long string and checks the result is valid UTF-8 passes trivially when the boundary happens to land cleanly. The suite searches for paddings where the naive cut would have split the character, asserts it found at least one, and only then checks the guard — for 2-, 3- and 4-byte characters.

8. 🔴 The conflict rule: a per-field three-way merge

Three schemes are plausible and two are wrong.

Scheme Verdict
PUT with the ETag stored at import time Safe but unusable. Refuses whenever any field on the card changed since the last sync. Correcting a phone number is blocked because somebody added a birthday last week.
GET the card, edit it, PUT with the ETag that GET just returned Never refuses anything. This is not convenience — it is the concurrency guard switched off, because If-Match can no longer fail: we just asked the server which value would make it pass.
What FreeITSM does GET fresh; compare the server's current value of each field being written against what FreeITSM held before the save; refuse only on a same-field difference; apply the edit to the fresh bytes; PUT with the fresh ETag.

The third is the only one that both preserves other people's work and notices when two people changed the same thing.

  • Somebody else edited a different property → their change survives, ours goes through.
  • Somebody else edited the same detail → refused, and the operator is told what their version says.
  • Somebody else edits between our GET and our PUT → If-Match catches it, 412.

🔑 A 412 is the feature, not a failure. It means somebody got there first. The right next step is to import again and look at their version, which is what the message says.

⚠️ cardDavPutCard() refuses an empty ETag rather than sending If-Match: *, which means "any existing card" and would be exactly the unguarded overwrite this exists to prevent. No stored ETag means we have not really read this card.

Where the third leg comes from

api/tickets/save_user.php reads the five person fields before the UPDATE, not after — comparing the new value with itself would make every write look uncontested.

9. Asking the server, not keeping a list

There is no roster of "CardDAV servers FreeITSM can write to", and there should not be. Writing is a plain PUT from RFC 6352, the same standard as the reads, so any server the import works against can in principle be written to.

What genuinely varies is permission. A shared, subscribed or published address book is very commonly read-only however correct the password is.

cardDavCanWrite() issues a PROPFIND for current-user-privilege-set and looks for all, write or write-content.

🔴 It writes nothing. Testing by creating a card and deleting it again is worse than it sounds: the delete can fail, and the operator is left with a contact called "FreeITSM test" in their real address book.

⚠️ A server that does not report privileges at all is treated as unknown, not permitted — the setting stays off and the screen says the server did not answer, rather than pretending it said no.

10. Storage

user_sso_identities gained two columns:

Column Meaning
source_ref where the record lives in the source — an LDAP distinguished name, or the URL of the individual .vcf
source_etag the source's version marker (a CardDAV ETag). NULL for LDAP and OIDC

Named for the job rather than the protocol, because subject on that table already means two different things depending on the provider.

Without source_ref, changing one contact's phone number would mean re-fetching the entire address book to find their card — slow, and a race against anybody editing in between.

🔴 Both are refreshed on every sighting, not just on first link. A card that moves between address books gets a new URL, and its ETag changes whenever anybody edits it. A stale pair is worse than none: the href would write to a card that has moved, and the ETag would make every write look like a conflict.

11. A failed write-back never loses the analyst's work

The push happens after FreeITSM's own record is saved, and cannot undo it.

An address book that is unreachable, read-only, or holding a newer value is a fact to report — not a reason to throw away the edit somebody just made, about a problem they cannot do anything about. Every outcome is returned; none of them is an exception.

save_user.php adds an address_book block to its response only when a write was actually attempted, so an ordinary save carries no new keys and nothing downstream has to learn about a feature it does not use.

12. The safety asymmetry, stated once

A bad import damages our data, and can be re-imported. A bad write-back damages the operator's address book, and we cannot undo it at all.

That is why write-back is deliberately narrower than import rather than its mirror: only fields FreeITSM owns, only on cards it imported, only when a human deliberately edited one.

There is no bulk reconciliation pass, and there should never be one. "Put the whole address book right" is the feature request that turns a careful integration into a data-loss incident.

13. What happens when it fails, and how anyone finds out

📖 This section is the why. For how to actually diagnose a broken write-back — what each result means, how to read the log, both tools rung by rung, and a symptom-to-cause table — see CardDAV write-back — when it goes wrong.

Three things, and the first two were missing from the first cut of this feature.

The analyst is always told

save_user.php returns an address_book block, and both people editors render it — Tickets and Assets. That second half was briefly missing and it mattered: the API reported the failure perfectly and no screen read it, so an analyst on an unreachable server saw "Saved" and had no way to learn the card had not been updated. A silent failure, in the feature built end to end to avoid them.

⚠️ The wording leads with "Saved here, but…". The first thing somebody needs to know is that their own work is safe; what happened to the address book is the second sentence.

A dead server costs seconds, not the form

A write-back runs inside a save. CARDDAV_INTERACTIVE_TIMEOUT (8s, 4s to connect) applies only on the push path, because the import's 20 seconds is fine for something you started and are watching, and unbearable when you have pressed Save and the page has gone quiet. Measured against a closed port: 2 seconds.

Every attempt is logged

carddav_write_log, one row per attempt, surfaced under Changes sent back on the History tab.

Outcome Means
ok written to the card
conflict refused on purpose — somebody had changed the same detail
failed the server could not be reached, or said no
skipped nothing needed sending

🔑 The server's own response is stored verbatim, truncated rather than summarised. When an address book refuses something the reason is in its error body, and a paraphrase is useless for diagnosing a server this install cannot log into.

🔑 conflict is coloured as a warning, not a danger. Refusing to overwrite a newer value is the safety net working. Red teaches an operator to treat it as a fault to clear, which is the opposite of what should happen.

⚠️ The log is deleted explicitly when a provider is deleted, in delete_sso_provider.php, rather than left to the foreign key. database/freeitsm.sql declares ON DELETE CASCADE, but Database Verify creates tables from a column list and adds no constraints — so on an install where the table arrived via a verify there is no cascade at all. Confirmed rather than assumed: directory_sync_entries has the same 0 foreign keys, so this is a general property of verified tables.

14. The two diagnostics

Both under System → Debug Tools, and they answer genuinely different questions.

D015 — CardDAV health

Walks the connection one rung at a time: reach → sign in → read the book → may this account write. Reported separately, because they fail for different reasons and a single verdict leaves four things to check.

It names the mismatch that actually bites: write-back on, no write privilege — every change saved locally and every one refused. It also reports how many people are missing source_ref, which is exactly what a pre-write-back import leaves behind.

🔴 Writes nothing. The permission question is the current-user-privilege-set PROPFIND, not a create-and-delete probe — that can fail halfway and leave a contact called "FreeITSM test" in a real address book.

D016 — CardDAV drift

The top-to-bottom compare: every imported person against their card, field by field, both values shown. Plus cards that have vanished, and contacts on the server that never came here.

⭐ It answers the question the reporter never did. The #133 reply asked which details actually drift and in which direction, because that is the evidence that would shape two-way sync. No answer came. D016 measures it on the operator's own data and summarises which fields drift most.

🔴 No "fix it" button, deliberately. Which side is right is a judgement, and one button reconciling hundreds of contacts is the data-loss incident this whole design avoids. ⚠️ It also names the trap it reveals: a drifted value is precisely what makes a later write-back get refused, so import first and edit second.

15. Testing it

docker/carddav-test/ — Baïkal 0.12.1 on port 8092. See its README for the seed script and credentials.

What the live run against it proved, and what to re-prove after any change here:

  1. cardDavCanWrite() returns real privileges.
  2. A PUT returns 204 and a new ETag, which is then stored.
  3. Reading the card back shows exactly one logical line changed out of the card's total — this is the assertion that matters most, and it is the one a regenerating implementation fails.
  4. A deliberately stale ETag returns 412 and is reported as a conflict.
  5. A same-field change made behind FreeITSM's back is caught by the three-way merge, with the other person's value in the message.
  6. With write-back off, nothing is attempted at all.
  7. employee_id sent through the push does not appear on the card.

⚠️ Restore the card afterwards. The rig is scratch, but a test that leaves the fixture changed makes the next run's "before" wrong.

16. A customer's own change, from the portal

api/self-service/update_profile.php uses the same cardDavPushPersonChanges(), with $fromPortal = true, when System → Portal profile allows address-book contacts to change their details (and the book has write-back on). Two differences, both deliberate:

  • 🔴 It pushes FIRST and saves only on success. §11 says a failed write-back never loses the analyst's work, because the analyst is told and can act. A customer cannot, and a local value the book refused is exactly what the next import reverts - the vanishing edit this whole design exists to prevent. So a refusal (attempted false, or ok false) returns an error and nothing is saved on either side.
  • The log says who. A written row ends "Changed by the person themselves, in the self-service portal", and a conflict row says "the person set, in the self-service portal," instead of "the analyst set". triggered_by_analyst_id is NULL.

Only the fields whose value actually changed are passed, so a save that only touched the preferred name sends nothing and logs nothing. Details and tests: Contact details — Developer Guide §3.

17. Adding a new contact

includes/carddav_create.php - Add to address book on an unlinked person (api/tickets/address_book_add.php, the button from assets/js/address-book-add.js on both people screens). Only for providers with carddav_write_back = 1 and carddav_allow_create = 1; the provider save forces the second to 0 whenever the first ends up 0, including when write-back was kept from the stored row.

🔑 This file generates a vCard, and §2 still stands. The rule is about EXISTING cards, where rendering one from FreeITSM's fields destroys what FreeITSM does not model. A new card has nothing to destroy. The one existing card this code touches - the group card - is still edited byte-wise: one MEMBER:urn:uuid:<uid> line inserted before END:VCARD, If-Match, one re-read on 412. It is a separate file so the import path, which must stay provably read-only, never loads it.

cardDavCreateTargets(PDO $conn, array $user): array   // books this person could go to, and why not
cardDavCreatePreview(PDO $conn, array $user, array $provider): array
cardDavBuildCard(string $uid, array $preview, ?string $organisation): string   // vCard 3.0
cardDavCreateContact(PDO $conn, int $userId, int $providerId, ?int $analystId): array

The order, and why:

  1. Read the book. Refuse (duplicate, logged skipped) if any non-group card has the person's email address - the import would adopt that card instead. For a group scope, find the group card with the import's either-or match (UID or FN); refuse (out_of_scope) if there is none.
  2. Create <book>/<uuid>.vcf with If-None-Match: *. UID is a v4 UUID. ORG:<company>;<department> and ADR;TYPE=WORK:;;;<office>;;; are written in the components cdsyncOrg() / cdsyncLocality() read. A tag-scoped source gets CATEGORIES:<first chosen tag>.
  3. Group scope: add the MEMBER line to the group card. On failure, DELETE the new card.
  4. 🔴 Ask the import. Read the book again and run the import's own cdsyncResolveScope() → cdsyncMapCard() → cdsyncInScope() on the new card. Not a copy of the rule - the rule. If it would not be imported, remove the MEMBER line again, DELETE the card, link nobody, return out_of_scope. Without this, a card outside the source's scope is invisible to the next import, which counts the person as missing and, after sync_deactivate_after runs, marks them as left.
  5. Link the existing record - is_managed = 1, auth_provider_id, sync_missed_count = 0 - and dsyncLinkIdentity() with the new UID as subject and the card's href and ETag as read back in step 4. The next import finds the person by UID (dsyncFindExisting()'s first rung), so it neither creates nor adopts.
  6. Log created (pill Added) with the fields written.

Also refused: a person already linked to a provider that still exists, a person with no email address, and a provider owned by a different company from the person's.

⚠️ Narrowing a source later (all → one tag or group) leaves earlier-added cards outside it, like any other card without the tag. That is the administrator's choice, and the help says so.

⚠️ A tag containing a backslash can never match in cdsyncResolveScope(), which unescapes only \,. The rollback test uses exactly that to force step 4 to fail; the import side is not fixed.

⚠️ Before Database Verification an upgraded install has no carddav_allow_create. The provider list must not select it (it once did, and System → Authentication showed No providers yet), and the save drops it - including from the keep-stored step - until it exists.

Tests (Docker clean room against Baïkal, 51 checks): the provider switch rules; what is offered and to whom; an add under "everyone", with an import afterwards that creates and adopts nobody; duplicate email refused with nothing written; tag scope with a control that an untagged card really is missed by a tag-only import (brake off, sync_missed_count preset to 2 so a real sighting must reset it); group scope, existing members untouched; an unfindable group refused before writing; a real post-write rollback; switched off, including a direct request.

18. What to read next

FreeITSM — an open-source IT Service Management platform · github.com/edmozley/freeitsm · MIT licence

FreeITSM

Getting Started

Modules

Multi-tenancy (planned)

Blue sky thinking

Bugs resolved

Links

Clone this wiki locally