Knowledge Assistant Developer Guide

Knowledge Assistant — Developer Guide

How the assistant finds knowledge gaps and drafts articles. For what it does and how to use it, see Knowledge Assistant.

---

1. The shape of it

Two halves that answer different questions, sharing one engine:

Half	Question	Where it lives
Gap finding	What is missing?	includes/knowledge/gap_analysis.php
Write-up	Is there enough here to write it?	includes/knowledge/writeup_ai.php

And two entry points into the same write-up engine:

• Tickets owns the moment — an analyst has just solved something and it is fresh. Button in the reading pane.
• Knowledge owns the judgement — the clusters, the thresholds, the provider and key, the drafts.

That split is why the endpoint lives under api/knowledge/ even though one of its callers is the ticket inbox: writing an article is a Knowledge action wherever the button happens to be rendered, and requireModuleAccessJson('knowledge') enforces it.

1a. Every file involved

🗄️ schema · 📖 read · ✏️ write · 🧠 engine · 🖥️ UI · 🧪 test

🎨	File	What you do there
🧠	includes/knowledge/writeup_ai.php	Prompts, the verdict protocol, ticket text bundling, richness, stemming, similarity, clustering. The interesting decisions are all here.
🧠	includes/knowledge/gap_analysis.php	gapAnalyse() — score every closed ticket against the KB, cluster what nothing covers, persist. Runnable without HTTP so it can be tested.
🧠	includes/knowledge/kb_ai.php	Pre-existing. kbCosineSimilarity() and kbGenerateEmbedding() are reused, not reimplemented.
📖✏️	api/knowledge/analyse_gaps.php	Thin adapter. status / embed / cluster.
📖	api/knowledge/gap_clusters.php	The findings + their evidence tickets.
✏️	api/knowledge/gap_dismiss.php	Dismiss / bring back.
✏️	api/knowledge/writeup_stream.php	SSE. Judge → refuse+questions, or article. Both entry points hit this.
✏️	api/knowledge/writeup_save.php	Creates the unpublished draft via KnowledgeService.
📖✏️	api/knowledge/assistant_settings.php	The four tunables. Clamped server-side.
🖥️	knowledge/assistant/index.php	The assistant page.
🖥️	knowledge/includes/header.php	The Assistant nav item.
🖥️	knowledge/settings/index.php	Assistant section in the AI tab.
🖥️	assets/js/inbox.js	buildWriteUpButton() + the wu* client, appended at the end of the file. ⚠️ Bump inbox.js?v= in tickets/index.php when you touch it.
🗄️	database/freeitsm.sql · includes/db_verify_schema.php	knowledge_gap_tickets, knowledge_gap_clusters, knowledge_gap_cluster_tickets.
🧪	tests/knowledge-gaps/run.php	The clustering harness. Read this before changing any similarity code.

What you do NOT touch: the article renderer, the portal reader, or the AI-answer retrieval. A draft is just an article with is_published = 0, and every customer-facing reader already filtered on that.

---

2. The verdict protocol

One model call does both jobs — judge, then write. Two calls would let the second overrule the first, and a single stream is a far better experience than a spinner, a verdict, and another spinner.

The model's first line is machine-readable:

VERDICT: ARTICLE
VERDICT: NOT_ENOUGH

⚠️ api/knowledge/writeup_stream.php buffers until it has that line. Nothing reaches the client until the verdict is parsed and stripped, otherwise the words VERDICT: ARTICLE appear at the top of the analyst's draft. Once resolved, deltas pass straight through.

Two guards worth knowing about:

• If the buffer passes ~200 characters with no newline, it stops waiting rather than streaming nothing for ever.
• After the stream ends, an unresolved verdict is settled from the complete text.

writeupParseResponse() is tolerant of a missing colon, a markdown fence and leading blank lines — but it never guesses. An unrecognised response is treated as not_enough, because wrongly publishing is a worse failure than wrongly asking.

Prompt modes

writeupSystemPrompt($mode) — single, cluster, answers. All three share one body plus writeupProtocolBlock(), so the response contract cannot drift between them.

answers mode carries the line that matters: the analyst's answers are authoritative even where they contradict the thread.

---

3. Richness: what it is and is not

writeupRichness() returns 0–100 from resolution length, notes length, conversation turns, time logged, and whether a Problem record exists.

⚠️ It is a heuristic for ORDERING, not the judgement. Only the model decides whether an article exists, having read the words. A long ticket can be pure back-and-forth about scheduling; a short one can contain the exact registry key.

It is used for exactly two things:

1. Sorting candidates so a cluster forms around — and is drafted from — its most writable ticket.
2. Deciding whether it is worth spending a model call at all.

Two implementations on purpose:

• gapBulkRichness() (in gap_analysis.php) — four aggregate queries over all candidates. Lengths are of raw HTML bodies so they are inflated, which is fine because the score is only ever compared against other tickets.
• writeupTicketBundle() — the real plain-text bundle, read once, for the single ticket actually being written from.

---

4. Two similarity engines

Finding gaps must work on an install that will never pay an embeddings bill. So there are two engines behind one interface, and the UI says which is running.

	Meaning mode	Wording mode
Needs	knowledge_openai_api_key	nothing
Ticket ↔ article	cosine over text-embedding-3-small	token overlap
Bar (article / cluster)	0.75 / 0.82, from settings	0.55 / 0.50, constants

⚠️ The wording bars are their own constants, not the vector settings scaled by a factor. Cosine over 1536 dimensions of meaning and token overlap on a subject line produce numbers that mean different things; pretending one converts to the other by multiplication gives you a setting nobody can reason about.

⚠️ Ticket and article embeddings must come from the same model. knowledgeOpenAiKey() deliberately reuses the key api/knowledge/generate_embedding.php uses. Change one without the other and every similarity score is silently meaningless — no error, just wrong answers.

4a. Overlap coefficient, not Jaccard

writeupTokenSimilarity() returns |A ∩ B| / min(|A|, |B|), and 0 if fewer than two tokens are shared.

The harness is why. Jaccard divides by the union, so it punishes a subject for words the other one happens not to have:

Pair	Jaccard	Overlap
"VPN keeps disconnecting every few minutes" ↔ "VPN disconnecting again"	0.29	0.67

Real subject lines vary wildly in length and filler, so the union is the wrong denominator for the question being asked — which is "is the shorter of these two contained in the longer?".

The two-token floor is what makes that safe. The overlap coefficient alone scores any pair sharing one word out of a two-word subject at 0.5, so "Broken window in the server room" and "Request access to the CAD licence server" would cluster on the word server. Requiring two shared tokens costs nothing real — no genuine recurring question is identified by a single word.

4b. Stemming

writeupStem() is a crude suffix stripper (-ly, -ies, -sses, -ing, -ed, -s, plus doubled-consonant collapse, never collapsing l/s/z).

Not cosmetic. Four real VPN tickets failed to cluster at all purely because "disconnects" and "disconnecting" are different strings — which is the single most common way a desk phrases the same question twice.

Numbers are stripped before tokenising, deliberately: "Laptop LT0431 won't boot" and "Laptop LT0899 won't boot" are the same question, and leaving the asset tags in hides the biggest recurring gaps behind a wall of unique-looking subjects.

4c. Single-linkage clustering

writeupCluster() grows a cluster outwards from a seed, pulling in anything close to any member, not just the seed.

⚠️ The transitive step is the whole point. Seed-only comparison makes the result depend on which ticket happened to be first:

vpn0 ↔ vpn3   0.67
vpn1 ↔ vpn3   0.67
vpn0 ↔ vpn1   0.40      <- one mentions minutes, the other mentions home

Seeded on vpn0, a seed-only pass produces {vpn0, vpn3} — two tickets, below the minimum, never reported. The assistant stays silent about the thing it most needed to say. Chaining outwards gives {vpn0, vpn3, vpn1}, which is the truth.

Single linkage can over-chain in principle. Here the pairing bar is high, wording mode needs two matching tokens before returning any score at all, and the harness carries six unrelated one-off tickets that must never end up in a cluster.

O(n²) and unapologetic: n is gap candidates in a 90-day window.

---

5. Cluster identity survives re-analysis

🔑 A freshly computed cluster is matched to a stored one by TICKET OVERLAP (≥ 50%), never by id, seed or label.

A cluster grows as new tickets close, and its seed changes as richer examples arrive. Any identity built on "same seed" or "same subject" breaks on the next run and re-raises something the analyst has already dismissed — which would make the dismiss button useless within a week.

The rest of gapPersistClusters() follows from that:

• computed fields are updated in place; status and article_id are then restored, so an update can never quietly reopen a decision
• an open cluster this run no longer sees is deleted — usually because an article now covers it, so leaving the card would be a lie
• dismissed and written clusters are kept regardless: those are the analyst's decisions, not our findings

---

6. Where the safety actually is

Concern	Where it is enforced
Nothing unreviewed reaches a customer	is_published = 0 on the draft. KB_VISIBLE_SQL (kb_ai.php) and includes/knowledge/portal_reader.php both already require is_published = 1.
Drafts are still findable by analysts	api/knowledge/knowledge_articles.php — the only reader that lists unpublished articles.
No workflow announces a draft	KnowledgeService::saveArticle skips knowledge.published when is_published is false.
Model output never hits raw innerHTML	safeHtmlFragment() on both clients — the one shared sanitiser, same rule as every message body.
Ticket scope	analystCanAccessTicket() in writeup_stream.php and writeup_save.php. A ticket reached via a cluster is still checked.
Cluster scope	activeTenantFilter(..., 'c') on the read and on the dismiss write — a cluster id is one guess away.
Spending money	Cap::KNOWLEDGE_EMBEDDINGS on analyse_gaps.php, the same capability that guards re-embedding articles.
Settings sanity	assistant_settings.php clamps every value. The number inputs' min/max are a convenience, not the guard.

⚠️ Clusters are derived from tickets, so they follow ticket tenancy (activeTenantFilter). Do not reach for knowledgeTenantFilter here — in Knowledge, NULL means shared with every company, which is exactly backwards for a ticket-derived row.

---

7. Degrading before Database Verify

writeupSchemaReady() is a cached SHOW TABLES check, and it is load-bearing.

An install that pulls this update but has not run Database Verification must degrade to "the assistant has nothing to show you" — not an Unknown column fatal that takes out the Knowledge module. Same gate, same reason, as snoozeSchemaReady() in the ticket inbox.

Remember a PHP fatal is served as HTTP 200. Check the body, not the status code.

---

8. Testing

php tests/knowledge-gaps/run.php

16 checks, everything inside one transaction that is always rolled back, in wording mode so it spends nothing. The fixture is five recurring questions phrased the way real people phrase them, plus six genuine one-offs.

The discipline worth copying:

• every negative assertion is paired with a positive control. "The one-offs did not cluster" is equally true of a harness that clustered nothing at all, so it is only meaningful next to "the five themes did cluster".
• "the dismissed cluster is still dismissed" is paired with "the ones we did not dismiss are still open".
• the final section asserts the rollback actually rolled back. Without it the harness is a liability, not a test — it writes to a live database.

Both algorithm decisions in §4 came out of this harness failing. Run it before and after touching anything in writeup_ai.php.

Testing the model side

Forge a session and curl the SSE endpoint directly — it streams plain text and is easy to read:

curl -s -b "PHPSESSID=<id>" -H 'Content-Type: application/json' \
  -d '{"ticket_id":209}' http://localhost/freeitsm-app/api/knowledge/writeup_stream.php

⚠️ Test a thin ticket AND a rich one. "It refused" proves nothing on its own — an endpoint that always refuses looks identical. The positive control is feeding answers back via {"ticket_id":N,"answers":"..."} and getting VERDICT: ARTICLE.

---

See also

• Knowledge Assistant — the user-facing page
• Knowledge Base · AI Providers
• Database Verification Developer Guide
• Multi-Tenancy Developer Guide