Help Assistant Questions Answered

❓ Help assistant: every design question, answered

Part of the internals series: 1 - the index · 2 - the hybrid search · 3 - answers and misses

The help assistant was designed in one long conversation on 2026-11-11, and most of what makes it good came from pushing back with questions. They are kept here, in the order they were asked, with the answers as built - because anyone changing it will ask the same things.

---

The idea

"How feasible would it be to make an AI help assistant that answers from the in-app help? The problem is the i18n work."

Very. The help is already a good, structured, permission-aware source, and the hard part - other languages - turns out to be the part modern models handle best. The help pages are already translated into German and Spanish through the language files; the index stores each section in every language it exists in, and falls back to English per section (part 1). Questions in languages FreeITSM has no files for are handled by meaning (the vectors are multilingual) and by the AI searching in English itself (part 3).

"Accessible across all modules - click an icon, a draggable chat window appears and persists unless you click Reset."

Built exactly that way. A button in the header of every analyst page; a draggable window (a bottom sheet on a phone); the conversation saved on the server per analyst, so it follows you between pages and devices, and goes only when you click Reset.

"Option 2 implies you would have to ask the question in English - is that right?"

With word search alone, partly: a question has to share words with the help, and the help exists in English, German and Spanish. That is why the design added two things that cross languages - vectors, which match meaning whatever the language, and the AI's English search. Measured on six other-language questions: words alone 2, meaning alone 4, the written answer 3 plus one honest "not covered".

---

How it searches

"Does every question have to read every help file in full?"

No. The help is read once, ahead of time, into an index (help_sections, 722 sections in English) - part 1. A question is scored against that index in about 80 ms. If a chat AI writes the answer, it is given only the best four sections, each at most 3,000 characters - not 74 pages.

"I thought vector mappings were the magic that makes AI chat so powerful?"

They are very good at one thing - meaning - and that is why they are in. "My boss wants a weekly PDF of how we are doing" shares no words with the Report packs help, and only meaning finds it. But they are weaker at exact names: "Warbot" or "MoSCoW" are rare words that word search nails. So it uses both and merges them (part 2).

"This sounds too good to be true - is there a performance hit?"

Small, and measured:

	Time
Word search	~80 ms
+ meaning (embed the question, compare)	~0.4 s
+ a written answer (Claude Haiku)	~2.7 s

The cost of a written answer is about a third of a penny a question.

"Walk me through an example - 'how do I create a ticketing department'."

That is the running example of part 2, with the real numbers: the words become creat, ticket, department; word search ranks Orphaned tickets first (the right words, the wrong meaning) and Teams → Departments third; meaning ranks it second; the merge puts Departments first and Tickets → Settings second - a section that was first in neither list, because both methods agreed on it.

"So you are honestly saying there is no trade-off for not using vectors?"

No - there is one, and the design said so too softly at first. Without vectors, vague questions (sharing no words with the answer) were found 2 times in 6; with them, 6 in 6. Plain questions went from 43 to 46 of 47. Words alone are good, free and private; vectors are clearly better.

"Could we go for a hybrid - vectors for the English help only, then hand over to the other method?"

That is what was built. Vectors are made for the English sections only (722, shipped ready-made), word search covers every language, and the two rankings are merged with reciprocal rank fusion, words at half weight. A German section found by meaning through its English twin is read in German, so menu names come out as the asker sees them.

"Would you get the benefit of vectors even if you asked in Hungarian?"

Yes. The embedding model was trained on many languages, so "Hogyan hozzak létre jegykezelő részleget?" lands near the English Departments section. Word search contributes nothing there - there are no shared words - so meaning carries it, and the AI's own English search covers what meaning misses.

"Is our hybrid approach novel, or do other people do it?"

The core isn't new: word search plus vectors, merged by reciprocal rank fusion (Cormack, Clarke and Büttcher, 2009), is the standard "hybrid search" in search engines and retrieval systems. What is particular to FreeITSM is the combination around it: vectors only for English with a per-section language fallback, ready-made vectors shipped with each release, a four-level ladder that works with no AI at all, permission-aware answers, and canned replies whose confidence is set from measured scores.

"As it's not novel, why didn't you suggest it earlier?"

A fair question. The first answer leaned towards the simplest option and oversold how far words alone would go; the hybrid only came up when the trade-off was pushed on. The lesson is recorded: say the trade-offs up front, even for the option you like.

---

Working with and without AI

"Can we design this so it works both with and without an AI key?"

Yes - four levels (part 3): Search help (nothing needed), + matching by meaning (an embedding key), + written answers (a chat key), or both. Each falls back to the level below if its AI fails.

"If you've got no AI key configured, will the help chatbot work AT ALL?"

Yes. It searches the help by words and replies with the best sections as cards. That is the default level and needs nothing.

"Can the key screen make this abundantly clear? Will it feel like chat, or like search results? Could we have canned phrases?"

The System page opens with a panel - "The help assistant works without AI" - before any key field, and a live line saying what level it is at right now. The window is chat-shaped at every level: without AI it replies with a short line chosen by how good the match is (clear winner / several / from your module / weak / nothing), several variants of each picked at random, all in the language files - and none of them pretends an AI read anything. The thresholds came from the question set's real scores (part 2).

"So vectors still work without actual AI?"

Without a chat AI, yes. Matching by meaning needs only an embedding model - a different, much cheaper kind of model that turns text into numbers and writes nothing. The help's vectors ship ready-made, but each question still has to be turned into numbers when it is asked, so it needs an embedding key. With no key at all, it is words only.

"Does the question have to go to OpenAI or any provider? ...Wait - the vectors and the question must come from the same provider?"

Exactly right - the same model, in fact. Two vectors can only be compared if the same model made them. So the question is embedded with whichever model made the stored vectors; FreeITSM stores each vector with the model's identity (aiEmbeddingModelId()) and never compares across models. The chat AI is separate and can be any provider.

"How come Haiku answers are compatible with matching by meaning in OpenAI?"

Because the two never meet. OpenAI's embedding model only chooses which sections are relevant; Haiku only reads the text of those sections and writes the answer. Haiku never sees a vector. Two jobs, two keys, any mix of providers.

"How do I get the best experience?"

Both levels on: an embedding model for matching by meaning (OpenAI text-embedding-3-small at 512 dimensions has ready-made vectors shipped, so no rebuild is needed) and a small, fast chat model for written answers (Claude Haiku is the measured default). Give analysts access, ask real questions, and watch the unanswered list - it shows where the help itself needs work.

---

Storage

"Should it go into the knowledge table with a new is_help column?"

No. 101 queries across 60 files read knowledge_articles; every one would need a filter so help never turned up as a knowledge article, and one missed would leak. A separate table, help_sections, is safer and is built only for this (part 1).

"So we are re-using the code, not duplicating?"

Yes. The embedding call existed in four copies and the cosine comparison in two; step 1 moved them into includes/ai_embeddings.php, used by Knowledge and the help assistant alike. A test (tests/ai-embeddings.php) fails if a fifth copy appears. Doing it found a real bug: long non-Latin articles were cut mid-character, so they never got an embedding.

"Can we add a language column?"

Built in: help_sections.locale. Each section is stored once per language it exists in, and an analyst gets their language where it exists, English for the rest, section by section.

---

Shipping and rebuilding

"Can we ship the vector mappings in each release, or is that up to whoever manages FreeITSM? Will System → Help AI handle the build?"

Both. The English vectors for the default model ship in each release (data/help-vectors/, about 1 MB) and are imported automatically. System → Help assistant shows their state and has a Rebuild button that runs in the background - for a different model, or after help has changed.

"Is there any way the System page can warn you if your vectors are out of date?"

Yes. Each section stores a fingerprint of its text. Help file dates are the cheap first check; the fingerprints are the real one - a vector made from a different text is out of date. The page says how many sections have changed and offers Rebuild.

"Should we allow someone to choose a different model, then warn them and offer a button to regenerate the vectors?"

Built. Choosing a different model shows a warning and a Rebuild button. Until the rebuild finishes, the old model keeps answering - its settings, key encrypted, are kept separately (help_vectors_active_cfg) - so the assistant never goes dark during a switch (part 2).

---

Who gets it, and privacy

"Only when it's turned on, and per analyst."

Off until an admin turns it on, then ticked per analyst on System → Help assistant.

"Unanswered questions visible to an admin, a bell notification - but never to the person who asked."

Built (part 3). The bell goes to admins only, coalesced into one entry, with the asker as the notification's actor - and nobody is notified about their own action. A question is not recorded when the help exists but only in a module the asker cannot open.

"Recording the asker's name - an admin setting, per analyst."

Record who asked, ticked per analyst. Off by default ("an analyst"); when on, that analyst's window tells them.

"Any model - Azure, OpenAI, Anthropic, OpenRouter - and its own key, for granular billing."

Both AI parts use the shared AI provider panel, so any provider FreeITSM supports, including a model server on your own network. Each has its own key, separate from every other AI feature, so its cost shows on its own bill.

"The key must be encrypted."

Both keys are saved through aiSettingsSave(), encrypted with the install's key like every other AI key, and shown back only as **** and the last four characters. The copy kept for the model in use during a switch is encrypted too.

"Can the chat respect role and module permissions, or is that a step too far?"

Not too far - built in two layers (part 3): help for modules you cannot open is never searched, by the search or the AI; and sections about settings you cannot change are answered with who can change them.

The portal, and System help in other languages

No portal for now. The 35 System help pages stay English until they move into the language files (maybe other languages for the vectors too, later).

---

The help pages

"Do we have (or should we have) special markup that tells the code what a section is?"

It already existed: <div class="help-section" id="..."> with an <h3>. Two optional attributes were added - data-cap (the permission the section needs) and data-help-search="off" (leave it out). It is in the house style.

"Are all the help pages in line, so they all have sections in the markup?"

Audited: all 74 pages have sections and every section has an id. The only text outside sections was each page's intro, which is now indexed as an entry of its own.

---

After it shipped

"Can we tweak the prompt so it signposts better - e.g. Tickets → Settings → Departments or System → Database Verify?"

Yes, and it was a small change (#2778). The help already writes paths, but from inside a module ("Settings → Renewals" in the Software help) - so each section given to the AI is now labelled with its module's menu name, and a rule asks for the way there as a bold path built only from the sections' own words, in their language. Real answers afterwards: Tickets → Settings → Departments, System → Database verification, Software → Settings → Renewals, and in German Tickets → Einstellungen → Abteilungen. When the help does not say where something is, it still does not make a path up.

---

Things found on the way

"Clicking a result made the main window jump, chopping off the top nav bar."

The browser's own jump to an anchor scrolls the whole page. The window now puts the page back and scrolls only the panel holding the section, so the top bar stays put (part 3).

"Web exposure: all jobs failed" - and "something about Node.js being deprecated"

Two separate things. The failure was real: a test started its own small fake AI server, and the Web exposure check (which lists every PHP file reachable on the web) flagged it. The test now uses the existing stand-in server tests/ai-compatible/mock.php, which is on the allowed list. The Node.js message is GitHub's notice that actions/checkout@v4 runs on Node 20 - a warning, not a failure, and harmless until GitHub moves on.

---

Back to: Help assistant developer guide · Design · User page