Skip to content

Help Assistant Design

Jump to bottom
Ed Mozley edited this page Nov 11, 2026 · 11 revisions

πŸ“ The help assistant - design

Status: built 2026-11-11, all six steps, coming in 5.0.0. This page is the design as agreed; for how it was built see the developer guide, the internals series (1 Β· 2 Β· 3) and every design question, answered. An assistant on every page that answers "how do I...?" from FreeITSM's own in-app help, in the user's language. It works on every install - with an AI provider it writes an answer, without one it still finds the right help section. It is for analysts and admins learning the system. The main decisions are made - see Decisions.

Why

FreeITSM's in-app help is large: every module has a help page, plus 35 System help pages - about 186,000 tokens of English text, much of it translated into 24 other languages. It is all there, but finding the paragraph you need means knowing which module's help to open and scrolling to the right section.

People learning the system do not think in modules. They think "how do I create a ticketing department?" or "why did that reply open a new ticket?". The assistant takes the question as asked, finds the help that answers it, wherever it lives, and either answers it or takes the reader straight to it.

What the user sees

  • An icon on every page, in the shared header, in every module - for analysts who have the assistant turned on (see Turning it on).
  • Clicking it opens a chat window that can be dragged anywhere on the screen. Where it sits and whether it is open are remembered.
  • The conversation is kept across pages. Every module is a separate page load, so the conversation is stored on the server for that analyst and reloaded on each page. It is cleared only when the user clicks Reset.
  • Every answer links to the help it came from - πŸ“– Tickets help β†’ Settings - so it can always be checked, and so the reader learns where things are.
  • It knows where the user is. In Workflows, "how do I add a step?" means a workflow step. Elsewhere it may answer "that depends - in Workflows..., in Starters & leavers...".
  • It answers in the language the question was asked in, and writes menu and button names exactly as they appear on that user's screen.

Four levels - it works with or without AI

Two separate keys are involved, and the assistant uses whatever the install has:

  • a chat AI - writes the answer (any provider FreeITSM supports: Anthropic, OpenAI, Azure, OpenRouter, a local model)
  • an embedding key - lets the search match by meaning (OpenAI today)
The install has The window How it searches
No AI at all Search help - the 5 best help sections, each with a short extract and a link straight to it. No written answer, but instant and free Word search, in the user's language and English
An embedding key only Search help, better at loosely worded questions Word search + vectors
A chat AI only Ask - a written answer, with links; searches again itself when the first results miss Word search + the AI's own searches
Both Ask, with the full hybrid Word search + vectors + the AI's own searches + translating the question when needed

Same icon, same window, same saved conversation at every level - only the label (Search help / Ask) and what comes back differ. An admin sees which level is active and why ("Ask is off - no AI provider is set up") with a link to fix it.

The no-AI level is worth having on its own: a question in plain words lands the reader on the exact paragraph, instead of a long page to scroll. Measured on the question set: a right section in the top 5 for 91% of plain questions and 100% of exact names, with no AI at all.

Say so on the settings screen

Most admins will assume an "AI assistant" does nothing without a key. System β†’ Help assistant opens with a panel that says the opposite, before the key fields:

βœ… The help assistant works without any AI. With nothing set up it searches every help page and takes your analysts straight to the right section - free, instant, and nothing leaves your server. An embedding key makes it better at loosely worded questions. A chat AI makes it write the answer in a sentence or two, in the asker's own language.

with a live line under it: "Right now: Search help (no AI set up)."

It still feels like a chat

At every level the window is a conversation: the question as a bubble on the right, the reply on the left, kept until Reset. Without AI, the reply is a short line followed by the results as cards - section title, a line of preview, Open.

The line is one of a few canned phrases, chosen by how good the match is - so it never sounds sure about a weak result - and picked at random so it does not sound robotic:

Situation For example
One clear winner - the top result well ahead of the rest "This looks like exactly what you need:" Β· "Good news - the help covers this:" Β· "I think this is the one:"
Several decent matches "Here's what I found in the help:" Β· "These look like the closest matches:" Β· "You might find these helpful:" Β· "I had a look through the help and found these:"
Weak matches only "I couldn't find an exact match, but these might be close:" Β· "Not a perfect match, but worth a look:"
Nothing "I couldn't find anything for that - try different words, or name the module you're in?" Β· "Nothing in the help matches that. Maybe try a shorter question?"
The top result is from the module the asker is in "Here's what the Workflows help says:"

Two rules: the phrases never pretend an AI read anything ("I found these in the help" - not "I think the answer is..."), and they live in the language files like all other text. Where clear winner ends and weak begins is set from the question set's real scores, not guessed.

How a question is answered - a worked example

how do I create a ticketing department

Everything quoted below is the real help text.

0. Before anyone asks: the help, cut into sections

The help is stored as hundreds of short keyed strings - tickets.help.settings.card_dept_body and so on. Too small to search one at a time, so a builder reads each help page and groups the strings into the sections the reader sees (each <div class="help-section" id="..."> and the t('...') keys inside it). Each section gets:

  • an address - tickets/help.php#settings
  • a title - Settings
  • its text - every string in it, joined
  • its module - for the access filter

Reading the pages, rather than the language files alone, means a string that no page shows is never searched, and every result links somewhere real. The sections are cached in a table (below) and rebuilt only when a help page or language file changes.

1. The question goes in

The browser sends the question, the user's language and where they are (Tickets). With a chat AI, the model is given:

  • the rules - answer about FreeITSM only from the help it is given; if the help does not say, say so and never invent a button or menu; answer in the question's language; quote screen labels exactly as the user sees them
  • the question
  • the best sections already found (step 2)
  • one tool, search_help(words), to look again

2. The first search - word search and vectors together

Two searches run at once, both on the server, both in milliseconds:

Word search (the classic search-engine scoring, BM25 - no AI involved):

  1. The question is cut into words, common ones dropped (how, do, I, a), plurals trimmed: create, ticketing, department.
  2. Rare words count for more. ticket appears in most sections - it says almost nothing. department appears in about 30 - it narrows things right down.
  3. Every section is scored: each word's weight Γ— how often it appears, with a bonus for a match in a title, and a small bonus for the module the user is in.
Section Why it scores Score
Tickets β†’ Settings a card titled Departments: "Create departments like IT, HR, and Finance. Each department becomes a folder in the inbox and a filter on the dashboard" - both rare words, side by side, in a title high
Tickets β†’ Inbox "Departments are configured in Settings and appear as sub-folders automatically" good
Tickets β†’ Working with tickets "Route tickets to the correct team. Departments drive folder grouping and dashboard filters" good
Reporting β†’ Dashboards "narrow dashboard results to a specific department" - department but not create medium

Vectors, when there is an embedding key: the question becomes a vector and is compared with every section's vector, ranking them by meaning. This is what finds the right section when the question shares no words with it - "how do I stop the system spamming my requesters?" against a section about notifications.

The two lists are merged (reciprocal rank fusion - a section near the top of both comes first), and the top few go to the AI in full.

3. The answer

The AI reads only those few sections - a few thousand words, never the whole help - and writes:

Go to Tickets β†’ Settings β†’ Departments and add one (for example IT, HR or Finance). Each department becomes a folder in your ticket inbox and a filter on the dashboard, and you can make one the default for tickets that arrive by email. πŸ“– Tickets help β†’ Settings Β· Tickets help β†’ Inbox

Most questions end here: one AI call.

4. When the first search was not enough

If the sections do not answer the question, the AI calls search_help with words of its own - search_help("notification email requester") - and the same hybrid search runs again. Each extra search is one more round trip (1-3 seconds). This is the safety net for questions both searches missed.

What the example also shows

The help has no step-by-step guide for creating a department - only that one card. The assistant gives what the help says and stops; it does not invent "click the green + button". That rule is what keeps it trustworthy, and questions it struggled with are the best possible list of help that needs writing (see Open questions).

Languages

A question in a language FreeITSM has help for

The search runs against that language's help as well as English (word search), and the answer is read in the user's own language - see the key trick below. A German user asking in German gets a German answer quoting German screen labels.

A question in a language FreeITSM has no files for

Hogyan hozzak lΓ©tre jegykezelΕ‘ rΓ©szleget?

  • Vectors still work across languages. The embedding model places that question near the English "Create departments like IT, HR..." - less sharply than an English question, and less well for Hungarian than for German or Spanish, which have far more training text behind them.
  • Word search cannot - there are no Hungarian words in the help.
  • So the question is translated first, only when needed. If it is in neither English nor the user's interface language, a quick call to a small, fast model turns it into an English search phrase ("create a department in Tickets"), which both searches use. About 0.5-1 second, only for these questions.
  • The answer is in Hungarian; the things to click are in English, because that is what is on the screen:

Nyissa meg a Tickets β†’ Settings β†’ Departments menΓΌt (Jegyek β†’ BeΓ‘llΓ­tΓ‘sok β†’ RΓ©szlegek), Γ©s adjon hozzΓ‘ egy rΓ©szleget...

Asking Screens in Answer in Menu names in
English user English English English
German user German German German
Hungarian user English Hungarian English (Hungarian in brackets)
German user, screens switched to English English German English

The key trick: find in English, read in your own language

Every language file uses the same keys - tickets.help.settings.card_dept_body is the same card in all of them. So the search can find a section using English vectors, then read that section from the user's own language file by its keys. A German user's AI receives the German text, with the German menu names exactly as on their screen - nothing is translated from English in the model's head, so a menu name cannot come out wrong. An untranslated string falls back to English, as the help pages themselves do.

The trade-offs, honestly

Vectors, word search and the AI searching for itself each fail differently. That is why the design uses all three.

Good at Weak at
Word search Exact names - buttons, settings, error messages (HY093, Enforce approvals). Instant, free, and you can see why it matched A question that shares no words with the answer; a language with no help files
Vectors Meaning - finds the right section on the first try when the wording differs; consistent whatever chat model is used Rare exact words (returns something related instead); needs an embedding key and an index
The AI searching again Recovering from a miss with better words Costs a round trip each time; a weak or small local model may accept a poor result - or, worse, answer from general knowledge of ITSM tools rather than from FreeITSM's help

Answer accuracy - representing the help faithfully - depends on the model sticking to the text it was given, which is the rules in step 1. Retrieval affects it only indirectly: the worse the search, the more a model is tempted to fill gaps. Better search is better accuracy.

None of this is new. Hybrid search (word scoring plus vectors, merged by reciprocal rank fusion) is now standard - Elasticsearch, OpenSearch, Azure AI Search and others ship it. An AI that searches with tools is how most assistants find things. Rewriting a question before searching and matching across languages are standard too. The one FreeITSM-specific touch is reading a section in the user's language by its keys, which falls out of how the translations are stored.

Speed

AI calls Typical wait
A question the first search answers (most) 1 2-4 seconds
A question in a language without help files 1 small + 1 3-5 seconds
A question the first search missed 2-3 4-9 seconds
No AI at all 0 instant

On the server it is negligible: the word search is milliseconds, a vector lookup about 0.2 seconds, loading the saved conversation one small read. Answers stream in as they are written, and the window shows Searching help... while it looks.

Cost

Building vectors for all the English help costs well under a penny with OpenAI's small embedding model; so does each further language. A question costs a few thousand tokens - a fraction of a penny on most models. A long conversation sends its earlier messages back each time, as every AI chat does, so the history sent is capped, and Reset clears it.

Turning it on

  • Off until an admin turns it on, on System β†’ Help assistant.
  • Then per analyst - an admin chooses who has it, so a team can start with a few people.
  • Without it, an analyst sees no icon, and the endpoints behind the window refuse them.

Respecting roles and permissions

Two layers, both applied on the server before anything reaches the AI:

  1. Modules the analyst cannot open are never searched. Someone without Asset Management never gets Asset Management answers, and nobody learns about modules hidden from them.

  2. A section that needs a permission says which. Help pages mark a section with the capability it describes - for example the Tickets settings section with the permission to change Tickets settings (a data-cap on the section, read by the section builder; capabilities are the Cap enum, checked with analystHasCapability() in includes/rbac.php). The analyst's own capabilities go to the AI with the sections, and the rules say: when the analyst lacks the permission a section needs, tell them who can do it rather than how:

    An administrator does this, in Tickets β†’ Settings β†’ Departments. You don't have access to that, so ask your admin.

    That is more useful than silence ("I don't know"), and nothing is given away - the help is documentation, already public on this wiki. Hiding is reserved for whole modules, where the reader cannot reach anything at all.

Sections with no mark are open to anyone who can open the module.

Storage

A cache of its own - not the Knowledge table

Putting help into knowledge_articles with an is_help flag was considered and rejected. 60 files hold 101 queries that read that table - the knowledge list, the self-service Help Centre, the REST API, the Knowledge AI chat, gap analysis, reports, review reminders, exports. Every one, and every future one, would need AND is_help = 0; one missed and help sections appear where they should not. And help is not an article: it has no author, owner, version, company or review date, it lives in the language files, and it changes on every upgrade.

So, a table of its own that is only a cache: the language files and help pages stay the truth, the table can be emptied and rebuilt at any time, and nothing else in FreeITSM reads it.

help_sections

Column Holds
section_key which section - e.g. tickets/help.php#settings
locale en, de, pt-BR... - the language folder codes
module for the access filter
title shown in results
body the section's text, for word search
text_hash a fingerprint of the text - only a changed section is rebuilt
embedding the vector (empty without an embedding key)
built_datetime when

One row per section per language, unique on (section_key, locale).

English vectors come first; other languages are optional. The locale column means vectors for German (or any language) can be added later without changing the table:

  • The fallback is per section, not per language. For a German user: each section's German vector if there is one, otherwise its English one. A half-translated German index never hides the untranslated half.
  • A German vector is only made from German text. Where a string is untranslated the German file shows English, and a "German" vector of that would just duplicate the English one - so the builder skips it.
  • A German question scores slightly higher against German text than English text saying the same thing, so translated sections tend to rank a little ahead - mostly harmless (it prefers what the reader can read), and the word search evens out real misses.

help_assistant_messages - the saved conversation: analyst, role (question / answer), text, the sections used, when. Visible only to that analyst; Reset deletes their rows.

help_assistant_unanswered - questions the assistant could not answer (see Unanswered questions): the question, its language, the module the asker was in, the best search score, when, whether an admin has dealt with it, and the asker - only where Record who asked is on for them, otherwise empty. Kept separately from the conversation, so Reset does not remove it.

The embedding code: one copy, shared - not a fifth

FreeITSM already calls OpenAI's embedding API from four places (includes/knowledge/kb_ai.php, api/knowledge/ai_chat.php, api/knowledge/generate_embedding.php, includes/services/knowledge.php) and has two copies of the vector comparison (kbCosineSimilarity(), cosineSimilarity()). kb_ai.php's own header notes that the copy in ai_chat.php should be switched over "when it's next touched". The general pieces also sit in a Knowledge file that loads Knowledge-only rules (audience, visibility, tenancy) with them.

So the first step (done - Embeddings - Developer Guide) is to move them into one small shared file - includes/ai_embeddings.php, with aiEmbed() and aiCosineSimilarity() - point all four existing callers at it, delete the copies, and prove with the Knowledge tests that nothing changed. The help assistant then uses the same file. A bonus: adding another embedding provider later (Azure OpenAI, a local model through Ollama) becomes a change in one place, which Knowledge, web chat and help all get at once.

Shipping the vectors with each release

The help text is identical on every install of a version, so its vectors are too. They can be built once, when the release is cut - a step in RELEASING, beside the notes - and shipped in the release (data/help-vectors/en.bin). On upgrade, or when Database Verify runs, the install loads them into help_sections: no cost, no wait, and every install has the same, tested index.

What it does not remove:

  • The question still needs an embedding key. A question only exists when someone asks it, so it is turned into a vector there and then - with the same model the shipped vectors were built with (text-embedding-3-small). An install with that key gets vector search immediately; one without uses word search, and the shipped vectors simply wait.
  • A different embedding provider means a local build. Vectors from different models cannot be compared, so an install using another provider (once there is one) builds its own.
  • A changed section is rebuilt locally. If an install has edited a language file, that section's fingerprint no longer matches the shipped one; it alone is rebuilt here (with a key) or searched by words.

Size: a full-size vector is about 6 KB, and there are several hundred sections - a few MB per language, too much to rewrite in git every release. The model can return shorter vectors (512 numbers instead of 1,536) for little loss, bringing English to roughly 1-2 MB. The question set (below) checks the shorter size before it is adopted.

System β†’ Help assistant

One page for admins. On most installs nobody needs a button on it: the word-search index builds itself and the English vectors arrive with the release. It is there to see what is going on, and for the unusual cases.

Part What it does
On / off, and who The master switch, and the analysts who have the assistant
Status Which level is active and why - "Ask is off - no AI provider set up" - with a link to fix it
AI provider The assistant's own provider, model and key - Anthropic, OpenAI, Azure OpenAI, OpenRouter, a local model - through the shared AI Providers panel, like every other AI feature. Its own key means its own bill
Embedding key The assistant's own, separate from the Knowledge module's - again so its cost can be seen on its own
Search index Per language: sections, how many have vectors, whether they came with the release or were built here, how many are out of date. Rebuild changed, Rebuild all, Build German... with progress, or left to the scheduler
Out-of-date warning See When the vectors are out of date
Try it Type a question, see the top 5 sections with their scores - for understanding why an answer was poor
Unanswered questions The list below

When the vectors are out of date

The page warns when help text has changed since its vectors were built.

Compared by the text, not by file dates. Dates give false alarms both ways: git pull, FTP or unzipping a release can put a new date on a file whose help did not change (a warning about nothing), and restoring a backup can put an old date on text that did change (no warning when it matters). Each section's row already holds text_hash, a fingerprint of the text its vector was built from; the page fingerprints the current text and compares. A different fingerprint is an out-of-date section - exact, and countable.

File dates are still used as the cheap first check: fingerprints are only recalculated when a help page or language file is newer than the last check, so the page stays quick.

⚠️ 12 English sections have changed since their vectors were built (Tickets 9, Workflows 3). Answers about those parts may miss the new text. [Rebuild changed]

  • With the assistant's embedding key, the scheduler rebuilds changed sections by itself; the warning stays only if that fails, with the reason ("the OpenAI key was refused").
  • Without a key, it says those sections are searched by words only until they are rebuilt.
  • Shipped vectors carry the version they were built for, so the index shows "from the 5.1.0 release" or "built here 2026-11-12", and an upgrade whose help changed is spotted at once.

Unanswered questions

When the assistant cannot answer from the help - the search finds nothing good enough, or the AI reports the help does not cover it - the question is recorded in help_assistant_unanswered.

  • Admins see the list on System β†’ Help assistant: the question, its language, the module, when. Each can be marked dealt with once the help is written.
  • A bell notification goes to the admins - "The help assistant could not answer a question", linking to the list.
  • Never to the person who asked. They have already been told the help does not cover it; a notification about their own question would only be noise.
  • The asker's name is recorded only where an admin chooses - a Record who asked setting, turned on per analyst. For anyone it is on for, their chat window says so in small print ("If the help can't answer a question, it's shared with your admins with your name"). Recording names silently would cost the trust that makes people willing to ask "basic" questions. Where it is off, the question is kept without a name.

This is the most useful output of the whole feature: a running list, in users' own words, of the help that is missing - like the department question in the worked example, where the help has one line and no steps.

Proving it works - the question set

Before the window is built, a set of about 50 realistic questions, each paired with the help section that should answer it:

  • plain ones (how do I create a department)
  • vague ones that share no words with the answer (stop the system spamming requesters)
  • exact names (what does Enforce approvals do)
  • other languages - German and Spanish (which have help files) and Hungarian and Polish-style questions (which may not)

Each search is run word search only, vectors only, and both, counting how often the right section is in the top 5. The numbers decide the details - whether the translation step earns its second, how much the module bonus should weigh - rather than opinion. The set then stays as a test, so a later change cannot quietly make the search worse.

Decisions

Made 2026-11-11:

  1. Who gets it - off until an admin turns it on, then per analyst. See Turning it on.
  2. Unanswered questions are kept and shown to admins on System β†’ Help assistant, with a bell notification to admins - never to the asker. See Unanswered questions.
  3. Its own AI provider and model, any FreeITSM supports - Azure, OpenAI, Anthropic, OpenRouter.
  4. Its own keys - chat and embedding - so its cost shows separately on the bill.
  5. The 35 System help pages stay English for now. Searching them works in any language; the only cost is that a non-English answer about a System screen relies on the AI to translate menu names itself, which it can occasionally get slightly wrong. It goes away when those pages move into the language files.
  6. No portal for now. This is for analysts and admins learning the system.
  7. Roles and permissions are respected - modules hidden, permission-marked sections answered with who can do it. See Respecting roles and permissions.
  8. The English vectors ship in each release - and other languages may follow. See Shipping the vectors.
  9. The page warns when vectors are out of date, by comparing text fingerprints. See When the vectors are out of date.
  10. Naming the asker is an admin setting, per analyst, and the analyst is told when it is on. See Unanswered questions.

Build order (each useful on its own)

  1. βœ… One shared embedding file - the four copies become one; Knowledge behaves exactly as before. Done 2026-11-11 (#2769) - see Embeddings - Developer Guide. It also fixed a bug all four copies shared (#2770).
  2. βœ… The section builder, the help_sections cache and word search - and the module-access filter, and the question set measuring it. Done 2026-11-11 (#2771) - see The help assistant - Developer Guide. Word search alone: plain questions 91%, exact names 100%, vague 33%, other languages 33% in the top 5. data-cap is read and stored; the marks go onto the pages with step 5.
  3. βœ… System β†’ Help assistant and the window, at the no-AI level - the switch and per-analyst access, the icon, dragging, the saved conversation, Search help. Done 2026-11-11 (#2773) - for users, under the hood.
  4. βœ… Vectors - English first, with the assistant's own embedding key; the question set shows the gain; then shipping them in the release. Done 2026-11-11 (#2775): vague questions 2/6 β†’ 6/6, plain 43 β†’ 46 of 47 - under the hood. Model changes warn and offer Rebuild, with the old model answering meanwhile.
  5. βœ… Ask - its own AI provider, the written answer, search_help, permission-aware answers, reading sections in the user's language, the translation step. Done 2026-11-11 (#2777) - under the hood. The translation step became the AI's own English search_help.
  6. βœ… Unanswered questions - the list and the admins' bell notification. Done 2026-11-11 (#2776) - under the hood. Questions answered in a module the asker cannot open are not kept.
  7. Optional: vectors for other languages.

Related

FreeITSM β€” an open-source IT Service Management platform Β· github.com/edmozley/freeitsm Β· MIT licence

FreeITSM

Getting Started

Modules

Multi-tenancy (planned)

Responses to feedback

Blue sky thinking

Bugs resolved

Pull requests

Links

Clone this wiki locally