Blue sky: full-text search across tickets and attachments
Records where the design conversation from discussion #53 landed, including the
ideas ruled out -- which is the part that otherwise gets re-derived.
The governing insight is that extraction is the hard half and indexing is not.
MySQL 8.0 already does full-text natively; turning a PDF into text is the
genuinely difficult problem. So the useful third-party integration is a document
parser, not a search engine.
Covers what phase 1 would be (ticket bodies and notes first, attachments in
optional tiers), the security rule that the QUERY must carry the permission
controls rather than results being filtered afterwards, the settings path for
very large installs, and the anti-brick-wall rules that keep the backend
swappable.
Ruled out, with reasons: vectors as the foundation (they sit downstream of
extraction and don't remove it, cosine-in-PHP doesn't scale, semantic search is
weakest at serial numbers and error codes, and embedding every attachment ships
customer documents to a third party on a self-hosted product); an external
search engine as the STARTING point (still extracts nothing, adds a sync problem
MySQL doesn't have); post-filtering results; and any hard PDF dependency.
Two corrections to how the request was framed, marked on the page: the extracted
text must NOT be one of the things a settings screen relocates -- it stays in the
FreeITSM database and is the source of truth the index is rebuilt from, which is
what stops an engine change becoming a re-extraction project. And the
conversation had drifted attachment-heavy: ticket bodies and notes are the bulk
of the value and the cheapest part.
Also adds what the brief left out: whether history gets backfilled, whether
deletion and GDPR erasure cascade to the index, that snippets are themselves
content, and that extracted text is a plaintext copy carrying the database's
protections rather than the file's.
Listed on Blue-Sky-Thinking and in the sidebar.
Blue sky: CRM and a real service catalogue
A parked-ideas page in the established shape — honest about the catch,
not just the upside.
Grounded in the actual schema rather than written as a wishlist, and the
research turned up two facts that became the spine of the argument:
1. FreeITSM knows far more about the companies it BUYS FROM than the ones
it SERVES. `suppliers` has 19 columns (legal name, VAT, reg number,
address, due-diligence dates); `tenants` has 6. `contacts` records a
supplier contact's job title, mobile, direct dial and switchboard;
`users` records a customer contact's theme preference and no phone
number at all. Every contract in the system hangs off supplier_id —
there is no customer-side agreement anywhere.
2. There are ALREADY THREE unrelated ideas of "a service" — CMDB objects
of the Service class, the status page's hand-kept status_services, and
portal-visible forms. None know about each other. So a catalogue's
first job is reconciling those three, not becoming a fourth.
The page argues the useful version is not a sales CRM (no pipeline, no
campaigns, not the system of record for a sales team) but the account
layer an ITSM tool is uniquely placed to hold, and ranks the ideas by
value over effort:
- cost-to-serve, which is buildable almost entirely from data already
captured — ticket_time_entries.time_spent_minutes exists; the only
missing input is an analyst cost rate
- renewal warning, where contracts already has end dates, notice periods
and a contract.expiring workflow trigger, just pointed at suppliers
- account health from evidence (SLA breaches, CSAT, reopen rate), with
the rule that every number must click through to its tickets, and NO
single composite score
- entitlement, which is what makes a service catalogue earn its place
- feeding the CMDB blast radius a commercial question
Avoid-list includes the ones that would actually bite: two sources of
truth for "company" when tenants is a security boundary; sales users
inside an analyst RBAC model; personal data arriving with no retention
answer; a portal that reads like a storefront; and cost-to-serve pointed
at individuals, which would just stop people logging time.
Left explicitly undecided: whether a customer contract is a flag on
contracts or a separate entity, and whether an "account" is a tenant or
something below it — the second changes the schema, which is the main
reason this is parked rather than scheduled.
Every schema claim on the page was re-verified against the live database
after writing. Indexed on Blue-Sky-Thinking and added to the sidebar; all
eight outbound links resolve.
Wiki: split the PHP floor page, refresh Roles & Permissions, restructure the sidebar
The PHP floor page had grown to 861 lines and was three documents in a trench
coat: a decision, a language tour, and a deep argument about permission keys.
Split into three, each standing alone:
Raising-the-PHP-Floor (307) the DECISION — where we are, who breaks, the
verdict, and the constants-to-enums bridge
PHP-8-Language-Features (261) the 101 tour of 8.0/8.1 with before/after from
real code, including the features that WOULDN'T
help us, said plainly
PHP-Enums-and-RBAC (353) the one feature the argument turns on — and the
reason a capability is a Cap:: constant and never
a string. Worth reading even if the floor never
moves, because it explains the permission system.
Section numbers and cross-references rewritten by meaning rather than by
find-and-replace, since the sections changed pages.
Roles & Permissions was stale: written before the manifest became the single
declaration and before the endpoint audit existed. Now says that capabilities are
derived from one file per module (and why the four-list version was the bug), and
carries a table of what D005 has actually found — the shared settings endpoint,
six Intune endpoints, all 49 RFP Builder endpoints, unauthenticated audit-log
forgery, and an LMS learner reaching the authoring settings. Every one found by
hand, by accident, which is the point.
Sidebar: the security pages are now grouped by the three layers they belong to
(module access → capabilities → administrator), rather than a flat list, and all
nine RBAC/PHP pages are linked. The two new PHP pages nest under the floor page.
Blue sky: raising the PHP floor to 8.1
The case for dropping PHP 7.4/8.0 to unlock enums for the RBAC capability
system, plus a 101 on what PHP 8.0/8.1 actually added, with before/after
examples from real FreeITSM code.
Parked, and the page says why: class constants close the dangerous failure
mode (a typo in a capability key silently 403-ing, invisibly to the admin)
without leaving 7.4. What the enum adds on top is elegance, not safety, and
that doesn't justify breaking every 7.4 user's upgrade path. Documents the
constant-to-enum bridge so the swap stays cheap, and the triggers for
revisiting.
Linked from the Blue Sky hub and the sidebar.
Add CSP hardening blue-sky page (issue #43)
Documents what strict-CSP support would take and why it's a large phased
refactor rather than a bug fix: ~1,300 inline handlers + 159 inline script
blocks, plus a measured risk assessment (defence-in-depth behind existing
output escaping; low-to-moderate urgency for a self-hosted internal app). Also
captures the working 'unsafe-inline' policy for today. Linked under Blue sky
thinking and cross-referenced from Email Rendering & Images.
Add "Blue sky thinking" section for parked ideas
New landing page explaining it's a holding area for explored-but-uncommitted
ideas (risk / complexity-vs-reward / uncertain demand), with Remote Image
Archiving moved under it. Added a sidebar section and back-links.