LMS Competency Tests Developer Guide

Competency tests - developer guide

3.0.0 · user guide: Competency tests

The questions are kept in a bank, tests are built from the bank, and each candidate sits a frozen snapshot of one test through a token link. This page covers where everything lives and the rules that must not be broken.

---

Files

	File	What it is
⚙️	includes/lms/competency_tests.php	every rule - validation, scoring, snapshot, token, clock, retention, the AI prompt and its parser
🖥️	api/lms/tests.php	the analyst API - tests, bank, candidates, settings. All behind Cap::LMS_TESTS
🌐	api/lms/test_public.php	the candidate's API - start, save, submit. No session; the token is the credential
🌐	lms/test.php	the candidate's page (public, i18n_guarded, self-contained CSS, works on a phone)
🖥️	lms/tests/index.php, edit.php, result.php, _page.php	the analyst pages; _page.php holds the gates and the <head>
🎨	assets/js/lms-tests.js, assets/css/lms-tests.css	one script for the three pages, chosen by <body data-ct-page>
🔐	includes/capabilities.php, lms/settings/manifest.php	Cap::LMS_TESTS = 'lms.competency_tests', claimed by the settings tab tests (sensitive)
⚙️	lms/settings/index.php	the Competency tests tab; the page now opens for LMS_MANAGE or LMS_TESTS, each seeing its own tabs
🗄️	database/freeitsm.sql, includes/db_verify_schema.php, includes/db_verify_indexes.php, api/system/db_verify.php	five tables, seven indexes, four FKs
🛡️	includes/csrf.php	api/lms/test_public.php is on CSRF_EXEMPT_PATHS
🧪	tests/lms-competency-tests.php	53 checks, the candidate endpoint over HTTP included

Tables

Table	Holds
lms_ct_questions	the bank. answers_json = [{"text","marks"}]; status draft / approved / hidden; source ai / manual; role_context (searchable)
lms_ct_tests	title, role description, time limit, pass mark, is_archived
lms_ct_test_skills	the skill rows: skill, difficulty, format, question_count, sort_order
lms_ct_test_questions	which bank questions are on a test, in order. Unique (test, question)
lms_ct_sittings	one candidate's attempt: token_hash, snapshot_json, responses_json, the clock columns, score_percent, skills_json, notes

FKs: skills and test-questions cascade with their test; a test-question cascades with its bank question. A sitting's test_id is ON DELETE SET NULL, because a sitting never needs its test (see the snapshot rule). Deleting a test that has sittings archives it instead, so that the Candidates list can still name it.

The rules

1. Nothing unchecked reaches a candidate

The AI only ever writes status = 'draft'. lmsCtBuildSnapshot() refuses a test holding any draft or hidden question, and refuses an empty test. Everything that sends goes through it. lmsCtValidateQuestion() is the one validator for the editor and the AI. It drops, never "fixes", a multiple-choice question with two right answers, because that is a bad question rather than a labelling slip.

2. The paper is frozen at Send

create_sitting writes lmsCtBuildSnapshot() into snapshot_json: question text, answers in a shuffled order (Fisher-Yates over random_int), marks and max. From then on the sitting reads only its snapshot:

• the candidate's page renders from it
• scoring reads it
• the result page shows it

Editing, hiding or deleting a bank question, or deleting the test, changes nothing about a sent paper. Don't add a code path that joins a sitting back to lms_ct_questions.

3. The candidate never receives the key

lms/test.php prints question and answer text only: no marks, no max, no explanation, and no JSON of the snapshot. save takes {q, a} indexes into the snapshot and validates them against it. The test suite asserts that the page HTML contains none of these.

4. The token is the credential, and only its hash is stored

lmsCtNewToken() returns 32 random bytes as hex, and the SHA-256 of that is stored. lmsCtFindSitting() rejects anything that isn't 64 lowercase hex before it touches the database. Because the raw link exists only in the create_sitting / new_link response, the UI says it is shown once. new_link works only before Start.

lms/test.php sends Referrer-Policy: no-referrer, Cache-Control: no-store and X-Robots-Tag: noindex, because the token is in its URL. The candidate API reads the token from the body. An unknown token and a cancelled one get the same 404 invalid.

api/lms/test_public.php is on the CSRF exempt list (like the web chat): it has no session to forge, and it carries its own credential. It is also an exemption in tests/module-access-coverage.php, for the same reason.

5. The server owns the clock

• started_datetime is set once (WHERE started_datetime IS NULL).
• The deadline is started + time_limit_minutes. An untimed sitting still closes after LMS_CT_UNTIMED_HOURS (24), so nothing stays open for ever.
• save is accepted until the deadline plus LMS_CT_GRACE_SECONDS (60), which allows for a click already in flight.
• After that, any request closes the sitting with finish_reason = 'time_up' and submitted_datetime = the deadline (not the moment it was noticed), scoring what was saved.
• lmsCtFinaliseOverdue() does the same for sittings nobody touches. It runs whenever the Candidates list or a result is read.
• The page's countdown is the server's remaining seconds against performance.now(), so changing the device clock changes nothing.

save uses JSON_SET(..., CAST(? AS UNSIGNED)) in one statement, so two quick clicks on different questions can't overwrite each other.

6. Scoring: every question weighs the same

For each question, the score is the marks for the chosen answer divided by the best marks on offer. Unanswered scores 0. The overall percentage is the mean of those fractions, and so is the per-skill percentage (grouped by skill + difficulty). A graded question worth 0-5 therefore counts no more than a 0-1 multiple-choice one. score_percent and skills_json are stored at close, so the list doesn't recompute them.

7. Retention is personal-data housekeeping

lmsCtPurge(), at most once a day (lms_ct_last_purge) and run from the Candidates list, deletes sittings older than lms_ct_retention_days (default 180; 0 = never) that are over: submitted, cancelled, or never started with the link out of date. A sitting started and never finished is closed by rule 5 first, and only then aged out.

The AI

lmsCtGenerate() makes one call per skill (at most 20 questions per call), using the LMS AI config (aiSettingsLoad($conn, 'lms_ai')). The prompt carries:

• the role description and the skill
• a description of what the difficulty level means
• up to 40 existing question texts on that skill, so it doesn't repeat itself
• a rule that every answer must be similar in length and detail. A live run showed the right answer was always the longest; shuffling hides its position, not its length.

Graded questions come back as four strings, best first, and get the install's marks in that order. lmsCtParseDrafts() is separate from the call, so the parsing is tested against sample replies (fenced JSON, a question with two right answers, a graded question with three answers, a prose refusal) without spending anything.

generate (the API) fills a skill's shortfall: what the test already holds for that skill + difficulty + format counts first. Then come approved bank questions (ORDER BY RAND(), not already on the test, unless use_bank is off), and only then the AI.

Permissions

Who	Can
Cap::LMS_TESTS (Competency tests, sensitive)	everything on this page: LMS → Tests, and the Settings tab
Cap::LMS_MANAGE	nothing here - course management only
admin	everything, as always

The LMS header shows Tests to LMS_TESTS holders, and Settings to holders of either grant. Every analyst page calls requireModuleAccess('lms') and requireCapability(Cap::LMS_TESTS); the API calls requireModuleAccessJson and requireCapabilityJson.

Settings keys

lms_ct_link_days (7), lms_ct_time_limit (45), lms_ct_graded_marks (5,3,1,0), lms_ct_retention_days (180), lms_ct_show_score (0), lms_ct_last_purge. They are written by api/lms/tests.php (save_settings), not by the generic settings writer, so the manifest tab has no setting_keys.

Strings

The analyst pages' strings are in lang/en/lms.php under tests, and the candidate page's under candidate. Every call site passes its English (lt() in PHP, L() → window.tf in JS, tr() on the public page), so an untranslated key shows English, never a key.

Testing

FREEITSM_URL=https://your-install/ php tests/lms-competency-tests.php

The suite creates its own questions, test and sittings (tagged), and deletes them in finally. It puts the purge's last-run day back. It skips the retention checks if the install already has old candidates of its own, because a test must never delete someone's real results.