Removed
- Sigma eval subsystem decommissioned (2026-08-19): the end-to-end Sigma rule eval scored generated rules against hand-authored ground truth in
config/eval_articles_data/sigma/ground_truth.json, reporting detection-atom and logsource precision/recall. Operator review concluded the headline metric is not defensible.sigma_eval_scorer.score_sigmacomputes precision asfp = actual_atoms - expected_atomsover exact normalized-atom set equality, so any correct detection the ground-truth author did not happen to write down scores identically to a hallucination, and a generator choosing a tighter discriminator than the reference is penalized twice for the improvement. The benchmark therefore measures resemblance to one analyst's rule set rather than detection quality, capped at -- and actively penalizing exceeding -- that analyst's recall of their own domain. The 2026-08-16 arm-blind ground-truth protocol addressed contamination between the multi-extractor and one-shot arms, but a neutral referee measuring the wrong quantity is still measuring the wrong quantity, so expanding the corpus from 4 to the prescribed 15-20 articles would only have bought statistical power on that metric. Every remaining signal was already covered elsewhere: the logsource axis is largely downstream of extractor correctness (measured by Eval1/Eval2 against ground truth that does have a fact of the matter), and rule validity is independently enforced bySigmaGenerationService._validate_all_rulesplus its Phase 3 repair loop. Removedsigma_eval_scorer.py,sigma_eval_service.py,sigma_evals.html, the/mlops/sigma-evalspage, the four/api/evaluations/sigma-eval-*endpoints, theSigmaEvaluationTablemodel,config/eval_articles_data/sigma/, and 8 test/doc files. Thesigma_evaluationstable is dropped viascripts/migrate_drop_sigma_evaluations.pyrather than retained, because its unqualified FK toagentic_workflow_executionswas guarded bydata_retention_service._EXECUTION_REFERENCE_TABLES; keeping the rows while removing the model would have left the shared retention job to hit a FK violation on the first aged execution it tried to purge. The Eval1/Eval2 extractor evals are untouched -- the workflow'ssigma_evaloverride collapses to an identity ((not False) and X == X) for extractor runs and production, andtests/api/test_removed_dead_endpoints.pynow pins both the removal and the survival of everysubagent-evalroute. Deploy note: runscripts/migrate_drop_sigma_evaluations.pyas part of shipping this change, not after it. The retention guard is removed in the same commit, so any environment running this code with the table still present will raise a FK violation on the first aged execution the retention job tries to purge -- and that job is shared with article pruning and the extractor evals, so the failure would take retention down for Eval1/Eval2 data too. Applied to the local dev database on 2026-08-19 (9 rows).
Added
- SIGMA enrichment supports the Codex subscription and bounded follow-up chat (2026-08-24): the queue enrichment modal now exposes the same opt-in, subscription-backed Codex provider and model catalog as agent configuration, without requiring an OpenAI API key. Each successful enrichment also persists its provider, model, and rule-scoped transcript.
Start Chatreopens that context after a reload and replays it through the original provider (OpenAI, Anthropic, LM Studio, or Codex), so follow-up turns stay limited to revising the current SIGMA rule or discussing earlier turns. System instructions remain server-side, and the continuation guardrail refuses unrelated research, tool use, or external actions. Covered by focused API and Playwright tests. - Retention policy for workflow config history (2026-08-22):
agentic_workflow_confighad no pruning at all and grew with ordinary UI use. It is now adata_retention_servicepolicy running from the existing scheduled maintenance job rather than inline on write, with a 60-day window and a floor of 2,000 revisions -- whichever keeps more wins. The floor is what does the work: this table is the undo history for the workflow config (the CmdlineExtract prompt was recovered from config id 7224), so the depth is set by how long a regression can plausibly go unnoticed, and age alone would have kept everything from a busy week and nothing from a quiet month. The busiest observed day wrote 632 rows, so a smaller floor could be exhausted by one afternoon of editing. Never deleted at any age: the active row, and any row cited bysubagent_evaluationsor -- while it still exists --sigma_evaluations, which carry that eval's provenance. Windows are overridable viaRETENTION_DAYS_WORKFLOW_CONFIGandRETENTION_MIN_WORKFLOW_CONFIG_REVISIONSinapp_settings. - Agent prompt validation runs on load and badges the failing agent (2026-08-21):
_collectPromptIssues(static/js/workflow/prompt-editor.js) already detected prompt-contract drift accurately -- verified across all seven extractors on 2026-08-19, where four passed clean, RegistryExtract and ServicesExtract reported one warning each, and a degraded CmdlineExtract seed reported seven naming exactly the contract sections its test-seed replacement had dropped. But it only ever ran when a human clicked Validate inside a panel three levels deep (Step 3 -> sub-agent -> Prompt panel -> Validate), so a prompt that had been broken for two days sat behind a button that would have flagged it instantly. The checks are reused unchanged; only the surfacing is new.refreshPromptWarningBadges()validates all nine prompt-bearing agents with no click and writes a pill onto the collapsed.sa-headerand the owning step's.section-header, so an operator sees it without expanding anything. It is wired at the tail ofrenderAgentPrompts()-- the single funnel for initial load and every save, cancel and preset load -- so a badge cannot outlive the prompt it describes. Warnings are amber and never block saving or running; a hard-fail (missingsystem/role,instructionsorjson_example) is red. The Validate button and the badges now read through one shared_promptValueForValidation(), so the two can never disagree about what was validated. One deliberate exemption: an agent with no stored prompt is running the shipped default (default_agent_prompts.AGENT_PROMPT_FILES;rank_article()falls back tosrc/prompts/rank_article.txtat the call site,routes/ai.py), which is the baseline state and not drift -- RankAgent ships exactly this way today, so badging it would have put a permanent false alarm on every page load and taught the operator to ignore the badge, the precise failure this feature exists to prevent. Extraction sub-agents get no such exemption: an emptysystem/rolethere really does raisePromptConfigValidationError(services/llm_prompting.py). No new Tailwind classes were introduced, so the committed build artifact is unchanged. Covered by 5 tests intests/ui/test_workflow_comprehensive_ui.py-- badges settled at load with no interaction, badge/button agreement, refresh through therenderAgentPromptsfunnel, a degrade-and-restore case asserting amber for a stripped role body and red for a removed required key, and the exemption case pinning both halves (an unset non-extractor is silent, an emptied extractor is not). All five were confirmed to fail against the unwired code rather than passing vacuously. - Security response headers, including a two-tier Content Security Policy (2026-08-21): the app sent no security headers at all -- no CSP, no framing control, no MIME-sniffing control. Escaping is the primary defence for the DOM-XSS sinks in these templates, but it has been missed more than once, and nothing was behind it. A new
SecurityHeadersMiddleware(src/web/security/headers.py) is registered outermost, so headers land on every response including authorization denials, error pages, and static assets -- the responses an unauthenticated attacker can reach. Enforced CSP carries only directives the app already complies with, so it cannot break a page:object-src 'none',base-uri 'self'(an injected<base>otherwise repoints every relative URL on the page),frame-ancestors 'none', andform-action 'self'(stops an injected form posting operator data off-origin). Alongside it:X-Frame-Options: DENYfor older browsers,X-Content-Type-Options: nosniff, andReferrer-Policy: strict-origin-when-cross-origin. Report-Only carries the strict policy that cannot be enforced yet. The blocker isscript-src: 26 inline<script>blocks (~22,000 lines across 19 templates) and 341 inline event-handler attributes. Nonces would authorise the blocks, but nothing authorises a handler attribute -- permittingonclick=necessarily permits an injectedonerror=, which is exactly the payload the recent escaping fixes addressed. Realscript-srccoverage therefore requires migrating those handlers toaddEventListener; Report-Only measures that surface rather than guessing at it.style-srcdeliberately retains'unsafe-inline', because reporting everystyle=attribute would bury the script violations that matter. HSTS is intentionally not sent from the app: pinningmax-ageagainst a plain-HTTP dev host is a footgun with no benefit, and it belongs with the TLS-terminating deployment config. Covered bytests/unit/test_security_headers.py(11 tests), which pin that the enforced policy never grows ascript-src/default-src/style-srcdirective without the Phase 2 migration, that Report-Only keeps constraining scripts so the measurement stays meaningful, and that the middleware is registered and outermost -- without that last pair, deleting theadd_middlewarecall leaves every other test green while the app serves no headers at all. /articles: a visible indicator when the filters panel is collapsed but a filter is active (2026-08-21): the only on-screen sign that a search or filter was active -- theActive: "<term>" Clear allchip -- lived inside the collapsible filters panel, so collapsing it (a preferencesessionStorageremembers across visits) left a filtered list of, say, 279 of 7,741 articles with no visible reason why. Added#filters-active-badge, shown only while the panel is collapsed and a filter is active, reading "N filter(s) active - Clear"; it tracks the real open/closed DOM state from both the initial-load path and the header-click toggle, and the existingsessionStorage-remembers-collapse behavior is untouched. Covered by 3 new tests intests/ui/test_articles_advanced_ui.py: no badge when unfiltered, badge appears on collapse-with-filter (the exact regression case), badge hides again on re-expand.- SIGMA Queue summary gets a fifth card for
needs_review, its largest bucket (2026-08-21):#queueStatsrendered exactly four cards -- Pending Review, Approved, Rejected, Submitted -- summing to 394 against a queue of 808 rules;needs_review(414 rules, 51%) was read by neitherupdateQueueStats()nor any stat card, so the largest single bucket had no one-click filter shortcut despite the status dropdown listing it. Added aNeeds Reviewcard wired to the samesetQueueStatusFilter()pattern as its siblings, and widened#queueStats's grid to 5 columns without affecting the sibling 4-card Executions stats grid, which shares the same CSS class. Covered by a new test intests/playwright/sigma_queue_lifecycle.spec.tsasserting the card renders, its count is numeric, and clicking it filters the queue.
Fixed
-
LLMGenerationServicehad no codex branch and would raise on a codex benchmark run (2026-08-31):_get_model_name()fell through to"template"for codex and_call_llm()'s dispatch raisedValueError(f"Unknown provider: {provider}")-- a second, duplicated provider-dispatch chain that drifted the moment codex was added toLLMService/llm_client.py(the chain the running app actually uses) without a matching update here. Reachability check (rg -n "llm_generation_service|LLMGenerationService" src/ tests/ scripts/) found exactly one consumer:scripts/benchmark_llm_providers.py, a dev-only tool, not the running app. Decision: delete rather than extend, removing the duplicate chain instead of adding a second place a future provider must be wired into. Deletedsrc/services/llm_generation_service.pyand repointed the benchmark script atLLMService.request_chat(), which already dispatches openai/anthropic/lmstudio/codex correctly -- the script now also benchmarks codex, and force-enables LM Studio's workflow-scoped enablement toggle for itself since it benchmarks LM Studio directly, independent of that agent-config gate. Verified:rgreturns no remaining hits,run_tests.py smoke(87 passed) and the fullllm_service/llm_clienttest suite (100 passed) are green,ruff checkis clean. Covered by 7 new tests intests/scripts/test_benchmark_llm_providers.pypinning the LM Studio gate override, the codex availability gate (mirroring openai/anthropic, never raising), andrequest_chatcall/response wiring; 6 of the 7 were confirmed to fail against the parent revision (still onLLMGenerationService) when run in an isolated worktree. -
Sigma rule validation silently ran the wrong provider, reporting a provider/model pair that existed nowhere in config (2026-08-31): with
SigmaAgent_providerset tocodex, clicking Validate Rule reportedProvider: Lmstudio | Model: gpt-5.6-soland failed three times with "model is not loaded". Neither half was wrong on its own -- the model came from the config, the provider came from a fallback -- but they were resolved independently, so an unrecognised provider name was swapped for_first_enabled_provider()while the configured model name was kept.codexwas simply absent from the validate path's allowlist, though it is a first-class provider in generation and enrichment and_call_traced_sigma_provideralready had a workingCodexAppServerClientbranch. The result turned a config problem into a phantom failure attributed to a provider the operator never selected. Three gates needed it: the resolver's allowlist, the retry-loop dispatch, and an API-key guard keyed onprovider != "lmstudio"that demanded a key for keyless Codex -- fixing only the first two moves the failure to "No Codex API key is configured". An unrecognised provider now raises and names itself instead of substituting,_first_enabled_providercan returncodex(keyless, so invisible to the API-key scan), and both keyless providers read their enablement flag through the same AppSettings-then-env path. Verified end to end against the live app: the same rule now reportsProvider: Codex | Model: gpt-5.6-soland validates on attempt 1. Covered by 28 tests intests/api/test_sigma_queue_validate_api.py, all 10 new ones confirmed failing against the parent revision. -
The workflow config accepted provider/model pairs that can only fail at call time (2026-08-31): nothing validated that
<Agent>belongs to<Agent>_provider-- not the PUT handler, notworkflow_config_schema.py, not the DB -- which is how an agent could come to hold one provider's name and another's model (static/js/workflow/config.jsdocuments anExtractAgentstored aslmstudiowith an OpenAI model). Such a pair is invisible until a run dies with a confusing "model not found" from the wrong provider.PUT /api/workflow/confignow rejects a pair the catalog positively attributes elsewhere. Three properties make it safe rather than merely strict:openaiandcodexshare one model namespace (Codex serves the OpenAI family, and the catalog has nocodexkey -- without this the live config and 3 of the 12 shipped quickstart presets would all be rejected); models absent from the catalog are never rejected, so LMStudio locals and new releases pass; and only pairs the request changes are checked, because autosave sends the entire blob and swallows its own 400 to the console, so whole-blob validation would silently discard unrelated edits and make the repair itself unsaveable. Ownership reads the catalog file as written rather than throughload_catalog()'s dropdown display filters. Covered by 25 tests intests/unit/test_workflow_config_write_safety.pyand 10 intests/services/test_provider_model_catalog.py, including a guard that all 12 shipped presets still import and per-agent coverage for the<Agent>_modelkey shape the 7 sub-extractors use. -
POST /api/validate-modellet a nonexistent codex model ID through as valid (2026-08-31):api_validate_model()ranis_valid_openai_chat_model()only forprovider == "openai"; acodexrequest fell through with no model check at all, so the endpoint reported a made-up model likegpt-5.6-nonexistentas valid even though codex serves the same OpenAI model catalog (gpt-5.xseries) that theopenaibranch already validates against.provider in ("openai", "codex")now shares that check, and the rejection message names the provider correctly ("Codex" vs "OpenAI") instead of always saying OpenAI. Covered by 11 new tests intests/api/test_model_validation.py, including a codex/openai parity case pinning that both providers reject the same invalid model ID. -
HTML page routes with a malformed request param now render the styled error page instead of a raw FastAPI JSON blob (2026-08-28): a non-integer path or query param (
/articles/abc,/articles?per_page=abc) hit FastAPI's defaultRequestValidationErrorhandling, which returns{"detail": [...]}regardless of whether the request came from a browser tab or the API -- an operator navigating to a bad URL saw raw JSON instead of the app's chrome. A newvalidation_error_handlerfollows the same request-path split the existingnot_found_handleruses:/api/*requests keep FastAPI's structured JSON contract unchanged, everything else renderserror.htmlwith a 422. Covered by 3 new tests intests/api/test_endpoints.py, including one pinning that the API contract (detail[0]["type"] == "int_parsing") is untouched. -
/articles: a zero-result search no longer shows "Articles 1-0 of 0" (2026-08-28): the pagination header always renderedArticles {start}-{end} of {total}, andstart_idxis computed as>= 1even whentotal_articlesis 0, so a search matching nothing read as the nonsensical "Articles 1-0 of 0" instead of communicating there were no results. The header now shows "No articles" when the total is zero. Covered by a new test intests/api/test_endpoints.py. -
/diags: Queue Status no longer hidescollection_immediateor attributes every queue to the wrong worker (2026-08-28):GET /api/jobs/queueshardcoded an 8-queue literal that had drifted fromceleryconfig.task_queues(9 entries) -- missingcollection_immediate, the queuecollect_from_source(the on-demand source-collection path) actually routes to, so a real backlog on the queue most likely to back up was invisible and the header chip undercounted at "8 Queues". The queue dict is now a comprehension overceleryconfig.task_queuesdirectly, so a queue added or removed there can't silently drift out of what this endpoint reports; the "default" queue's Redis-key special case (llen("celery"), notllen("default")) is preserved but keyed offceleryconfig.task_default_queueinstead of a string literal.priority_checks-- present intask_queueswith zerotask_routesentries and consumed by neither worker's-Qflags indocker-compose.yml-- was explicitly removed as dead configuration, with a comment recording why, rather than silently dropped from just the display. Separately,updateQueueStatus()attributed each queue's consumer by testing the Celery node name (celery@<hostname>) for the substring"workflow", which a container hostname never contains, soworkflowsalways showed "General" while the Worker Status card 400px away correctly showed "Workflow Worker" for the same worker -- fixed by testing the queue name instead (name === 'workflows'), the same rule Worker Status already used, so the two cards can no longer disagree. Verified end-to-end, not mocked: stoppedcti_worker, fired 3 realcollect_from_sourcetasks, confirmedcollection_immediateshowed a live "3 Pending" on the page whilecti_workflow_workerstayed up andworkflowscorrectly attributed to "Workflow" in that exact state; restarted the worker and confirmed the backlog drained to 0. Covered by 2 new tests intests/api/test_api_route_coverage_gaps.py, including one pinningset(queues.keys()) == set(celeryconfig.task_queues.keys())as an explicit CI drift guard. Fullrun_tests.py api: 584 passed. -
/diags: Job History attributes each row to its real task and queue instead of a UUID-substring guess (2026-08-28): the card promised "inferred worker/queue assignment", but/api/jobs/historynever returned a task name, sogetTaskQueueInfo()matched substrings (workflow,check_source,cleanup, ...) against the raw task UUID -- which can never contain them -- and every row fell to the sameGeneral · default, presented as derived fact. Root fix:celeryconfig.pynow setsresult_extended = True, so Celery itself stores the real tasknamein eachcelery-task-meta-*Redis entry going forward (no custom instrumentation).api_jobs_historyreads that name and resolvesqueueby looking it up inceleryconfig.task_routes(falling back totask_default_queuefor a real-but-unrouted name, "unknown" only when the name itself isn't recoverable -- e.g. a pre-migration entry within the 30-minuteresult_expireswindow), and derivesworker_typefrom the queue via the same worker/queue mapping as the Queue Status fix above. The template deleted its substring ladder entirely and renders the server-provided fields; the card subtitle now reads "worker/queue from Celery's routing config" since it no longer is a guess. Verified with a genuinely fresh task (not a mock): triggered a realcollect_from_sourcerun, and both the API and the live page correctly showedsrc.worker.celery_app.collect_from_source/collection_immediate/General. Covered by 4 new tests intests/api/test_api_route_coverage_gaps.py(routed task, workflow-queue task, unrecoverable name, unrouted-but-known name). Fullrun_tests.py api: 582 passed (before the Queue Status fix landed in the same session; 584 after both). -
/diags: the false "TESSERACT missing" alarm is gone, and a genuinely degraded service now actually reddens the card (2026-08-28):check_tesseract_available()was probed in the web process, which has never hadpytesseractinstalled -- it lives in the optionalingestextras group used only bycti_worker/CLI -- so the External Services card permanently showed a rawModuleNotFoundErroras a "missing" failure for a dependency this process was never meant to have; an operator learns to ignore a standing false alarm, which is how real ones get missed.api_services_healthnow catchesModuleNotFoundErroronimport pytesseractdirectly and reportsstatus: "not_applicable"with a plain-English message, falling through to the real check unchanged if pytesseract is ever importable there. Two more defects in the same card: the top-levelstatuswas hardcoded"healthy"regardless of any component's real state (extracted into_compute_services_rollup_status(), now"unhealthy"when any service reportsunhealthy/error/missing), andgetHealthIcon/getHealthColordidn't handle"ok"(tesseract's own success status) or"missing", so even a working check silently rendered?instead of a checkmark. Fixing the roll-up exposed a masking bug:updateServicesHealth()'s per-service breakdown was gated ondata.status === 'healthy', harmless while that was always true, but once real, a genuine failure hid every service's individual status behind a bare "Unknown error" -- changed the gate to check fordata.servicesbeing present instead, which still correctly falls through only on the route's actual catastrophic-failure path. Verified live: the real (unmocked) response now shows a clean not_applicable message and a green card; a mocked genuinetesseract: missingfailure correctly turns the card red while still listing every service. Covered by 6 new unit tests (tests/unit/test_services_health_rollup.py), an updatedtest_health_reports_tesseract, and 2 new Playwright tests (DIAGS-TESSERACT-001/002). -
/diags: the loading overlay is keyboard-trapped and focus-managed; auto-refresh honestly reflects what it covers (2026-08-28): building on the batch-overlay/button fix ina72c3a31, two gaps remained. The overlay had no focus management at all -- opening "Run All Health Checks" never moved focus into the dialog (aria-modal="true"against a background that stays fully tabbable) and never restored it on close; fixed withtabindex="-1"plus a keydown trap pinning focus on the overlay itself (it has no focusable descendants) for the run's duration,aria-busy="true"on the trigger, and focus restored to the trigger on completion. Separately, the five health cards (System Health, Database, External Services, Deduplication, Celery) are only ever revalidated by a manual click, but shared the toolbar's 5-second "Last updated" clock with the auto-refreshing job data, implying a freshness they didn't have; each card now carries its own "checked at" timestamp (turning amber after 5 minutes stale, re-evaluated live on each job-data tick with no new requests), and the toolbar clock is relabeled "Job data last updated" with a tooltip naming its actual scope.isAutoRefreshalso derived from a hardcodedtrueliteral rather than the checkbox when no stored preference existed yet; it now reads the checkbox's own state in that case. Verified live via JS: focus enters/traps/restores correctly around a real batch run; forcing an old timestamp flips the stale class; unchecking auto-refresh survives a real reload. Covered by 2 new Playwright tests (DIAGS-FOCUS-001,DIAGS-FRESH-001). -
/workflow#executions: the Step filter can now select every step that actually occurs, including the failure step (2026-08-28): the dropdown offered 6 values whilecurrent_stepcontains 7 distinct non-empty values plus a 106-row empty-string bucket;os_detection(pipeline step 0, 58 rows) andcontext_length_check(a pre-flight gate failure, 2 rows -- the exact FAILED execution most worth filtering for) were both unreachable. Added both as real options (verified as genuinecurrent_stepvalues set inagentic_workflow.py, not typos).rank_articleandsimilarity_searchshow 0 rows in the current dataset but are liveworkflow.add_node(...)steps, not dead code, so they're kept with a comment explaining why rather than silently dropped. The empty-string/NULL bucket gets an explicit, deliberate treatment rather than being ignored: an(unset)option using a__unset__sentinel (the empty string was already taken by "All Steps"), translated server-side by_step_filter_condition()intocurrent_step IS NULL OR current_step = '', applied at both the count and list query sites. Verified against the live dataset:os_detection-> 20 matches,context_length_check-> 2,__unset__-> 99 including acurrent_step: nullrow. Covered by 2 new integration tests seeding real NULL/empty/named rows (tests/api/test_workflow_executions_unset_step_filter.py). -
Keyword highlighting no longer fragments IOCs and hyphenated words mid-token (2026-08-28): a keyword match ending mid-word (e.g.
"detection"matching inside"detections", or the[.]defang marker matching inside45.153.34[.]132) was wrapped in a<span>covering only the matched substring, visually splitting the surrounding token -- 58 of 180 highlights on one test article sat mid-token, and the worst cases were exactly the strings a CTI reading pane exists to make legible. Fixed entirely inrender_highlighted_content(the rendering layer, as diagnosed -- the underlying match data driving the Keyword Matches panel is untouched): each match's displayed span is widened to its natural token edge using a curated continuation-character set (alnum plus.-/:[]_@, covering dotted/defanged IPs, hyphenated compounds, paths, ports) before being wrapped, rather than an unconstrained "expand to nearest whitespace" (which was tried first and caught by the existing XSS-escaping test: it would swallow long punctuation-heavy runs, e.g. literal<span class="...">appearing as escaped plain text, into the highlight). When two distinct matches land inside the same token (e.g. a filename split across two keyword hits) their natural bounds overlap and are rendered as one combined span rather than re-split at an arbitrary internal point. Also removed thepx-1padding andborderfrom the highlight<span>so it reads via background and font-weight alone, per the second half of the fix. Verified on the article named in the original report: 175 highlighted spans (was 180 DOM spans before the merge-adjacent fix -- fewer spans over the same underlying matches, not fewer matches), 0 boundary violations checked programmatically. Covered by 5 new tests intests/test_keyword_resolution.py, 3 of which were confirmed to fail against the pre-fix code (extracted from git history into an isolated module, no working-tree changes) before passing against the fix. -
/settings: Scheduled Jobs shows a plain-English cron preview, marks the exact field a rejected cron names, and promotes the editable control over its own internals (2026-08-28): raw cron (0 2 * * *) sat one section below friendly time pickers for the conceptually identical Backup Configuration, with no preview, no next-run time, and no client-side validation; a rejected cron collapsed to a bare422 {"detail": "Validation error"}with no way to tell which of four fields was wrong, even thoughvalidate_cron_expressionalready built a precise message that the route discarded.ScheduledJobsConfigErrornow optionally carries ajob_id, attached wherevernormalize_scheduled_job_configfails a specific job's cron or enabled value; the route returns{"detail": {"message": ..., "job_id": ...}}instead of the flat string, and the UI puts a red border,aria-invalid, and an inline error box on the exact offending field, with the toast naming the job by label. Each Cron Schedule field now shows a live plain-English preview and next-run time (a small client-side cron parser + a bounded brute-force next-occurrence finder, both preview-only -- the server stays authoritative), andsaveScheduledJobs()validates client-side before the round-trip so an obviously malformed cron never reaches the network (still atomic: nothing persists either way). Card hierarchy inverted: Cron Schedule is now the first, largest element; Task/Queue/Registered name/Default cron moved into a collapsed native<details>disclosure. Verified live end-to-end, including the real 422 job_id attribution round-trip and a full valid save. Covered by 5 new backend tests (job_idattribution and omission) and manual Playwright-script verification of the preview/validation UI. -
/sources: metadata columns align across cards, search matches the domain it displays, and action buttons no longer bury Configure/Stats behind an unlabeled menu (2026-08-28): three coupled findings on the 39-row Sources list. DOMAIN/LAST CHECK/FREQUENCY/LOOKBACK were content-sized per card, so a long domain (symantec-enterprise-blogs.security.com) pushed every later column out of alignment with its neighbors -- fixed with a fixed 4-column CSS grid plus ellipsis truncation.filterSources()matched onlycard.dataset.name(the source's display name), so searchingsecurelist.com-- the exact domain text printed on the Kaspersky card -- returned 0 of 39 results; it now also matches adata-domainattribute, and the query is trimmed so pasted whitespace no longer misses. Collect Now, which starts a live scrape job, was the filled primary button on every card while the cheaper, more frequent Configure/Stats/Toggle Status sat two clicks behind an unlabeled···; those three are now always-visible one-click buttons and Collect Now is a de-emphasized outline button pinned right, with a tooltip explaining why it's disabled for an inactive source. Verified live: all three fixes hold across the real 39-card dataset. Covered by 3 new Playwright tests intests/playwright/sources_page.spec.ts. -
/sources: status filter chips and the config modal are operable from the keyboard; each card shows the hunt score the list is sorted by (2026-08-28): the four header chips (37 Active, 2 Inactive, etc.) were<span>s with only anonclick-- 0 of 4 were focusable -- despite being the page's primary filter affordance; they now carryrole="button" tabindex="0" aria-pressedand a keydown handler activating on Enter/Space, matching the/settingsaccordion reference pattern. The source config modal setaria-modal="true"against a background that stayed fully tabbable, never moved focus in on open, never trapped Tab, and never restored focus on close; fixed with a scoped focus trap, initial focus on the first field,aria-labelledby(replacing a stalearia-label), body scroll lock, and focus restored to the exact Configure button that opened it -- which surfaced a real bug in the sharedmodal-manager.js: it auto-registers any[id$="Modal"]element on its ownDOMContentLoadedlistener before a page's own registration runs, silently dropping a page'sonClosecallback unlessforceUpdate: trueis passed. Separately, the list header read "↓ Hunt Score" but no card displayed one (huntScoreShownOnCard: falseon all 39 cards) -- the value already existed server-side in the route's own sort key (hunt_score_lookup, unused by the template) and is now rendered on every card, with the label restyled from purple (reading as a live sort control) to muted gray with a tooltip clarifying it's a fixed order. Verified live: chip Enter/Space toggles the filter andaria-pressed; the modal's Tab wraps correctly at both boundaries and Escape/Cancel both restore focus and unlock scroll; all 39 scores render in confirmed-descending DOM order. Covered by 3 new Playwright tests. -
Opening Junk Filter Tuning issues one feedback request instead of one per chunk (2026-08-26): the modal called
GET /api/feedback/chunk-classification/{article_id}/{chunk_id}once per chunk while rendering, unbatched and un-awaited -- 150 requests in a 224ms burst on article 7216, and 1,250 after a full analysis -- with essentially every one returningfeedback: null, because chunk feedback is rare by nature. A newGET /api/feedback/chunk-classification/{article_id}returns all stored feedback for an article keyed by chunk id, newest-per-chunk, matching what the single-chunk route (ORDER BY created_at DESC LIMIT 1) would have returned for each chunk individually. The reduction runs in Python rather than SQL (noDISTINCT ON/ window function) because per-article feedback volume is tens of rows at most, which keeps the query portable and the ordering contract obvious. Measured after the change: 1 request on a 60-chunk article and 1 on a 150-chunk article. The single-chunk route is retained --submitFeedback()legitimately re-reads one chunk after a write -- and both paths now render through one sharedrenderChunkFeedbackIndicator(). Also fixes an adjacent bug: a threshold change rebuilds every chunk card viagenerateChunkDetails()and nothing put the indicators back, so moving the slider silently erased the fact that a chunk had been reviewed; the bulk call now runs on re-render too (verified 18 indicators survive a real slider move on article 4790). Before trusting the new route its output was compared against the single-chunk route across all 18 of that article's feedback rows -- zero mismatches -- plus the unfed-chunk and empty-article cases. Covered by 5 API tests intests/api/test_feedback_bulk_api.py, including a newest-wins case proven to fail if the reduction keeps the wrong row. -
Junk Filter Tuning dialog is usable by keyboard and screen reader (2026-08-26): five gaps. The close button contained only an SVG, so it announced as a bare "button" -- it now carries
aria-label="Close Junk Filter Tuning"and an explicittype="button". The dialog advertisedaria-label="Chunk debug"against a visible title of "Junk Filter Tuning", so a voice-control user saying the name they can see could not address it (WCAG 2.5.3 Label in Name); it now usesaria-labelledbypointing at the heading itself, which cannot drift from the visible text the way a duplicated string does. Focus was landing on the threshold slider -- notBODYas originally reported, but incidental: it was whateverModalManager's first-input heuristic found -- and now goes deterministically to the close button, which also keeps the Correct/Incorrect buttons (they POST straight into the ML training corpus) off the opening focus position. Closing dropped focus toBODY, restarting a keyboard user's traversal from the top of the page; it now returns to whatever opened the modal. Background scroll was never locked; it now locks on open and restores the previous value on close, on both the Escape and close-button paths.ModalManagerprovides none of this -- it stacks, shows, and focuses the first input it finds, with no focus trap, focus restore or scroll lock -- so the modal wires them itself;docs/contracts/ui-designer.mdnow records that gap and the accessible-name rule so the next modal author does not assume otherwise. Covered by 4 tests intests/playwright/chunk_debug_modal_polish.spec.ts, confirmed to fail against the pre-fix markup. -
A near-duplicate SIGMA rule no longer suppresses the novel rules generated alongside it (2026-08-26): the queue-promotion gate in
queue_sigma_rulescomputed a batch aggregate --max_similarity = max(per-rule scores)-- and, when that single number clearedSimilarityThreshold, setqueued_rules = []before the per-rule loop ever ran. One near-duplicate therefore discarded its entire batch, including genuinely novel siblings, and nothing in the queue or the UI recorded why. It also defeated the fail-open routing added earlier for inconclusive comparisons: a rule the comparator could not assess is meant to reach the review queue flaggedneeds_review, but the batch gate dropped it first. The decision now runs throughselect_queueable_rule_indices, a pure function judging each rule on its own score, with the batch aggregate removed entirely. Semantics are otherwise unchanged and deliberately pinned: the boundary stays inclusive (>= thresholdis a near-duplicate), an inconclusive comparison is still promoted, and a rule that is unscored without being flagged inconclusive is still suppressed. Suppressions are now logged with a count rather than passing silently. The help bubble and the slider caption both said "above this threshold" for what is an inclusive gate, and the help bubble implied batch-wide behaviour; both now state the boundary, the per-rule scope, and the inconclusive fail-open. Covered bytests/unit/test_agentic_workflow_queue_selection.py(7 cases including the exact-boundary and novel-sibling regressions) and 4 wording guards intests/unit/test_workflow_help_bubbles.py. -
Long chunk analyses report progress instead of an undifferentiated spinner (2026-08-26, corrected 2026-08-27): opening Junk Filter Tuning on a large article (7216: 825,777 chars / 1,250 chunks) took ~65 seconds behind a static "Processing article content..." message with no chunk count, no progress and no elapsed time -- indistinguishable from a hang.
GET /api/articles/{id}/chunk-debugnow publishes progress as it works and a newGET /api/articles/{id}/chunk-debug/progressserves a snapshot the loading modal polls once a second. The first attempt instrumented only the per-chunk loop, which is not where the time goes. Measured on 7216: a 64.9s run reported progress from t+28.5s to t+35.6s and nothing outside it -- 11% of the wall time.filter_content()(a full sklearn pass over all 1,250 chunks) runs before the loop and took ~28s uninstrumented, and response assembly afterwards took ~29s during which the client, which only overwrote its text whilein_progresswas true, sat frozen on "Processed 149 of 150" -- reading as a hang, the exact failure being fixed. Progress now initialises immediately after chunking (both totals are known there) and reports aphase:filtering->analyzing->finalizing, with the key cleared in the handler'sfinallyso response assembly is covered and an error path cannot leave a dead run reporting as live. Re-measured: progress is live from t+1.8s to t+67.0s, 97% of the run, and the operator sees "Filtering 1,250 chunks..." -> "Analyzed 5 ... 144 of 150 chunks in this pass (1,250 in article)..." -> "Building results..."._set_chunk_debug_phasewrites only to an existing key, sincehsetcreates missing keys and a late phase write would otherwise resurrect a zeroed record and report a finished analysis as freshly started. Progress lives in Redis, not process memory: the web service runs multiple uvicorn workers, so a poll can land on a different worker than the analysis and would otherwise always read zero. Every progress operation is best-effort -- a Redis outage degrades the display, never the analysis -- with a 300s TTL. One Redis client is shared across the run rather than opened per chunk. The existing Cancel button and "operation continues" dismissal note are unchanged. Covered by 17 tests intests/test_chunk_debug_utils.py. The helper-level ones pass just as happily with the reporting wrapped around the per-chunk loop only -- verified by re-introducing that exact regression, which left all of them green -- so four of them driveapi_chunk_debugitself and pin where the calls sit: progress live whilefilter_contentruns, still reporting while the response is assembled, and cleared on both the return and the error path. Four more intests/playwright/chunk_debug_modal_polish.spec.tsstub the progress endpoint and assert what the loading modal actually displays for each phase, rather than inferring it from the poll requests firing -- which is how the original attempt was mistakenly reported as working. -
The ML Model Performance panel refreshes with the threshold (2026-08-26): moving the Junk Filter Tuning slider re-fetched and correctly updated the KPI tiles, cost analysis, visualisation and chunk details, but left this panel showing the previous threshold's numbers. Measured on article 5772 going 0.7 -> 0.5, the panel kept reporting 30 matching / 83.3% / 6 mismatches while the payload said 36 / 100% / 0. The visible symptom was a self-contradiction: the panel claimed 6 mismatches while Show ML Mismatches -- reading
ml_mismatchfrom that same live payload -- revealed none. The four values were emitted once inside the modal'sinnerHTMLtemplate literal with noidattributes, soupdateChunkDebugResults()had no handle on them and never tried. Both render paths now write through a singleupdateMlPerformancePanel(); a second copy of the formatting is exactly how the two drifted apart, so the fix is one writer rather than two. The agreement colour band recomputes with the value (fullclassNamerewrite, so a panel going 100% -> 50% cannot carry both colours), and the header's "(based on N of M chunks)" caveat refreshes with the numbers it describes rather than captioning fresh figures with a stale sample. Verified live across 0.5 / 0.7 / 0.8 on article 5772 and covered by 5 tests intests/playwright/chunk_debug_modal_polish.spec.ts, each confirmed to fail against the unwired code. -
Settings: stored credentials are no longer returned to the browser, and an empty field no longer deletes one (2026-08-22):
GET /api/settings/{key}returnedsetting.valueverbatim andGET /api/settingsreturned every stored setting in one payload, so/settingsloaded full plaintext secrets intotype="password"inputs -- observed: a 164-char OpenAI key, a 93-char GitHub PAT, the Anthropic key and both Langfuse keys, all readable from devtools or any DOM dump._SENSITIVE_KEYSand_is_sensitive_setting()already existed in that module but were wired only to the audit path. Both read routes now returnvalue: nullplusconfigured/hintfor a matching key (value: nullrather than the hint string, so a caller that passessettings[key]straight to a provider fails its own truthiness check instead of shipping eight characters of a key),POST /api/settingsstops echoing the value back, andGET /api/settings/{key}setsCache-Control: no-storeto match its sibling. Masking makes every credential field load empty, which would have been catastrophic on its own:saveSettings()sentDELETE /api/settings/{key}for every empty field, with no way to tell "the user cleared this" from "the GET failed and blanked it" -- reproduced 2026-08-19 by stubbingfetchto reject forGITHUB_TOKEN, which took the field from 93 chars to 0 withshowNotificationnever firing. The two are fixed together: every managed field records whether its load succeeded, and Save reads that record instead of inferring intent from emptiness -- typed value writes, an explicit Clear (confirmed throughModalManager) deletes, a failed load touches nothing and says so, and an empty secret after a good load means "leave it alone". The bulk provider-key payload gets the same treatment; sending''for an untouched field would have overwritten the stored key just as destructively as the DELETE. Consumers that used to read keys out of/api/settingsnow let the server resolve them:sigma_queueenrich/validate and/api/test-{openai,anthropic}-keyfall back toworkflow_provider_options.resolve_provider_api_key(), and the GitHub connection test moved toPOST /api/settings/github/test(which also retires a standingconnect-srcReport-Only violation from the browser calling api.github.com with the PAT). -
Settings: unsaved changes are visible, partial saves report per section, and act-now buttons are distinguishable (2026-08-22): the page is ~5,500px tall with a single Save button at the bottom, no dirty-state indicator and no
beforeunloadguard, so whether anything was pending was unanswerable without scrolling and navigating away discarded edits silently. A sticky save bar now carries an unsaved-change count computed against a baseline captured once loading finishes, so editing a field back to its original value clears the flag rather than latching, and abeforeunloadguard fires only when something is actually pending.saveSettings()runs seven sections with no rollback between them, and reported failure as a bare list of section labels that never said what did apply; it now records applied/failed/skipped per section into a persistent summary under the save bar (the toast times out in 5s; a partial save is not something to read in five seconds). Interleaved with the deferred fields were buttons that fire immediately -- Create Backup Now, Apply/Disable Backup Schedule, Refresh Scheduled Jobs, Launch MkDocs server, Export Annotations CSV, Restore from Backup/File, and the four Test connection variants -- styled identically to inputs that wait for Save; all 18 now carry a warning-coloured left bar explained by a legend beside the save bar. -
/workflow#config: loading the page no longer saves the config back (2026-08-22): opening the config tab and touching nothing issued aPUT /api/workflow/config, which is whyagentic_workflow_configreached 8,152 rows with exactly one active (632 rows on 2026-08-19 alone). Three independent causes:OSDetectionAgent_selected_osis an array rebuilt on every read and the dirty check compared it with!==, so it always reported changed;SigmaEmbeddingModelis stored by the server but has no config panel, so form state read it asundefinedand always differed; and -- the one that fired three times per load --applyAgentConfigs()writes stored values into the form viasetAgentProvider(), which dispatches a realchangeevent to rebuild dependent UI, and that reached the user-edit handler and scheduled a save. Arrays are now deep-compared, form state carries forward everyagent_modelskey the server holds so the payload is complete on its own rather than depending on the backend's merge to repair it, and autosave refuses to schedule while stored config is being applied. A model select that reads back''because it cannot represent the stored value (ExtractAgentis stored as providerlmstudiowith an OpenAI model, so the option is absent from the list) is treated as the form having nothing to say rather than as the operator clearing the field --performAutoSavealready dropped empty model values, so calling that a change produced a write that could not express it. -
/workflow#config: provider changes no longer throw before finishing (2026-08-22):onAgentProviderChangereferenced anoptionsvariable that function never had (introduced in b482190), so every provider change -- including the ones the loader makes for each agent on page load -- raisedReferenceError: options is not definedand abandoned the rest of the handler: the temperature capability UI, the sub-agent commercial inputs, the control bindings and the inheritance hints. Found while instrumenting the phantom-save path on the same function. -
agentic_workflow_config.versionis unique and race-free (2026-08-22):models.pyhas always declaredversion = Column(..., unique=True), but this database predates that attribute so the index was never built -- and_next_workflow_config_version()allocated with an unguardedSELECT max(version) + 1. Two writers read the same maximum and both committed it: 164 version numbers were shared by two or more rows, the worst by five, while the UI presents version as the config's identity andconfigVersionSearchlooks configs up by it. Allocation now draws from a sequence, which cannot hand the same number to two callers however their transactions interleave; the advisory lock from 77506e6 stays as defence in depth, and the arithmetic remains as a fallback where the sequence is absent so this is safe to deploy ahead of the migration.scripts/migrate_workflow_config_version_unique.pyrenumbers the losers of each collision (the lowestidkeeps the number it has always been addressed by; later rows move above the maximum and record their original indescription), then creates the unique index and the sequence.idnever changes, sosigma_evaluations.workflow_config_idandsubagent_evaluations.workflow_config_idare untouched. Deploy note: the migration has not been run against the live database -- run it before relying on the constraint. -
/articles: RegexHuntScore filter buckets no longer strand articles in a dead zone between adjacent buckets (2026-08-21): the five score buckets used inclusive-inclusive bounds against non-shared boundary labels (0-19,20-39, ...,80-100), so the open interval between adjacent buckets -- e.g.19 < score < 20-- matched neither; 113 of 7,741 articles were unreachable through any bucket, andGood (60-79)+Excellent (80-100)undercountedHigh Quality (60+)by 17. Separately, an article with nothreat_hunting_scoreat all was defaulting to score0via.get(..., 0), incorrectly landing it in the bottom bucket. Fixed by relabelling the dropdown option values to share boundaries (0-20,20-40,40-60,60-80,80-100; visible labels unchanged) and extracting the bucket logic inpages.pyintoparse_threat_hunting_range()/score_in_threat_hunting_bucket()-- upper bound exclusive except the top bucket,None/missing score matches no bucket, malformed range now logged and ignored instead of silently swallowed by a bareexcept: pass. Verified against the live corpus: all five buckets now sum exactly to the loaded article count with no double-counting, andGood + Excellent == High Qualityexactly. Covered bytests/unit/test_threat_hunting_range_buckets.py(17 tests): every score in a swept set (including the old dead-zone boundaries) lands in exactly one bucket, boundary scores go to the higher bucket, 100 is inclusive in the top bucket,Nonematches nothing, and combined-bucket reconciliation holds across the whole domain. -
/articles: keyword chips expose the keyword as their accessible name, not the shared category hint (2026-08-21): every keyword chip carriedtitle="{{ hint }}"-- the category hint, identical for every chip in a group -- andtitlesupplies the accessible name, so a screen reader announced "Good discriminator" repeated across a row instead of the keywords distinguishing that article. The+Noverflow chip was worse: same generic hint, not interactive, so the keywords it hid were unreachable from the list page entirely. Fixed in thekeyword_chipsmacro (articles.html): removedtitlefrom the per-keyword chip so its accessible name is the visible keyword text, moved the category hint onto the (aria-hidden) color dot so it's still discoverable on hover, and added asr-onlygroup label per category for screen readers. The+Nchip now getstitle="{category} (hidden): kw1, kw2, ..."listing every hidden keyword. Covered by 2 new tests intests/ui/test_articles_advanced_ui.py: chip accessible name equals the keyword text (no title override), and the overflow chip's title lists the hidden keywords. -
/articles: concurrent toast notifications no longer occlude each other, and multi-line messages render their line breaks (2026-08-21):showNotification(base.html) appended every toast independently asfixed top-4 right-4, so concurrent toasts landed on top of each other -- only the last was legible until it timed out, and the toast covered the Settings nav link while visible. Multi-line error summaries (built byhandleAdhocUrlSubmit) also collapsed into one run-on line, since\nwas set viatextContentinto a container with nowhite-space: pre-line. Fixed by appending toasts to a flex-column#notificationStackcontainer positioned below the 70px nav bar, and addingwhite-space: pre-line. Message assignment staystextContent/role="alert"-- unchanged. Verified live: 3 concurrent toasts render at non-overlapping y-ranges, and an injected\nrenders as separate lines. -
/articles/{id}: restored annotation passages render at body type size instead of a 25% step-down (2026-08-21): restored huntability annotations were wrapped in a span carryingtext-xs font-medium(12px), sized for a short inline badge, while#article-contentistext-base(16px) -- so a long annotated passage stepped down in size at its start and back up at its end, mid-paragraph. The same oversized class string was used in both the live-selection highlight path and all three truncated-long-selection fallback strings inarticle_detail.html. Droppedtext-xs font-mediumfrom all four, letting annotations inherit body type while background/border still carry the highlight signal; also switched the click-delegation selector from a now-stale class-string match tospan[data-annotation-type], which is robust to the class change. Verified live: both annotations on a two-annotation test article now compute to the same 16px font-size as the surrounding body text. -
SIGMA Queue: the "Showing rules from job" banner no longer renders on every plain visit (2026-08-21): the deep-link filter banner (
#queueJobFilterBar) rendered on every visit to/workflow#queue, reading "Showing rules from job " with a trailing space and no ID, falsely implying the listed rules were a filtered subset. Root cause was CSS source order: a page-local.q-job-filter-bar { display: flex }rule was emitted after Tailwind's.hidden { display: none }in the page's<style>block, and both selectors share specificity (0,1,0), so the later rule won regardless of which class JS applied. Fixed by adding.q-job-filter-bar.hidden { display: none }(specificity 0,2,0), which wins independent of source order. The banner still renders correctly, with the right job ID, for a real?jobId=deep link, and CLEAR FILTER still clears it. Covered by a new test intests/playwright/sigma_queue_lifecycle.spec.ts. -
/workflow#config: 41px of horizontal page overflow at 375px fixed (three separate root causes) (2026-08-21): at a 375x812 viewport,documentElement.scrollWidthexceededclientWidthby 41px, pushing the Steps 2-5 collapse chevrons off-screen (no visible affordance that those sections expand) and clipping the tab bar's Agent Evals link mid-word. Three independent contributors, all fixed: (1).section-metaspans usedwhite-space: nowrapwith no shrink limit, so long agent-list text pushed section headers past the viewport -- givenoverflow: hidden; text-overflow: ellipsis; min-width: 0plus amax-width: 30vwat <=480px; (2) the tab bar (<nav class="-mb-px flex items-center justify-between">) laid out three tab buttons plus the Agent Evals link with no wrap and no shrink -- made horizontally scrollable in its own container instead of blowing out the page; (3) the actual root of the residual 41px:workflow-config-display.js's "Selected Models" list rendered items likeCmdAttnPreprocessor:Enabledas one unbroken token with no spaces, whichwhite-space: normalcannot wrap -- fixed by addingbreak-wordsto that<li>. Verified live at 375x812 after a hard reload: no scroll overflow, all six section chevrons on-screen and tappable, Agent Evals reachable within its own scrollable container. -
/workflow#configand/workflow#executions: SIGMA Queue rule IDs no longer truncate; STEP cells expose their full value on hover (2026-08-21): the Queue's ID column was ~7px too narrow for its own 4-character content (2097), truncating every rule ID in the table with notitleto recover the value by hover; the Executions STEP column had the same gap for any step not ingetStepBadge()'s known label map (e.g.context_length_check). Fixed by widening the ID column's default width from 60px to 76px (DEFAULT_Winworkflow.html; a user's own drag-resized width, persisted tolocalStorage, still takes precedence) and addingtitle="${escapeHtml(exec.current_step)}"to the STEP cell. Verified live: 5 different queue IDs render in full with no truncation and no new page overflow; STEP cells report their raw value (e.g. "generate_sigma") astitle. -
/workflow#config: 30 redundant/api/validate-modelrequests fired on every page load (2026-08-21):validateProviderModelCombination()fired its async server-side validation POST as a side effect whenever client-side validation passed, and it was called from 6 bulk/load-time loops that iterate every one of ~30 agents (applyProviderSelections,syncProviderVisibilityAndInputs,refreshAllProviderBlocks, the mainAgents provider-visibility loop, and both autosave pre-save validation passes) -- not only from a real user-driven change. Client-side validation is authoritative in those bulk passes; the async pass only matters for an actual edit. Added an opt-inskipAsyncoption threaded throughvalidateProviderModelCombination()/updateAgentProviderVisibility()and set at all 6 bulk call sites; the genuine single-agent change paths (onAgentProviderChange,validateAgentModelOnChange) are unchanged. Verified live: a hard reload now issues 0 requests (down from 30); an explicit single-agent validation call still fires exactly 1. Covered by 2 new tests intests/playwright/agent_config_validation.spec.ts. -
/workflow: auto-refresh polling cadence now follows the active tab instead of the tab active at page load (2026-08-21): the auto-refresh timer computed its polling period once, at script-load time, from whichever tab was active then (setInterval(..., currentTab === 'executions' ? 10000 : 30000)), while its callback re-checked the tab on every fire -- so switching tabs changed what was polled but never how often. Loading on#config/#queueand switching to Executions polled at 3x-stale 30s instead of 10s; the reverse polled Executions-cadence 10s against the Queue, 3x the intended DB load. Replaced the fixedsetIntervalwith a self-reschedulingsetTimeoutthat readscurrentTabfresh at each fire and re-arms with the matching delay, exposed aswindow.rearmAutoRefreshand called fromswitchTab()so cadence updates immediately on switch rather than waiting for the stale interval to fire first;clearTimeoutbefore each re-arm prevents stacking a second timer. Verified live by measuring real poll timestamps: 6 polls over 55s at ~11s spacing after switching to Executions, 2 polls at ~31s spacing after switching to the Queue, no duplicate/overlapping requests. Covered by 3 new tests intests/playwright/workflow_auto_refresh_cadence.spec.ts, including an anti-stacking check. -
Site footer no longer shows a hardcoded, increasingly stale copyright year (2026-08-21):
base.htmlhardcoded© 2025 Huntable CTI Studio, rendered on every page, already a year stale and set to silently go stale again each January. Added_current_year()tosrc/web/dependencies.py, registered as the Jinja globalcurrent_year, computed server-side per render. Verified on the dashboard,/articles, and/workflow. Covered bytests/unit/test_footer_copyright_year.py(2 tests). -
Settings page: unified disabled-button styling, password-toggle accessibility, backup button colors, status-tile clarity, and input precision/minimums (2026-08-21): a cluster of independent presentation and control-state issues on
/settings, fixed together. Disabled buttons looked live:applyBackupCronBtn/disableBackupCronBtnkept full-saturation brand fills withcursor: progress-- the busy cursor, implying the action was running, the opposite of "unavailable" -- while/sources' Collect Now used the correct desaturated-grey +not-allowedtreatment; the shared.settings-btn-*:disabledrule now matches that treatment app-wide, plus a tooltip on the cron buttons explaining why they're disabled. Icon-only password-reveal toggles had no accessible name: the 5 API-key visibility toggles had emptytextContent, noaria-label, noaria-pressed; all 5 now exposearia-label="Show/Hide API key"andaria-pressed, synced on click. Backup Actions used five unrelated button colors with the safest action (Restore from File) the second most prominent (orange); rationalized to neutral for reads, brand purple for the safe write, red reserved for both destructive restore paths -- neither restore path's confirmation gate was touched. Contradictory backup-cron status tiles (Availability: Unavailable/Managed Backups: Enabled) had their reconciling sentence buried below the fold in small amber text; promoted to a note directly under the tile grid. Documentation section ambiguity between a rawnohup mkdocs servecommand and a Launch button was resolved by making Launch the stated primary action and the command an explicit fallback, plus a copy button for the command. Threshold step mismatch:autoTriggerHuntScoreThresholdhadstep="1"against fractional hunt scores (50.3); changed tostep="0.1". Retention accepted 0:dailyRetention/weeklyRetention/monthlyRetentionall hadmin="0", silently allowing "keep nothing"; raised tomin="1". Covered by 8 new tests intests/playwright/settings.spec.ts. -
/ml-model-performance's browser tab title matches the page (2026-08-21):ml_hunt_comparison.htmlset{% block title %}ML vs Hunt Scoring Comparison - Huntable CTI Studio{% endblock %}, a leftover from an earlier purpose of the template, while the page's own<h1>reads "Model Performance" -- so the tab, bookmarks, and history all showed a name the page no longer uses. Title now reads "Model Performance - Huntable CTI Studio", matching the<h1>. Verified live in-browser. -
The Registry and Services extractor prompts pass validation again (2026-08-21):
RegistryExtractandServicesExtractwere the two agents reported as carrying one warning each by the validation pass above -- both flaggedRole/system missing expected token for VERIFICATION CHECKLIST (sec 12): "[ ]". The section had not been dropped, which is what the symptom suggests and what the task assumed: both prompts carry the checklist in full, with every item intact. What drifted is the- [ ]checkbox marker, which had degraded to plain-bullets, and_collectPromptIssuestests for the literal[ ]token, so a complete checklist rendered without checkboxes reads as a missing section. The fix is therefore a marker restore, not a section restore -- every changed line is exactly- X->- [ ] X, verified mechanically across all 14 files by diffing the role strings rather than trusting the patch. The drift was present in all 12 quickstart presets and bothsrc/promptsseeds simultaneously, so the presets were not the clean reference the task expected them to be; the authoritative text isdocs/contracts/registry-extract.mdanddocs/contracts/services-extract.md, which both still carry the section with- [ ]at the position the envelope splits role from instructions. The live active config was reconciled in the same pass with a targetedjsonb_seton the twoagent_promptskeys rather than a whole-config PUT, since that path read-modify-writes the entire blob and has clobbered sibling agents before; the other eight agents were verified byte-identical afterwards and only therolekey moved within the two targets. Verified in-browser on/workflow#config: all seven extractors report clean and no badge renders, confirmed against a positive control that degraded a prompt in memory and made the step badge appear. Covered bytests/config/test_shipped_extractor_prompt_contract.py, which validates all 12 presets x 7 extractors plus the 7 seeds against the whole validator contract rather than the one token that broke -- the same class of drift hit CmdlineExtract with seven missing tokens at once, so guarding only sec 12 would test the symptom. The token lists are parsed out ofprompt-editor.jsinstead of restated, so a token added to the validator is enforced here automatically; a non-vacuity guard pins the parsed lists, and it earned its keep by catching thatEXTRACT_SUB_AGENTSlives inworkflow.html, not the extracted JS modules. All 15 assertions were confirmed to fail against the pre-fix data in an isolated worktree. -
Subresource Integrity restored on every remaining third-party script (2026-08-21): the Tailwind CDN removal left three siblings behind.
hunt_metrics.htmlandscraper_metrics.htmlloadedhttps://cdn.jsdelivr.net/npm/chart.jsunversioned with nointegrity, andarticle_detail.htmlloaded html2pdf 0.10.1 with nointegrity-- unverified third-party JS executing in an authenticated operator session, the same supply-chain exposure the Tailwind change was meant to close.ml_hunt_comparison.htmlwas worse than unpinned: it paired an unversioned jsdelivr URL with a hash specific to Chart.js 4.5.1, so the integrity check would have started failing the moment upstream published a newlatest, silently blocking the script and leaving a half-working page with no server-side signal. All four now use the pinnedchart.js@4.5.1/dist/chart.umd.min.js(or the pinned html2pdf path) withintegrity,crossorigin, andreferrerpolicy, matching the one call site that was already correct. Verified in-browser: Chart.js 4.5.1 loads and all 5 hunt-metrics charts render, and html2pdf loads on article detail -- a wrong hash would have blocked them. Covered bytests/unit/test_security_headers.py, which fails on any external script lackingintegrityand on any integrity hash pinned against an unversioned URL. -
Stored XSS in the dashboard, hunt-metrics and article-detail chunk dialogs (2026-08-21): five inline
innerHTMLbuilders interpolated scraped, attacker-influenceable text into an HTML string without escaping. On/(dashboard.html:823-824,848-850) and/hunt-metrics(hunt_metrics.html:459-476,621-626) the value isarticle.title, copied verbatim out of JSON-LDheadline/<meta content>during ingestion, returned raw by the dashboard and analytics JSON APIs, and repainted on a timer -- so a poisoned title executes with zero clicks in an authenticated operator session, and the app ships no CSP to backstop it. On/articles/{id}the value is a verbatim slice ofarticles.contentreaching two modals:showRemovedChunksDialog(article_detail.html:5500, auto-opens after a filtered ranking run) anddisplayFeedbackComparisonModal(:10033-10034, both a body sink and atitle=""attribute sink, inserted viainsertAdjacentHTML). Because CSRF tokens here are client-minted request headers, injected JS can read one and drive admin-only endpoints as the operator. Every interpolation now routes through the shared globalescapeHtml()(static/js/utils.js, loaded for every page bybase.html); for the removed-chunks dialog the escape happens before\n -> <br>so intended line breaks survive while attacker markup does not. Escaping is applied uniformly rather than only to the obviously-tainted fields --escapeHtmlis a no-op on markup-free values (hex colours, fixed class strings, numbers) and is null/number-safe, so blanket escaping costs nothing and leaves no gap for the next field added to one of these widgets. Non-goal: no stored article content was sanitized or migrated; this is render-path only. Sibling of thehighlight_keywordsfix below, which closed the server-rendered arm of the same class of bug. Covered bytests/unit/test_template_escaping_contract.py(a scanner pinning every${...}inside an.innerHTMLtemplate literal on the two dashboard pages, plus a non-vacuity guard so a broken regex cannot pass for free) andtests/playwright/chunk_dialogs_xss_regression.spec.ts(2 tests driving both dialog renderers directly with a combined body/attribute-breakout payload: no live nodes, noon*attributes, payload visible as inert text, line breaks preserved, and no handler fires). -
The workflow Configuration tab is operable from the keyboard (2026-08-21): all six pipeline step headers on
/workflow#configwere<div>s carrying only an inlineonclick, with norole, notabindexand noaria-expanded, so no step section could be expanded without a mouse -- and the nested prompt accordions, which are keyboard-operable via the sharedinitCollapsiblePanels(base.html), sat inside sections a keyboard user could never open. The shared initializer was not reused, despite the task proposing it: these headers drive an accordion throughscrollToStep, where state lives in.step-section.open, exactly one section is open at a time, and opening also aligns the scroll container and the left rail.initCollapsiblePanelsexpresses none of that. The six headers instead gainedrole="button",tabindex="0",aria-controlsandaria-expanded, with matchingrole="region"/aria-labelledbyon each body, and one delegatedkeydownhandler inprompt-editor.jsmaps Enter/Space onto.section-header,.rail-itemand.sa-headerby callingclick()-- reusing the existing inline handlers rather than duplicating their logic.preventDefaultis required so Space does not scroll#config-contentout from under the sectionscrollToStepis about to align, which in turn required a nested-control guard so Enter on the sub-agent help buttons keeps its native activation instead of toggling the panel. The rail and sub-agent headers already advertisedrole="button"while ignoring Enter, so they are fixed in the same pass.aria-expandedis synced fromscrollToStep, the only live mutation path for.step-section.open. Focus is now visible on all three trigger types via an inset:focus-visiblering --.step-section,.sa-itemand.oc-railall clip overflow, so an outward ring would be invisible. Covered by 5 tests intests/ui/test_workflow_comprehensive_ui.py: button/region semantics across all six steps, a guard that the template's staticaria-expandedvalues match the seeded.openclasses, Enter and Space activation (asserting the accordion invariant, the aria sync and rail highlight), rail-item Enter, sub-agent-header Enter, and the help-button non-toggle case. -
Tailwind is built locally instead of loaded from a CDN at runtime (2026-08-21):
base.htmlloadedhttps://cdn.tailwindcss.com/3.4.17on every page -- a 407KB unverified third-party compiler executing in an authenticated operator session, with no CSP anywhere in the app to backstop it, and Tailwind's own "should not be used in production" warning firing on every page load. The obvious minimal fix, addingintegrity+crossoriginto match the Chart.js line below it, is not available: that host sends noaccess-control-allow-origin(verified against the live CDN), and SRI on a cross-origin script forces a CORS fetch, so the script would be blocked and every page would render unstyled. That exact change had already shipped on 2026-07-17 and was reverted three days later incff9e7ba, which addedtests/unit/test_base_template_external_resources.pyto stop it recurring -- the guard's stated rationale ("dynamic") was the wrong reason for the right rule, and is corrected here to "CORS-incompatible". Tailwind now compiles tosrc/web/static/css/tailwind.css(74KB minified, committed) viatailwind.config.jsandmake css, using the pinned standalone CLI throughnpxrather than an npm devDependency, so nonode_modulesenters the production image and thecheck-pinned-versionsCI gate is unaffected.darkMode: 'class'moved from the inline block inbase.htmlinto the build config. Two content-scanning traps, both silent:src/utils/keyword_resolution.pyholds 59 Tailwind class strings that reach the DOM throughrender_highlighted_content, and no default content glob scans Python -- omitting./src/**/*.pypurges all 59 while every existing test still passes, because the tests assert on class attributes in rendered HTML rather than on CSS. Four call sites also assemble class names from a colour fragment (article_detail.html:7616-7617,sigma_similarity_test.html:302-303), which no scanner can see; bothstatusColorvalues come from closed chains, so thesafelistpattern is exhaustive rather than precautionary. Verified across/ml-model-performance,/mlops/agent-evals,/mlops/agent-evals2,/articles,/settings,/workflow, and an article detail page (which carries the arbitrary-value and keyword-highlight concentration). No plugins are installed:proseclasses appear in ~10 templates but are inert today, and adding@tailwindcss/typographywould newly style pages this change is not meant to touch. Contract and command docs updated (docs/contracts/ui-designer.md,AGENTS.md); the old guidance to preserve the inlinetailwind.configblock would otherwise have told the next agent to restore something that no longer exists. -
Stored XSS in the article body closed; keyword highlighting restored on the affected articles (2026-08-20):
article_detail.html:602renders{{ article.content|highlight_keywords(...)|safe }}, andhighlight_keywords(src/web/utils/jinja_filters.py) short-circuited on the literal substring<span class=in the stored article text, returning that text verbatim into the|safeexpression -- so the entire article body reached the DOM unescaped. It did the same whenever keyword metadata was absent (metadata is None), a second raw path not identified when the issue was filed. The reachability is ordinary rather than exotic: scraped text is entity-decoded during ingestion, so HTML written as<span class="x">inside a code block in a CTI blog post lands inarticles.contentas literal<span class=. An attacker controlling any page reachable by the ingestion pipeline -- their own blog added as a source, a URL fed toPOST /api/scrape-url, a crafted PDF/OCR ingest -- could pair that substring with a payload that executes in an authenticated operator's session, and because the CSRF token is a client-minted request header, injected JS can read it and drive admin-only endpoints as that user. Measured on the live database at fix time: 21 stored articles carried the trigger, 5 of them alongside raw<img>/<script>tags. Both raw returns are removed; every non-empty input now flows throughrender_highlighted_content, which html-escapes each non-match segment and emits only its own generated highlight markup -- so the "avoid nested spans" concern the heuristic existed to serve is satisfied structurally rather than by a substring guess. The heuristic was also silently disabling keyword highlighting for exactly those 21 articles, which now highlight for the first time; that is a visible change on those pages, not only a security fix. PRE-EXISTING, not introduced by v7.8.0 -- deliberately deferred from the v7.8.0 release security review because it touches the rendering path of every article page and warranted real coverage. Non-goal: no stored article content was sanitized or migrated; this is render-path only. Covered bytests/unit/test_jinja_filters_xss.py(9 cases across the metadata shapes, verified to fail against the pre-fix filter), a rewrittentests/test_jinja_filters.pycase that had been asserting the vulnerable passthrough as correct behavior,tests/playwright/article_content_xss_regression.spec.ts(3 tests against a real rendered article: no live nodes, noon*attributes, raw response body inert, highlighting intact), andtests/unit/test_template_escaping_contract.py, which pins every Jinja autoescape bypass in the template tree -- both|safesinks,{% autoescape false %}, andMarkup()-- to a reviewed allowlist so a new unescaped sink cannot arrive unnoticed. -
Eval scoring no longer grades a no-output extraction as 100% precision (2026-08-20):
agent_evals2.htmlcomputed per-article precision asmatched / (matched + extra)with a: 1fallback, so an extractor that returned nothing was credited with precision 1.0. The server uses0.0for that degenerate case (evaluation_api.pyper-config aggregates andeval_item_scorer.py), so the SYS.03 summary strip and the server-computed SYS.04 trend chart reported different numbers for identical rows -- 64.4% against 51-55% on the CommandLine dataset. Worse than the discrepancy, the metric rewarded silence: an extractor emitting zero items for every article averaged 100% precision, precisely the failure an eval harness exists to catch. Measured on the live corpus at fix time, 223 of 911 scored rows (24%) hit the degenerate branch, includingwindows_servicesat 3/3 -- a displayed 100% while producing nothing at all. Three client sites disagreed on the same row: the table cell renderedn/a, the summary strip counted 1.0, and the item-detail modal displayed100% (0/0). All now route through a singlescoredArticleMetrics()helper that returns 0 for an empty denominator andnullfor a genuinely unscored row (no ground truth), the latter staying out of every aggregate. The aggregation was additionally extracted to a pureaggregateScoredMetrics()so the headline number is testable without a DOM, and so membership and divisor are decided by one list rather than a filter condition duplicated from the helper. The reported dataset now reads 54.4%, matching the server. Recall'sexpected_count == 0arm and theperfectRecallcounter follow the same convention. Non-goal: no storedsubagent_evaluationsrow or ground truth was changed; this is display/aggregation only, forward-only. Covered bytests/unit/test_evals2_item_scoring.py(11 tests, executing the template's JS under Node and pinning it against the Python convention case-by-case, including the live 10-row CommandLine dataset). -
PUT /api/workflow/configrefuses a config whose enabled extractors have no prompt (2026-08-19): the only gate was a dismissible browserconfirm()that swallowed its own errors, so a config could be saved with an enabled extractor carrying no prompt at all -- producing a run that completes with zero observables and zero rules and reports no error. The route now runs the same scanners server-side and refuses, withallow_prompt_warningsas the explicit "Save Anyway" override. Narrowed twice during implementation: a settings-only autosave is not a prompt edit, and a disabledRankAgent's empty prompt is not a runtime defect (the latter present on the live config). Separately, generated Sigma rules now carryobservable_attributioninrule_metadata-- an absentobservables_usedpreviously returned early, warned nothing, and left a failed tie-back byte-identical to a success; the key is stripped from the emitted YAML so it cannot reach pysigma. Covered bytests/api/test_workflow_config_prompt_validation_api.py,tests/unit/test_save_prompt_override_contract.py,tests/config/test_quickstart_preset_quality.py, andtests/workflows/test_agentic_workflow_helpers.py. -
Articles are no longer silently dropped when re-scraping a known URL (2026-08-19):
create_article's duplicate branch committed refreshed threat-hunting metadata and then handed the ORM row straight to_db_article_to_model. Withexpire_on_commit=Truethe commit expires every attribute, so reading.contentissued an implicit SELECT from sync context and raisedgreenlet_spawn has not been called;create_articleswallowed the error and returnedNone, dropping the article on every re-scrape of a known URL that carried a hunt score. The row is now refreshed after the commit, matching the created path andupdate_article.expire_on_commit=Falsewas rejected becauseAsyncSessionLocalis a single shared factory andupdate_articledocuments a dependency on the expiring behavior. Covered bytests/integration/test_async_manager_duplicate_article.py, which needs a real async session -- a mocked session never expires attributes and the failure is specific to SQLAlchemy's asyncio greenlet bridge. -
Diags page: escaped health-check errors, honest HTTP error detail, single batch overlay, visibility-gated auto-refresh (2026-08-20): six low-severity issues on
/diags(src/web/templates/diags.html) fixed together.updateOverallHealthStatuswas the one health-card updater still interpolatingdata.errorstraight intocontent.innerHTML-- every sibling updater already usedescapeHtml()/textContent-- so a health endpoint (or a caught-fetch error message reflecting response content) returning markup inerrorwould execute as DOM-XSS; now wrapped inescapeHtml().runHealthCheckswallowed non-2xx responses into a baredata.error || 'Unknown error', discarding the HTTP status; it now checksresponse.okand returnsHTTP <status>: <detail>sourced from the body'serror/detail/messagefield orstatusText. "Run All Health Checks" fired five sequential calls that each independently toggled the single#loadingOverlay, so the overlay could flicker between checks and the trigger button stayed clickable mid-run, letting a second click start an overlapping batch; the click handler now owns the overlay for the whole batch (update*Health()calls take asuppressOverlayflag) and disables/re-enables the button around the run. The 5s auto-refreshsetIntervalkept polling in a backgrounded tab; avisibilitychangelistener now clears it on hide and resumes (with an immediate refresh) on show, and the auto-refresh checkbox preference persists tolocalStorageacross reloads..diag-grid'sminmax(420px,1fr)track forced horizontal overflow on any viewport narrower than ~452px; changed tominmax(min(420px,100%),1fr). All eight health/job-history card content regions gainedaria-live="polite"so refreshed results are announced to assistive tech. Covered bytests/playwright/diags_health_check_regression.spec.ts(escaped-error-markup and single-overlay/button-disable regressions); the responsive-grid and aria-live changes were verified manually in-browser (420px viewport, screenshot, console-clean). -
UI test runs can no longer leave the live workflow config corrupted (2026-08-20): the Playwright suite runs against the live dev app on
:8001, so the 13 specs in theagent-configandworkflowprojects write the sameagentic_workflow_configrow the operator uses. Each restored its own mutation in atry/finally, but those restores run inside the test worker and most need a livepage-- so none of them fire when a worker is killed, the global timeout trips, or the browser context dies.expanded_prompt_editor_save.spec.tsseeds a 403-byte hermetic CmdlineExtract prompt (role: "Hermetic test seed...") as part of a legitimate save-regression test; an interrupted run left that stub as the active production prompt, where it sat undetected from 2026-08-17 to 2026-08-19 (config rows 7224 -> 7949) degrading the observable class the ground-truth work is centered on. Reproduced deterministically by SIGKILLing a run mid-spec. Fixed by moving the restore anchor to the process level rather than retrofitting 13 specs, since a deadpagedefeats any of them: a newglobalTeardown(tests/playwright/global-teardown.ts) restores a baseline in Node with no dependence on a surviving worker, andglobal-setup.tscaptures that baseline after healing known corruption shapes from the canonical quickstart preset -- so damage from a previous run that died before teardown is never laundered into the baseline and faithfully restored. The two layers bound damage to at most one run; only SIGKILL of the Playwright process itself escapes teardown, and that is what heal-on-next-setup covers.TEST_SEED_MARKERnow lives inworkflow-config-snapshot.tsand the spec builds its seed from it, so a seed the detector cannot recognize is structurally impossible. Two implementation defects were caught by running it and are fixed: the detector flaggedExtractAgentSettings(adisabled_agentssettings blob with nopromptkey) as damaged, and the baseline was first written undertest-results/-- Playwright'soutputDir, which it clears at run start, silently deleting the baseline between setup and teardown. The baseline now lives in git-ignored.playwright-state/. Healing is deliberately limited to CmdlineExtract from the canonical preset; other damaged prompts (currentlyRankAgent.prompt, empty) are reported every run but never auto-rewritten, since silently overwriting operator prompts is not a test harness's decision. Covered bytests/playwright/workflow_config_pollution_guard.spec.ts(17 tests: pollution detection, snapshot healing including the never-overwrite-a-healthy-prompt case, baseline persistence, post-run damage attribution, and an anti-drift guard failing any spec that hardcodes the seed literal). Verified across the fullworkflow+agent-configprojects: 132 passed, 6 skipped. -
Junk Filter Tuning modal: ten small display/labelling issues fixed (2026-08-22): the Content Reduction KPI tile bound to
reduction_percent(a chunk-count ratio) instead ofcontent_reduction_percent, overstating actual content removed by up to 41% relative (50.0% shown vs 35.4% actual on one test article) -- both were already computed server-side, only the wrong one was wired up. The threshold preset cards (0.5/0.7/0.8) were static, so after moving the slider the header readout and the highlighted card could disagree; a newsyncThresholdPresetCards()keeps them in lockstep and the slider'sonchangebecameoninputso the readout updates live while dragging, not just on release. A filter matching zero chunks (e.g. "Show ML Mismatches" on an article with none) left blank space indistinguishable from a rendering failure; it now shows an explicit "No chunks match the selected filter" message. Other fixes in the same pass: the ML Mismatch legend swatch was solid amber while the marker it represents is an outlined ring, now matched; the two adjacent "Confidence" labels (model probability vs. filter's own confidence field) read as a contradiction, now "Model confidence"/"Filter confidence"; 36 identical chunk headings and 72 identical feedback buttons carried no chunk number in their accessible name, now included; ML Model Performance / Chunk Visualization / Chunk Details showed sample-derived numbers with no indication they were a 12% sample during Partial Analysis, now noting "(based on N of M chunks)"; the 8 filter buttons had no pressed state or match count, now exposearia-pressedand a live(N)count per filter. Also demotes the nav brand lockup from<h1>to<div>and adds a skip-to-content link (base.html) so every page has exactly one<h1>, and themes theevalAccuracyChart/maeChartaxes for the dark UI via a new--chart-ticktoken (previously black-on-near-black gridlines and sub-AA tick contrast), with the Y-axis domain now padded around the actual data range instead of pinned 0-100 against 80-90 data. Covered by 8 new tests intests/playwright/chunk_debug_modal_polish.spec.ts, driving the modal's render functions directly with synthetic data; the Content Reduction fix was confirmed to fail against the pre-fix binding. -
GPT-4o cost-rate literals consolidated and corrected (2026-08-22): the Junk Filter Tuning Cost Analysis panel derived its savings estimate from
$5.00/$15.00per 1M input/output tokens, hardcoded as six separate float literals acrossllm_optimizer.pyplus one more indebug.py-- stale relative to OpenAI's current published pricing ($2.50/$10.00 per 1M, verified againstdevelopers.openai.com/api/docs/models/gpt-4o), free to drift apart from each other, and overstating displayed savings by roughly 2x. Consolidated into two module-level constants (GPT4O_INPUT_COST_PER_MILLION_TOKENS,GPT4O_OUTPUT_COST_PER_MILLION_TOKENS) that every call site now reads from, including a dead-code default parameter (calculate_filtered_costs) the original audit missed and a structural AST scan intests/unit/test_llm_optimizer_cost_rate.pycaught. Covered by 5 new tests pinning the corrected rates, proving the public cost-estimate API derives from them, and scanning both source files for a reintroduced stale literal. -
Duplicate
config-versions-modelsandmodel-versions/feedback-countfetches on page load (2026-08-22):/mlops/agent-evals'sloadAggregateScores()andloadPreviousResults()each independently fetched/api/evaluations/config-versions-modelsfor the same version set, issuing two identical requests per load;/ml-model-performance'sloadInitialData(),refreshRetrainingStatus(), andrefreshModelVersionHistory()each independently fetched/api/model/versionsand/api/model/feedback-count, up to four duplicate requests. Both pages now share one in-flight fetch across their callers (_fetchConfigVersionsModels()with a per-load cache;_fetchVersionsAndFeedback()passed as a shared promise), and the version-history panel reuses the preloaded list for its default view instead of also hitting the paginated endpoint for data it already has.config_versionsis an unbounded, unpaginated query-string param carrying the full id set; length-capped at 4000 chars server-side so an oversized request gets a clear 400 instead of failing opaquely at a proxy's header-size limit. Verified live: each page now issues exactly one request per endpoint per load (down from two and up to three respectively), no console errors, version history and KPI tiles still populate correctly. Covered by 2 new tests intests/api/test_subagent_eval_metrics.py. -
/articles: sorting by Annotation Count returned RegexHuntScore order, not annotation counts (2026-08-25):annotation_countwas one of two keys_apply_article_sort(src/database/statements.py) routed through the JSON-storedthreat_hunting_scoreas an approximation, on the stated reasoning that counts are attached post-query and therefore can't be sorted in SQL -- so the default no-filter listing and the source-filter fast path both ordered by hunt score while labelled "Annotation Count", and the corpus's most-annotated article (60 annotations, id 1426) never surfaced regardless of how far a user paged. The approximation was unnecessary:annotation_countis a plainCOUNT(article_annotations)grouped by article, so it's added as a correlated scalar subquery (_annotation_count_expr()) and sorted directly, with the same hunt-score tiebreak real columns already use. Only the SQL fast path needed the fix -- the Python-side fallback used when a search term or score-range filter is active already sorts on the correctly-populated post-query metadata, which is why those two filter states were reported as already correct. Verified live:sort_by=annotation_count&sort_order=descwith no filters now returns article 1426 first, matching a directORDER BY COUNT(article_annotations)query exactly (confirmed for both directions and against a source-filtered case). Covered bytests/integration/test_article_annotation_count_sort.py(both sort directions checked against an independent COUNT oracle on a real Postgres scratch DB) and two rewritten cases intests/unit/test_statement_builders.pythat had pinned the old approximation as correct. -
SIGMA fallback checkbox copy no longer implies full-article content replaces extracted observables (2026-08-25): the
/workflow#confighelper text for "Use Full Article Content (Minus Junk)" described the flag as swapping the extracted-artifact summary out for the full article, with observables "still included ... either way" -- language left over from an earlier version of the fallback path. It now describes what the flag does today: SIGMA reviews the junk-filtered article alongside the Sigma-eligible extracted observables for the current detection category, using the article to ground rules in the extracted evidence and surface additional relevant behavior the extractors didn't capture.tests/unit/test_workflow_sigma_fallback_copy.pyupdated to pin the corrected copy. -
Article detail page: doubled URL-fragment handler and annotation-text console leak (2026-08-28): every
/articles/{id}load registered two near-identicalwindow.addEventListener('load', ...)blocks that each parsedlocation.hashand auto-opened the SIGMA/IOCs/ranking modals -- one via a namedprocessUrlHash()(also reused by thehashchangelistener), the other a dead second copy with a staledata.metadata.sigma_rulesfield lookup that never matched the real API shape. Every load logged "Setting up URL fragment handler..." and "Page loaded, checking URL fragments..." twice. Separately, restoring each stored annotation on load calledaddUserClassification()/highlightTextAtPosition(), which logged the first 100 characters of the annotated article text plus its length/offsets to the console on every restore -- a content leak, not just noise. Deleted the dead duplicate block and removed the text/offset-echoingconsole.logcalls; annotations still restore correctly and exactly once (verified live: 2/2 annotations on article 5772). Covered bytests/unit/test_article_detail_fragment_handler.py. -
target="_blank"links now consistently carryrel="noopener noreferrer"(2026-08-28): 5 of 7 such links acrosssrc/web/templates/(article_detail.htmlx2,observable_training.html,settings.htmlx2) set norelat all, and the other 2 (articles.html,sources.html) setrel="noopener"withoutnoreferrer. Modern browsers implynoopenerfortarget="_blank"by default so exploitability was already limited, but the inconsistency was a silent regression waiting on a future browser/extension change. All 7 now carry the full value; guarded bytests/unit/test_target_blank_rel_noopener.py, which scans raw template text (not just the parsed DOM) because theobservable_training.htmlanchor is built inside a JS template literal. -
Articles list: "Copy article content" silently failed off
127.0.0.1(2026-08-28):copyArticleContent()callednavigator.clipboard.writeText()with no fallback and one opaque failure message for every cause. That works on127.0.0.1(a secure context in a normal browser) but fails wherevernavigator.clipboardis unavailable -- a plainhttp://LAN address, or this repo's own Playwright/automation environment, wherewindow.isSecureContextisfalseeven at127.0.0.1. Added adocument.execCommand('copy')fallback with a cause-specific error when that also fails, verified end-to-end live (the automation environment's actual non-secure context exercised the real fallback path, not just a mock). Covered bytests/playwright/articles_copy_clipboard.spec.ts. -
Model Performance page: HTTP errors were indistinguishable from empty results (2026-08-28): 11 of
ml_hunt_comparison.html's 12fetch()call sites went straight fromawait fetch(...)toawait resp.json()with noresp.okcheck. A FastAPI error body ({"detail": ...}) has nosuccesskey, so a 500 on/api/model/versionsrendered "No model versions recorded yet." in muted grey -- indistinguishable from a genuinely empty result -- while the stat tile above still read the last-good count, and "Refresh Status" during a full backend outage looked like a successful refresh with stale tiles left unmarked. Addedif (!resp.ok) throw ...before every.json()so failures route into the existing (previously dead-for-HTTP-errors)catchblocks; a genuine 200-with-zero-rows result still renders the plain empty state. Separately,executeRollback()'s error path readresult.detaildirectly, and a FastAPI 422 puts an array of{msg, ...}objects there, so a validation failure rendered "Rollback failed: [object Object]"; addedformatApiErrorDetail()to extract the actual message. All four behaviors regression-tested against the pre-fix revision (each failing test reproduces the original defect, including the literal[object Object]string) intests/playwright/ml_hunt_comparison_error_handling.spec.ts. -
Junk Filter Tuning: a kept chunk's reason no longer reads as though it was removed (2026-08-25): the modal's per-chunk
reasonfield came straight fromContentFilter.filter_content()'s own reason string, which describes a whole-article filter pass rather than one chunk's decision -- so every kept chunk read "Content filtered successfully" (indistinguishable from "this chunk was filtered out") and every removed chunk read the equally uninformative "No huntable content found," naming no signal either way.src/web/routes/debug.pynow builds the reason from the chunk's actual keep/remove outcome plus the top entry of thefeature_contributionmap already computed for the ML Details panel (e.g. "Kept - cmdline artifact count was the strongest signal"), falling back to a plain decision-only reason when the model exposes no feature importances. Verified live on article 5772: kept chunks read "Kept - ...", removed chunks read "Not kept - confidence stayed below threshold (...)". Covered by 5 new tests intests/test_chunk_debug_utils.py. -
Article detail page: duplicate "Article Content" heading removed from the page's heading outline (2026-08-28): the Article Content panel's
<h2>and the annotation toolbar's inner<h3>both read "Article Content," rendering ~50px apart -- redundant visually and a duplicate entry for screen-reader users navigating by heading. The inner label is a toolbar caption, not an independent section, so it was demoted from<h3>to a plain<div>with identical classes/text; visual layout is unchanged. Verified live on/articles/5772: exactly one<h1-6>element reads "Article Content" (was 2). Covered bytests/ui/test_ui_flows.py::test_article_detail_has_single_article_content_heading, confirmed to fail against the pre-fix template before passing against the fix. -
404 pages no longer send operators to check database connectivity for a simple bad ID or URL (2026-08-28): a nonexistent article ID (
/articles/99999999) and any unmatched route both rendered the generic "Something went wrong" title plus "please check your database connection and ensure all services are running properly" -- correct advice for a genuine 5xx, actively misleading for a routine 404.error.htmlnow accepts optionaltitleandshow_db_hintcontext vars (defaults unchanged, so every existing 500/422 caller is unaffected); the article-detail 404 insrc/web/routes/pages.pyand the globalnot_found_handlerinsrc/web/modern_main.pynow passtitle="Article Not Found"/"Page Not Found"withshow_db_hint=False. Verified live: both 404 paths show the correct title with no database advice;/articles/abc(422) and a simulated genuine 500 both still show "Something went wrong" plus the hint, unchanged. Covered by 2 new API tests intests/api/test_endpoints.py, 1 new assertion intests/api/test_article_list_db_error_surfaces.pyproving the hint survives on a real 500, and 2 pre-existing UI tests intests/ui/test_ui_flows.pyupdated for the new titles. -
Articles list: the
#<id>badge no longer visually collides with a wrapped title on mobile (2026-08-28): at 390px, a long unbroken title token (e.g. a real article titledPHANTOMPULSE...) read as a single run-on with its adjacent#<id>badge (PHANTOMPULSE#4790). The mobile media query setswhite-space: normalso titles wrap instead of truncating, but a single word has no space to break on, so it overflowed past its flex-item box withoverflow: visiblepainting that overflow straight into the sibling badge -- the two elements' layout boxes never actually overlapped, only the painted text did. Addedoverflow-wrap: break-word; word-break: break-wordto the same mobile-only.article-row__linkrule so long tokens break mid-word instead of overflowing; desktop's single-line truncation is unchanged. Covered bytests/ui/test_ui_flows.py::test_mobile_title_does_not_overflow_into_id_badge, confirmed to fail against the pre-fix template (scrollWidth 468px vs. clientWidth 110px) before passing against the fix. -
Article detail page: keyword category "points" now read as a per-match weight, not points the article earned (2026-08-28): the Keyword Matches panel showed cards like "Negative Indicators · -10 pts · No matches" --
-10 ptsis a static per-category weight (src/utils/keyword_resolution.py), rendered in the exact position a reader expects this article's earned score, on an article that scored 99.9 despite the category contributing nothing. All 5points_labelconstants (75 pts,5 pts,10 ptsx2,-10 pts) now read "... pts each"; no template change needed sincearticle_detail.htmlalready renderspoints_labelverbatim. Verified live on/articles/5772. Covered by 2 new unit tests intests/test_keyword_resolution.py, one exercising the actual renderedpanel_groupscontext rather than just the underlying constant. -
Sigma similarity engine:
not (not x)no longer wipes out a rule's entire atom set (2026-08-31): insigma_atom_similarity/sigma_similarity/dnf_normalizer.py,_distribute_to_dnf'sNotNodebranch handled anAtomNode,AndNode, orOrNodechild but let aNotNodechild fall through toreturn []. BecauseAndNodedistribution is a cross-product, one empty operand emptied the whole branch list, so a condition likeselection and not (not filter1)produced an empty DNF:extract_positive_atomsreturned an empty set andsurface_score_from_dnfraisedUnsupportedSigmaFeatureError("DNF has no branches (malformed or empty rule)")out ofcompare_rules. The failure was loud rather than silent -- the novelty layer routed the rule toneeds_review, never to a truncated atom set -- but the message misdiagnosed a well-formed rule as malformed.NOT(NOT(x))now collapses toxbefore any otherNOThandling, matching the double-negation unwrapping theNOT(AND(...))andNOT(OR(...))sub-cases already did; odd nesting depth collapses to a singleNOT. No backfill was required: all 3,780 rules in the live corpus were checked and none contain a nestedNotNode, sosigma recompute-atomsis a no-op for the current corpus. Covered by 5 new tests intests/sigma_atom_similarity/test_dnf_and_containment.py, each confirmed to fail against the pre-fix normalizer. The public replicadfirtnt/SIGMASimTest, which carries this engine verbatim and documents the old behavior as a known limitation, needs the same change.