v7.9.0 "Europa"

Added

• Publisher-authored Sigma passthrough with enforceable provenance (2026-09-24): a deterministic, LLM-free scanner now finds complete Sigma mappings directly in articles.content, preserves the exact source substring and records article/source IDs, SHA-256, extraction offsets, license evidence and attribution. Source rules bypass generation and batch deduplication, receive reviewer-only similarity context, and remain immutable; edits happen on a generated copy. The queue adds a first-class local_review_only state and prominent provenance UI. Server-side single/bulk approval and the final PR preflight fail closed unless the destination repository explicitly allowlists the source license or a reviewer records source-specific permission; mixed PR batches are rejected before any Git/GitHub side effect. The schema migration and existing-article backfill are explicit, dry-run-by-default operator steps rather than startup mutations.
• Catalog-driven model parameter compatibility and per-agent reasoning effort (2026-09-04): the per-agent Temperature / Top_P sliders rendered for every provider and model, so an operator could move a control that the OpenAI reasoning path dropped server-side and that the Codex adapter never sent -- and the lever that matters on current models, reasoning effort, was not configurable at all. Compatibility was a hardcoded name-prefix list that misclassified new families silently and did not know that Anthropic Opus 4.7+/4.8/5, Sonnet 5 and Fable 5 reject temperature outright (the Anthropic client sent it unconditionally). A new config/model_capabilities.json (keyed by model id: supports_temperature, supports_top_p, effort_levels, default_effort; tiers verified against the OpenAI per-model pages and the Anthropic effort doc on 2026-09-04) is now the source of truth, resolved by get_model_capabilities(provider, model) in provider_model_catalog.py; model_supports_variable_temperature() is a thin wrapper over it and the old prefix heuristics survive only as the fallback for unlisted ids, which get "sampling as before, no effort control" instead of an error. The v2 config contract gains an optional per-agent Effort (flat key {Agent}_effort, None = provider default; the twelve quickstart presets and the live DB row validate unchanged), the migrator lifts it, and PUT /api/workflow/config rejects a tier the resolved model does not list -- Codex tiers are live and have shipped names outside the documented vocabulary (ultra), so the contract pins the token shape and membership is checked against the catalog. llm_client.py sends reasoning_effort (OpenAI) or output_config.effort (Anthropic) only when configured and listed, omits temperature for models that reject it, and records the field in the forensic _provider_payload; LLMService resolves {Agent}_effort (extractors inherit ExtractAgent_effort) so existing call sites needed no signature changes; the Codex adapter keeps supportedReasoningEfforts / defaultReasoningEffort from model/list via list_model_details() and passes the configured tier as the turn/start.effort override. The OpenAI temperature-rejected retry now logs CAPABILITY DRIFT naming the model whenever the file claimed support, the production alarm for a stale value. GET /api/workflow/provider-options returns model_capabilities per provider, and the Agents page disables (never hides) an unsupported slider with a "Not supported by this model" note and renders an Effort select listing exactly the model's tiers after "Provider default", hidden when the model has none; the autosave carries {Agent}_effort and the server clears the override on the blank option. The daily update_provider_model_catalogs job diffs fetched ids against the capabilities file, prints a ⚠️ per gap, and the Celery task logs a WARNING and returns unclassified_models so the gap shows in the Scheduled Jobs UI; the refresh-model-context-windows skill now verifies existing entries' tier values, the part no API exposes. Docs: docs/llm/cloud-model-reference.md (new capabilities section, corrected Anthropic sampling claim), docs/reference/api.md, the scheduled-jobs report. Tests: tests/unit/test_model_capabilities_resolver.py, tests/config/test_workflow_config_effort.py, tests/unit/test_workflow_config_effort_validation.py, tests/services/test_llm_client_effort.py, tests/worker/test_update_provider_model_catalogs_task.py, additions to the Codex client, provider-options and catalog-script suites, and tests/playwright/agent_config_effort_capabilities.spec.ts.
• Eval runs can be launched and polled over MCP (2026-09-03): the MCP server could read eval evidence and save diagnoses but could not start a run, so the edit-prompt -> run -> diagnose loop still needed the operator on the Agent Evals page. The launcher behind POST /api/evaluations/run-subagent-eval now lives in src/services/subagent_eval_launch_service.py: plan_subagent_eval resolves the active config, canonical subagent, committed URL set, server-side replicates, fixtures, DB rows and the extractor's provider and model, and writes nothing; launch_subagent_eval performs the writes, commit and staggered Celery dispatch the route always performed, with initiated_by in the snapshot extras (hash-excluded) and the audit payload. The route is a thin wrapper with an unchanged path and response shape, its dead use_active_config field is dropped and ignored on input, and the response's eval_record_id is now populated instead of null. A new MAX_EVAL_EXECUTIONS_PER_LAUNCH cap (default 100) is enforced before any write from both callers. Two MCP tools land on top. run_subagent_eval is a caller-attested write: with confirmed_by_user false it returns the plan with a billing line and writes nothing, it launches only on a fresh confirmed_by_user=true per call, checks the broker before writing, reports fixture-less and row-less URLs as skipped rather than running an extractor inside the MCP process, and audits evaluation.run_requested as service:mcp. get_subagent_eval_status is a read-only poll keyed by run label (v5139, v5139a) that returns the same progress and metrics as the HTTP status endpoint without an eval record id. The shared SUBAGENT_TO_EXTRACT_AGENT map in subagent_utils.py replaces three private CamelCase copies. docs/reference/mcp-tools.md no longer says that no MCP eval path bills tokens: diagnosis still spends nothing, launches bill the extractor's configured provider unless it is lmstudio. The live end-to-end launch also exposed that a desktop client runs the MCP server on the host with no broker URL, where Celery's config falls back to the compose-internal redis hostname; stdio_server.py now defaults REDIS_URL to redis://localhost:6379/0 for such runs, the same way it already assembles DATABASE_URL, so the broker preflight passes instead of failing closed.
• CI checks became CI gates (2026-09-02): eleven workflows ran on every push and PR, and none was required -- main had enforce_admins set with zero required status checks, europa-dev had no protection at all, and tests.yml skips push events on main, so a direct push to main ran no tests and nothing stopped it. Branch protection on main now requires all fifteen existing check contexts (the six test tiers, Security Audit, Ruff, pinned versions, UX contract, dead code, mypy, CodeQL, Semgrep, secret scanning, MkDocs build) with strict up-to-date enforcement, admins included, force-push and deletion blocked. Four new gates land with this change. (1) Lockfile Up To Date runs uv lock --check: every other job installs with --frozen, which trusts uv.lock as-is and never compares it to pyproject.toml, so a stale lock -- easy to produce, since a bare uv run re-locks silently -- passed CI. (2) Paired Surfaces (scripts/check_paired_surfaces.py, pull requests only) fails when one side of a documented pair moves without the other: src/database/models.py without a scripts/migrate_*.py (migrations are never auto-applied, so the runtime falls back silently and the schema change ships inert), and src/prompts/ or workflow_config_schema.py without config/presets/ (the runtime reads the presets, not the prompt files). Escape labels no-migration-needed and no-preset-sync-needed cover the genuine exceptions; PR labels and the base ref are passed through env:, not interpolated into the shell. (3) Every committed Sigma rule must parse (tests/unit/test_committed_sigma_parses.py): the standalone fixture rules, and any complete literal example rule fenced in docs/contracts/*.md (today the contracts hold only output templates and fragments, which are skipped -- the scan exists so a literal example added later is checked automatically), go through the same SigmaCollection.from_yaml + condition parse the runtime applies, with a non-empty-corpus guard on the fixtures so the gate cannot go vacuous if their paths move. Motivated by a rule carrying the nonexistent |any modifier reaching the Huntable-SIGMA-Rules repo through PR #5 -- parse-level only, deliberately: a 2026-09-02 measurement of all 838 queue rules found zero that parse but fail to compile on Splunk, ES Lucene, ES|QL, or Kusto, so backend conversion was not added. (4) Coverage ratchet for src.web, src.workflows, and src.core, which had no floor at all (the existing gates cover src.services and src.utils only); floors are set at measured coverage less a small margin and are meant only to rise. The release scripts had to change with this, because they owned main's protection: release_lock.sh wrote required_status_checks: null with lock_branch: true, and release_unlock.sh deleted protection outright -- so the release merge, the one PR into main the checks exist to gate, ran with none, and the next lock would have erased the required list. Both scripts now apply a single committed payload, scripts/main_branch_protection.json, toggling only lock_branch; unlock keeps every required check and the force-push/deletion blocks in force, with a --allow-force-push flag reserved for the documented rollback path. tests/unit/test_main_branch_protection_contexts.py pins every listed context to a job name that still exists in .github/workflows and every PR-eligible job to the list, because GitHub matches required checks by context string and a renamed job silently un-gates itself -- or, on a locked-then-unlocked main, blocks every PR waiting for a check that no longer reports. The committed list carries 18 contexts (the 15 live ones plus Dependency Review, Lockfile Up To Date, and Paired Surfaces); the live protection picks them up the next time release_unlock.sh runs. europa-dev gets force-push/deletion protection without required checks, since direct pushes to the integration branch are the working model.
• Real MITRE ATT&CK validation for Sigma attack.* tags (2026-09-02): the Huntable policy pass in sigma_validator.py only checked that tags were non-empty strings with an allowed character set, so a generated rule could cite a technique ID that does not exist, or one MITRE revoked years ago (attack.t1086 for PowerShell), and sail through. Every attack.* tag is now checked against a cached ATT&CK taxonomy: an unknown, revoked, or deprecated technique, sub-technique, group, software, mitigation, or campaign ID is a validation error with an actionable message (revoked IDs name their replacement, e.g. "replaced by T1059.001 (PowerShell) -- use 'attack.t1059.001'"; an unknown sub-technique names its existing parent), so the Phase 3 repair loop receives a concrete fix rather than silently accepting the tag. Unknown tactic names are warnings only, because ATT&CK v19 renamed Enterprise defense-evasion (now stealth / defense-impairment) while SigmaHQ and the Sigma spec still use the old name everywhere -- hard-failing that would reject correct community-style rules. The taxonomy lives in config/attack_taxonomy.json (2,535 objects across the Enterprise, Mobile, and ICS domains, ATT&CK v19.2) and is regenerated by the new scripts/build_attack_taxonomy.py, which reuses the STIX fetch from build_attack_platform_map.py and is run by the operator when MITRE ships a release (roughly twice a year), matching the existing platform-map cadence; it is deliberately not a worker job because the output is a committed config artifact. A missing or corrupt taxonomy file disables the check with one logged warning instead of failing every rule. Covered by tests/services/test_attack_taxonomy.py, tests/scripts/test_build_attack_taxonomy.py, and a new TestAttackTagTaxonomyValidation class in tests/test_sigma_validator.py that drives both SigmaValidator.validate_rule and the pySigma-backed validate_sigma_rule entry point with valid, unknown, revoked, and deprecated IDs.

Changed

• Docs drift fixes from the 2026-09-25 doc true-up (2026-09-25): docs/llm/cloud-model-reference.md gains the claude-fable-5-1 row (added to the catalog 2026-09-02) and records the 128K max output for the Claude 5 models, verified against platform.claude.com, which clears that TODO marker. docs/development/ui-test-tiers.md now lists WF-CFG-PHANTOM-003 in the quarantine Playwright project instead of calling it empty. docs/development/testing.md drops the claim that CI reports quarantine counts, since no workflow does. mkdocs.yml adds four 2026-08 solutions/ records to not_in_nav, matching their siblings.
• HuntQueriesExtract now has an explicit source-authority boundary (2026-09-24): LLM-captured query text remains available for analyst display, traceability, and evaluation, but every HuntQueries observable is marked content_fidelity: llm_best_effort and source_text_authoritative: false. Captured items with type: sigma are now always display-only, even if future metadata supplies an otherwise eligible logsource; non-Sigma backend queries with explicit target telemetry retain their existing generation path. The contract now enumerates every text consumer and requires the planned source_provided importer to scan articles.content directly for exact rule bytes rather than trusting LLM transcription.
• GitHub PR submission no longer persists PATs in Git remote configuration (2026-09-24): SigmaPRService previously rewrote the customer rules repository's HTTPS origin to include x-access-token:<PAT>, leaving the credential in .git/config after every submission and across token rotation. HTTPS fetch, pull, and push operations now receive the PAT only through an invocation-scoped GIT_ASKPASS environment whose temporary script contains no secret. Before network operations, legacy credential-bearing HTTPS origins are rewritten to the same clean repository URL; SSH origins are untouched. Focused tests use temporary Git repositories to prove clean HTTPS remains clean, embedded credentials are removed, SSH remains unchanged, and neither the command arguments nor askpass script contain the PAT.
• Backup freshness is visible in Settings (2026-09-03): the backup status API now exposes the newest backup timestamp, age, and a three-day stale threshold, and the Settings backup panel shows a distinct warning when the latest system backup is stale. The backup schedule and retention policy are unchanged.
• Cloud model reference data is current and catalog-sourced (2026-09-03): verified the OpenAI and Anthropic model output limits against their live documentation, dated the OpenAI source capture, and added o1-pro and o3-pro to the application model catalogs so their documented context windows are sourced consistently.
• Quickstart presets gain the four short rules they were missing from the standard (2026-09-03): the first preset pass applied only a four-item delta, so every variant still lacked the every-tactic rule, |windash, OriginalFileName, and the instruction boundary for article text (the LMStudio pair also lacked field order and the title rule). Because a preset's system text is the whole standard on that path, those gaps were live for every preset user. Each variant gained about 1,300 characters in its own structure (LMStudio 5.9k -> 7.2k, gpt-5/codex 8.0k -> 9.1k, main variants 10.0k -> 11.2k). The per-logsource field-name table was deliberately left out: it is 1,300 characters on its own, and the SigmaHQ validator layer now fails a wrong field name with the field named, which the repair prompt then fixes. The system message is not subject to the LMStudio user-prompt cap, so local runs are unaffected by the growth.
• The in-app Sigma validator now enforces the rules repository's blocking SigmaHQ validator set (2026-09-03): validate_sigma_rule ran pySigma parsing plus the Huntable policy pass, neither of which checks field names, logsource membership, custom top-level keys, or the required id/status/level/description fields, so a rule with url|contains or ServerName under network_connection, category: proxy with product: windows, or service: sysmon without an EventID parsed cleanly, was approved in the queue, and failed the blocking CI in Huntable-SIGMA-Rules. pySigma-validators-sigmahq==0.21.0 (the version the rules repo pins; compatible with the app's pysigma 1.3.3) is now a dependency, and src/services/sigma_validation_blocking.yml mirrors the repository's .sigma/validation-blocking.yml (36 validators). After pySigma accepts a rule the set runs over it; every issue becomes an error prefixed SigmaHQ <Issue> (so the repair prompt sees it), the result carries metadata["sigmahq"], and a rule is only valid when the list is empty. Three details matter: the pipeline's own grounding keys (observables_used, platform, observable_attribution, ...; now one shared SIGMA_GROUNDING_METADATA_FIELDS used by the validator, the generation service and the workflow) never trip sigmahq_unknown_field, because the pipeline strips them before publication; validator instances are rebuilt per call, because duplicate_title and identifier_uniqueness remember every rule they have seen and a shared instance reported collisions between unrelated calls; and each validator runs in isolation, so one that crashes (pySigma 1.3.3's sigmahq logsource checks raise on list-of-map selections under service:) is reported as a HIGH issue instead of aborting or passing the rule. A missing plugin disables the whole layer loudly (metadata["sigmahq"]["available"] == False) rather than narrowing it. The Huntable policy pass also stopped rejecting list-of-map selections (selection_img: as - Image|endswith: / - OriginalFileName:), a valid Sigma form that the generation prompt's own example uses. Test fixtures that predated the standard (no id/status/level, network_connection without a product) were updated; the repair prompt's error map gained entries for the SigmaHQ issue names. The live images need a rebuild for the new package; until then the layer reports itself unavailable and validation behaves as before.
• Sigma generation gives non-reasoning models a 4,000-token completion budget (2026-09-03): _call_provider_for_sigma capped every non-reasoning provider (LMStudio, Anthropic, local models) at max_tokens=800. A complete rule is roughly 250-400 tokens, so any article with more than two detectable behaviors came back truncated with finish_reason=length and the tail rules were lost or half-written. The budgets are now the named constants SIGMA_MAX_TOKENS_REASONING (10,000, unchanged) and SIGMA_MAX_TOKENS_STANDARD (4,000), with a regression test for the standard path. A new unit test (tests/unit/test_sigma_prompt_files.py) also pins each Sigma prompt file to the exact kwargs its caller formats it with, checks that a single format pass leaves no doubled braces, and fails if the bootstrap seed sigma_generation.txt drifts from sigma_generate_multi.txt.
• Sigma generate, validate and enrich prompts share one rule standard (2026-09-03): the three prompt surfaces that shape what reaches Huntable-SIGMA-Rules disagreed with each other and with the rules repository's CI. Findings and evidence are in docs/development/sigma-prompt-audit-2026-09-03.md; the headline defects were (1) the generation prompt's network-abstraction section taught field names that do not exist in the SigmaHQ taxonomy (url, UserAgent, HttpMethod, ServerName, ALPN), which pySigma and the in-app validator accept and the repository's blocking sigmahq_invalid_fieldname / sigmahq_unknown_field validators reject, so such a rule could be approved in the queue and never merge; (2) underscored ATT&CK tactic tags (attack.defense_evasion) in the contract, the presets and 8 of the 18 published rules, which pySigma's ATT&CK validator flags; (3) the queue validate route's first-attempt prompt was an inline f-string with its own weaker rules, and both it and the repair prompt handed the model only the first 500 characters of the rule to fix, so the model repaired a fragment and invented the rest; (4) the enrichment prompt escaped its JSON schema for two format passes but is formatted once, so the model saw {{ ... }}, and the modal's default system prompt duplicated the file's directive block with a contradictory d5 (author) rule while the default instruction asked for "more comprehensive conditions" that directive d4 forbids. src/prompts/sigma_generate_multi.txt (and its byte-identical bootstrap seed sigma_generation.txt) now carries the full standard -- SigmaHQ field names per logsource, category-first logsource with service only alongside an EventID, hyphenated tactics with every tactic a technique belongs to, |windash and OriginalFileName resilience, ISO date, title/description/level/false-positive conventions, a grounded example in canonical field order, and an instruction boundary for article text. The new sigma_validate_single.txt (first attempt of POST /api/sigma-queue/{id}/validate), the rewritten sigma_repair_single.txt, and the restructured sigma_enrichment.txt mirror that standard instead of introducing rules of their own; the validate and enrich system prompts are now short constants in sigma_queue.py (SIGMA_QUEUE_VALIDATION_SYSTEM_PROMPT, DEFAULT_SIGMA_ENRICHMENT_SYSTEM_PROMPT, mirrored in queue.js), the retry preview cap is 8,000 characters in both the route and SigmaGenerationService._repair_rules, _sigma_rule_date() emits YYYY-MM-DD, and the 12 quickstart presets had their tactic underscores and date: format corrected inside the SigmaAgent prompt. The example rules embedded in the new prompts pass the rules repository's blocking validator set (pinned pysigma 1.5.0 / pySigma-validators-sigmahq 0.21.0). The three preset variants' network-abstraction block (about 12,000 characters of url/UserAgent/HttpMethod/ServerName/ALPN examples per preset) was then replaced with a 2,500-character block that uses only SigmaHQ proxy, dns_query, network_connection and firewall fields, and each preset's user scaffold now passes {author} and {date} so the model no longer has to guess them; that matters because with a preset applied the preset text is the whole standard and the prompt file is never read. Still not done: the in-app SigmaValidator still has no field-name or custom-key check, so queue validation remains advisory where the repository CI is blocking. The contract-guardian pass over this change also corrected the contract's description of the preset path (a quickstart preset's own system text is the entire rule standard and its user scaffold is the user message, so the prompt file is not used on that path and preset drift is live drift), replaced the stale 150-line copy of the old enrichment system prompt in workflow.html with the short constant, and switched the rule export in src/web/routes/ai.py from YYYY/MM/DD to ISO dates so exported rules match what generation now emits.

Removed

• Legacy manual /generate-sigma endpoint, and the starvation it left behind (2026-09-03): POST /api/articles/{article_id}/generate-sigma (383 lines in src/web/routes/ai.py) had no caller anywhere in src/web/ -- no template, no JS, no dynamically built path -- and returned its rules to the caller instead of queueing them, so anything it generated billed a provider run and could never be reviewed. It was already half-removed: the 2026-08-10 dormant-subsystem cleanup dropped article_sigma_matches and named this endpoint as that table's sole writer, but left the generator standing. It was also the only writer of article_metadata["sigma_rules"], which starved its two remaining readers. POST /api/articles/{article_id}/sigma-matches -- still called from the article page -- read that key and therefore answered "No generated SIGMA rules found. Please generate rules first." for every article the workflow had actually produced rules for; it now sources this article's rows from sigma_rule_queue (rejected excluded, oldest first, unparseable YAML skipped with a warning) and assesses them for real. Verified against live data: article 481 yields its 7 queued rules with intact logsource and detection. The #sigma URL-fragment modal in article_detail.html, the other reader, is still metadata-backed and therefore still dead; it has no entry point in the UI either, and is tracked separately. Note for whoever picks that up: sigma_rules was empty (0 rows) when this landed, so novelty assessment returned 0 candidates regardless of source; the corpus is populated now (3,757 rules, fully embedded) -- see the git/runtime-os fix below for why ./setup.sh had left it empty.
• Dead observables-mode "mark-reviewed" surface removed (2026-09-04): the observables annotation mode has been disabled since 2026-01 (its page ships with the feature flag off), leaving a half-implementation behind. POST /api/articles/{article_id}/mark-reviewed in src/web/routes/articles.py drove an agent-writable article.reviewed flag that no UI ever saved, and had zero callers; the same mode left a commented-out "mark observables reviewed" modal in article_detail.html with an orphaned markObservablesReviewed() JS function. Removed the route, the mark_article_reviewed MCP tool and its tool-list entry in src/huntable_mcp/tools/articles.py, the ACTION_ARTICLE_REVIEWED audit constant in src/services/audit_service.py, the route's classification in route_manifest.py, and the dead template block plus orphaned JS. docs/reference/mcp-tools.md renumbered its tool table from 37 to 36 entries. Regression guards in tests/api/test_removed_dead_endpoints.py and tests/unit/test_mcp_write_tools.py prove both the endpoint and the MCP tool stay gone.
• Remaining observables-mode machinery and its training subsystem removed (2026-09-05): the previous day's cleanup removed the mode's "mark-reviewed" surface but left the mode itself and an entire dormant training pipeline standing. In article_detail.html, OBSERVABLE_MODE and its constants, the #article-content-plain surface and its applyContentVisibility()/updateModeUI() branching, the [data-annotation-mode] CSS gate, showObservableModal(), validateAndExtractObservableSelection(), submitObservableAnnotation(), and the three mobile touch-handler branches that called them are gone; setAnnotationMode() now only accepts huntability, and huntability's selection, modal, and mobile-touch paths are unchanged and re-verified in the browser. Deleted the subsystem this fed: src/services/observable_training.py, src/services/model_training.py (its only caller, shelling out to a Workshop/ training script tree that does not exist in this repo), src/web/routes/observable_training.py (GET /api/observables/training/summary, POST /api/observables/training/run), src/worker/tasks/observable_training.py and its explicit import in celery_app.py, src/web/templates/observable_training.html, and the /observables-training page route -- all four now 404. route_manifest.py drops both /api/observables/training/* entries plus a phantom /api/observables/evaluation/* entry orphaned since the 2026-08-10 observable_evaluation removal (docs/features/observable-evaluation.md updated to match: its "still-live" training section is now a removal note). The live DB had zero CMD/PROC_LINEAGE annotations and zero calls to either removed endpoint, confirming neither had a real caller. Eight tests/worker/test_*.py files carried a src.worker.tasks.observable_training sys.modules mock that predated this removal and are now trimmed to what still exists; tests/unit/test_target_blank_rel_noopener.py drops its observable_training.html exemption and lowers its known-links floor from 7 to 6.
• Unreferenced mobile annotation manager removed (2026-09-16): src/web/static/js/annotation-manager-mobile.js (1,103 lines) still carried OBSERVABLE_MODE, PROC_LINEAGE and the observables touch handlers after the 2026-09-05 cleanup, and no template or script loaded it (the article page ships annotation-manager.js and mobile-simple.js; this is a desktop-only tool). Deleted, and the three mobile-only cases in tests/playwright/annotation_xss_regression.spec.ts went with it; the desktop escapeHtml regressions stay.

Fixed

• Generated Sigma rules are no longer attributed to the article's publisher (2026-09-17): the SigmaAgent "AUTHOR PRESERVATION" instruction told the model to copy the author of a complete Sigma rule whenever the article itself published one. It keyed on what the article contained, not on whether the generated rule reproduced anything, so every newly written rule from a detection publisher was eligible to come back as author: The Hunters Ledger -- measured 2026-09-04 on queue #14, #15, #16 and #21, while sibling rules from the same execution said Huntable CTI Studio. The carve-out is removed from all four prompt sinks: both seed prompt files (still byte-identical), the ten quickstart presets that carried it (the two LMStudio presets never did), and the live active workflow config, saved through PUT /api/workflow/config/prompts as v66 with only that sentence changed (v65 had it; 14 older config versions still do). Because those older versions and any model can still emit a copied author, SigmaGenerationService now stamps SIGMA_RULE_AUTHOR on every generated rule after carrying the optional Sigma fields. Two generation-service tests pinned the old behaviour (a copied publisher author surviving, and no author when the model omitted one) and now assert the stamped author; new tests/unit/test_sigma_author_attribution_contract.py fails if any seed prompt or preset reintroduces the carve-out. docs/contracts/sigma-generate.md updated. Publisher-authored rules keep their attribution only through the planned source_provided import path.
• Intra-batch Sigma dedup compares matched values, not condition syntax (2026-09-17): _detection_leaf_values tried to drop Sigma condition expressions with a four-string denylist (selection, condition, all of them, any of them) instead of skipping the condition key, so real conditions such as selection_image and selection_command were counted as detection values and inflated the union of every rule pair. Execution 25 (article 989) queued #19 and #21 -- the same wscript.exe-from-Windows-Temp detection from two generation groups -- at 0.50 overlap against a 0.8 threshold. The collector now skips the reserved condition and timeframe keys at the top of the detection block. That alone lifts #19/#21 only to 0.75 (#19 adds OriginalFileName), so the duplicate test (_batch_rules_are_duplicates) gains a containment arm: Jaccard >= 0.8, or the smaller value set fully contained in the larger with at least 3 values and Jaccard >= 0.6. Measured before choosing it: of the live queue's same-execution pairs it catches all three true duplicates that share tokens (#2/#7 and #23/#26, which the condition leak had also hidden at 0.67, plus #19/#21) and keeps the distinct #9/#10; across 1,187,088 SigmaHQ same-logsource pairs of distinct published rules it flags 94 versus 77 for Jaccard >= 0.8 alone, 135 for Jaccard >= 0.7 and 411 for pure containment. Near-duplicates spelled with different tokens (#24/#27 urlcache vs -urlcache) remain separate by design. docs/architecture/workflow-data-flow.md described the old rule (and a detection.selection-only scope the code never had) and is updated. New tests/workflows/test_batch_rule_dedup.py reproduces #19/#21 from the live YAML; its correctness cases fail against the previous code.
• Submitted Sigma queue rules keep their YAML, stop matching themselves, and show their real title (2026-09-16): three defects surfaced on queue rows #962 and #984. (1) After a rule's PR merged and the customer repository synced, the corpus held its byte-identical cust-<yaml id> copy, so re-scoring the queued rule reported a 1.0 match against itself and hid genuine duplicates. Candidate retrieval (SigmaNoveltyService.retrieve_candidates) now skips that row on the exact-hash shortcut, the canonical-class path and the logsource-key fallback; customer_copy_rule_id() derives the id, and the queue's list, similar-rules and compare-similarity endpoints now pass the YAML id into scoring (they previously rebuilt the rule without it). Workflow generation was unaffected: a new rule has no customer copy yet. (2) rule_yaml was rewritten days after submitted_at: in view mode the queue's Validate and Similar Rules buttons re-save the modal text whenever it differs from the stored YAML, and no endpoint checked submission state. _apply_rule_yaml now guards the approve, reject and PUT /api/sigma-queue/{queue_id}/yaml paths: a YAML change on a row with pr_submitted or submitted_at returns 409 and nothing is committed (identical YAML stays a no-op); the reject path applies it outside its broad JSON-body fallback so the 409 is not swallowed. The queue's apply-validated and apply-enriched actions now show the server's reason instead of a generic error. (3) YAML edits never updated rule_metadata.title, so the queue header kept the generated title; every accepted edit now refreshes it from the new YAML when it parses with a title. Also corrected docs/features/sigma-rules.md: customer rules use the file_path prefix cust/, not customer/. New tests/unit/test_sigma_queue_submitted_rule_integrity.py and tests/integration/test_sigma_self_match_exclusion.py (real test database) fail against the previous behavior.
• A manual trigger no longer discards a staggered eval run's pending execution (2026-09-16): the eval launcher commits every execution row up front and schedules their Celery tasks with increasing countdowns (up to EVAL_STAGGER_SECONDS + a 60 s throttle per step), so the tail of a large launch sits pending with a live task well past five minutes. The trigger eligibility check treated any such row as stuck, marked it failed and started a fresh execution; when the eval row's task then arrived, the worker's atomic claim (below) found it no longer pending and skipped it, so the eval data point was lost behind a misleading "stuck in pending" error. is_stuck_pending_execution now gives rows whose snapshot carries eval_run a window of the 5-minute base plus the latest countdown the launcher can schedule (eval_pending_stuck_after(), read from MAX_EVAL_EXECUTIONS_PER_LAUNCH at call time: about 1 h 44 min at the default cap). Inside that window the trigger is blocked with the existing "already has an active workflow execution" reason and POST /api/workflow/executions/trigger-stuck counts the row as skipped; a row that outlives it is still recovered by both paths, and the retention reaper still fails anything idle past RETENTION_STALE_EXECUTION_HOURS. Non-eval rows keep the 5-minute rule. Ingest auto-triggers are not exposed: they dispatch only for newly inserted articles, and eval rows point at pre-existing fixture articles. The trigger-stuck response wording now describes a start window rather than a fixed age. New tests in tests/unit/test_trigger_stuck_dispatch.py, tests/services/test_workflow_trigger_service.py and tests/integration/test_execution_claim.py fail against the previous predicate; the integration case reproduces the loss end to end on the test database.
• The workflow worker claims its execution row atomically (2026-09-16): run_workflow logged the existing execution and then wrote status = "running" unconditionally, so two Celery tasks carrying one execution_id -- a trigger-stuck re-dispatch racing the original, a retry of a merely delayed row, worker prefetch across a restart -- both ran the graph and both wrote results into the same row (measured 2026-09-04 on execution 19: two fork-pool workers, one saw pending, the next saw running and proceeded; with 2 x 2 worker concurrency up to four can collide, and on a non-junk article that is double provider spend). The blind write is now a conditional UPDATE ... WHERE id = :id AND status = 'pending' with a rowcount check: exactly one caller moves the row to running; every other caller logs a skip and returns {success: false, skipped: true} without invoking the graph or touching the row. Gating on pending is safe because the two other status = "running" writes in the module (os_detection and rank_article nodes) re-affirm a row the function already owns, nothing resumes a running row, and restart-orphaned running rows are still failed by recover_orphaned_running_execution before a retrigger creates a new row. STUCK_PENDING_AFTER (5 min) stays as a dispatch heuristic for POST /api/workflow/executions/trigger-stuck and as the trigger path's definition of a lost row; the decision and its reasoning (the staggered eval tail can sit pending for minutes with a live task) are recorded next to the constant. New tests/integration/test_execution_claim.py runs two threads with separate sessions into one pending row against the test database and asserts exactly one graph invocation.
• Two web routes that imported workflow code no longer fail at runtime (2026-09-04): the web image's venv is built with uv sync --frozen --no-default-groups (Dockerfile stage builder-web), so langgraph is not installed in cti_web -- and two routes imported src.workflows.agentic_workflow anyway, raising ModuleNotFoundError on the first request. POST /api/workflow/executions/trigger-stuck was also running the LangGraph pipeline in-process; it now re-dispatches each pending execution onto Celery via trigger_agentic_workflow.delay, so the worker runs the graph, and it audits each re-dispatch. summarize_rule_novelty moved out of agentic_workflow into src/services/sigma_novelty_service.py so the sigma-queue routes can reach it without pulling in the workflow dependency group; agentic_workflow re-exports it, so its own callers are unchanged. Behavior of the novelty classification itself is unchanged -- inconclusive still yields max_similarity=None rather than a confident 0.0.
• Sigma sync, indexing and embeddings work again on a fresh setup (2026-09-04): the role-specific runtime split (f9a236b3) moved every service off the monolithic image, and the shared runtime-os base it introduced did not install git. sigma_sync_service shells out to git clone/git pull for the SigmaHQ corpus and sigma_pr_service drives the customer rules repo, so sigma sync failed inside the cli container on every setup after that commit -- and because startup_sigma_sync_and_index gates metadata indexing and embedding generation behind the sync's exit status, both were skipped. The call's 2>/dev/null discarded the real error, leaving only "Sigma sync failed (run manually...)", and the suggested manual command failed identically. A completed ./setup.sh therefore left data/sigma-repo empty, zero rows in sigma_rules, and no semantic search, rule similarity or MCP Sigma retrieval; Sigma PR submission from the web container was broken by the same missing binary. git is now installed in runtime-os (one shared layer for all seven role targets) and in the Dockerfile.prod runtime stage. _startup_sigma_cli replaces the three suppressions: stderr is appended to logs/sigma_index.log and its tail printed on failure, while stdout stays on the terminal so the long embedding job still shows progress. startup_ensure_runtime_directories now also creates test-results and data/diagnoses, two gitignored bind-mount sources whose absence on a fresh clone makes docker compose up fail with an OCI mount error. Contract tests pin git in both runtime images and require every relative bind-mount source to be tracked or scaffolded.
• Codex subscription login uses the device-auth flow, and the status message says what's actually wrong (2026-09-03): docker compose exec workflow_worker codex login starts a browser-callback listener bound inside the container, which a host browser cannot reach, so the documented login command hung with no explanation. .env.example, both Codex docs pages, and the Settings panel now point at codex login --device-auth, which prints a URL and code to open in the operator's own browser; the Settings panel shows the command in a copy-able block instead of hiding it behind an "ask an administrator" line. POST /api/settings/codex/test (test_codex_subscription) stopped collapsing every failure into that same "ask an administrator" message: a logged-out app-server ({"account": null}) now reports "Codex is not logged in. Run: ..."; a login on the wrong auth mode reports "...not with a ChatGPT subscription. Run: ..."; an unreachable app-server keeps its own message -- each names the runnable command. o1-pro and o3-pro were also dropped back out of config/provider_model_catalog.json: the project's OpenAI allowlist (filter_openai_models_project_allowlist) already excludes both from every Settings/Workflow dropdown, so listing them in the raw catalog was inert.
• Restore snapshots, CLI collection, workflow recovery, and Dev2 initialization are safer (2026-09-03): --force now skips only restore confirmation; both database restore paths create a pre-restore snapshot unless the operator explicitly passes --no-snapshot, including uploaded backups through the maintenance API. The lightweight CLI can now start collection without Playwright installed; a JavaScript-rendered source reports a clear instruction to use the Playwright-enabled ingestion worker instead of failing during import. A workflow row left running by a worker restart is marked failed on the next trigger attempt only after 30 minutes without an updated_at change, so a healthy slow run is not interrupted and a forced retrigger can proceed. Dev2 now mounts the standard init-scripts directory rather than a missing init.sql path; its deliberately limited service boundary is documented.
• Sigma generation no longer truncates the prompt for cloud providers, and trims article content rather than instructions for local ones (2026-09-03): generate_sigma_rules applied the LMStudio context-window cap whenever the caller passed ai_model="lmstudio", and the workflow passes exactly that as a placeholder meaning "use the configured provider". So every workflow run on Codex, OpenAI or Anthropic cut the user prompt at 8,000-12,000 characters (Truncating prompt from 21850 to 12000 chars on every run in the worker log), and because the article sits at the top of the template the cut removed the tail of the standard: level calibration, false positives, metadata, output format, the example and the final check. Execution 3900 showed the effect once the persona bug stopped masking it: no --- separators, no author, date, references or false positives, technique-only tags. The cap now keys on the resolved provider (_resolve_sigma_provider, shared with the multi-rule call), and when a real local model needs trimming the article content is shortened and the prompt rebuilt ([Article content truncated ...]), so the instructions always survive. Known limit: the 13.5k template alone exceeds the local budgets (8,000-12,000 characters), so on LMStudio the content is trimmed to almost nothing; either raise the caps for modern local models or shorten the template.
• A raw-text SigmaAgent record is the user template, not the persona (2026-09-03): parse_sigma_agent_prompt_data treated any plain-text prompt saved next to a sibling model key as a system persona, and that is exactly the shape reset-to-defaults and bootstrap write from sigma_generation.txt ({"prompt": <file>, "instructions": "", "model": "Not configured"}). The runtime therefore sent the whole 13.5k template unformatted as the system message, with literal {author} / {date} / {title} placeholders, and then the file again, formatted, as the user message. Execution 3899 showed the symptom: three of four rules came back with author: '{author}'. Now the placeholder check decides: text with at least one {identifier} is the user template (system falls back to DEFAULT_SIGMA_SYSTEM_PROMPT), text without placeholders is the persona, regardless of the model key. Every run on a raw-text record before this, including today's 3895, 3898 and 3899, was on the doubled prompt.
• Out-of-class Sigma rules are re-homed to the group whose logsource they match instead of being dropped (2026-09-03): every generation group's call sees the whole article, so the model regularly emits a rule that belongs to a sibling group. _rule_logsource_matches_group treated that as an error and dropped the rule; on execution 3898 the network_connection call produced the article's best detection (a wscript.exe -> node.exe process_creation rule) and it was discarded while the process_creation group, whose observables did not include that command line, never saw it. _find_rehome_group now keeps the emitting group when the logsource fits, otherwise moves the rule to the first sibling group (never the full-content fallback) whose logsource hint matches, and drops it only when no group can own it. observables_used are still rebased against the emitting group (the indices are positions in that group's observable list); attribution, platform, telemetry and sigma_generation_group come from the home group, which records rehomed_from. Group summaries gain rehomed_rules and dropped_rules no longer counts re-homed rules. Regression tests mirror execution 3898.
• Queued Sigma YAML no longer leaks generation_phase, and is written in SigmaHQ field order (2026-09-03): promote_to_queue kept a third private copy of the pipeline-key list, and that copy lacked generation_phase, so every queued rule carried it as a top-level key; the PR service commits queue YAML verbatim, so the rules repo's blocking custom_attributes / sigmahq_unknown_field check would have rejected each one (the in-app SigmaHQ layer showed it first: all 12 rules from the day's two runs failed on exactly that key). Promotion now uses the shared SIGMA_GROUNDING_METADATA_FIELDS (which gains generation_phase) through canonical_sigma_rule_dict, which also orders the remaining keys title, id, status, description, references, author, date, tags, logsource, detection, falsepositives, level instead of the service's construction order.
• Generation prompt no longer credits rules to the article's publisher (2026-09-03): the first live run on the new prompt (execution 3898, Codex gpt-5.6-terra) set author: Microsoft Security Blog - Defender on rules from an article that contains no Sigma rule at all; the model read the author-preservation clause ("if the article contains a verbatim Sigma rule with a non-blank author") as "credit the source". The clause in sigma_generate_multi.txt, its seed, and the 10 preset variants that carry it now says the supplied author is used unless the article itself contains a complete Sigma rule in YAML with its own author, and that the article's author, publisher, blog or vendor name is never the rule author. The live SigmaAgent record was re-seeded.
• The Workflow Config effective-prompt preview shows the SigmaAgent messages the backend really sends (2026-09-03): config.js carried a hand-copied LOCKED_SIGMA_USER_TEMPLATE (the pre-audit sigma_generate_multi.txt) and always showed it for SigmaAgent, and _effReadCurrentSystem fell back to the user text when no persona was saved, so a raw-text SigmaAgent record was rendered as both the system and the user message and a preset's own user scaffold was never shown. New GET /api/workflow/config/prompts/defaults/{agent_name} serves the on-disk user template (sigma_generate_multi.txt, stripped the way PromptLoader does) plus DEFAULT_SIGMA_SYSTEM_PROMPT; the preview fetches it once on first use, shows the DB user template when the record carries one (preset / raw-text, badge "editable · from DB") and the file otherwise ("LOCKED · code-owned"), shows the code default as the system message with a "default · code-owned" badge when no persona is saved, and documents the {author} / {date} placeholders. The old JS mirror is gone, so the preview cannot drift from the file again.
• Evals2 item scorer reads RegistryExtract's real key/value_name fields (2026-09-03): item_candidates() for registry_artifacts read registry_hive/registry_key_path/registry_value_name, which RegistryExtract has not emitted since the 2026-05-12 schema change, so every registry item was silently dropped from matching and rows showed matched_count=0 with a passing actual_count -- a false green checkmark on the Agent Evals table despite correct extraction. Found via the first real mcp_agent eval diagnosis (execution 3866). Covered by updated cases in tests/services/test_eval_item_scorer.py and tests/api/test_subagent_eval_rescore.py.
• Generated Sigma rules keep author, date, references, and falsepositives (2026-09-02): SigmaGenerationService.generate_sigma_rules rebuilt every validated rule into a dict with a fixed key list (title, description, id, tags, level, status, logsource, detection) and the review queue dumps its YAML from that dict, so those four standard fields were silently dropped from every rule on every source. The effect was invisible until the Hunter's Ledger detections pages were ingested: the workflow trace for execution 3865 shows the model emitted an author: line on all 12 raw rules (7 attempting the publisher's name from the embedded rules, 5 with the template default Huntable CTI Studio), yet all 4 queued rules carried none, so the SigmaGenerate AUTHOR PRESERVATION directive could never reach the queue. The rebuilt dict now carries author, date, modified, references, falsepositives, and fields when the model emitted them (SIGMA_OPTIONAL_RULE_FIELDS), coerced to JSON-safe values: yaml.safe_load turns an ISO date: 2026-08-03 into a Python date, which the JSONB sigma_rules column cannot store (execution 3889 failed on exactly that during verification), so dates become ISO strings and other scalars become strings. Queue promotion already keeps every non-grounding key, so these appear in rule_yaml with no further change.
• Eval diagnosis UI now shows run signals, the help button says what it does, and the skill copies match (2026-09-02): the v2 diagnosis schema requires run_signals and the huntable-eval-diagnosis skill checks them first, but renderDiagnosisPanel in agent_evals.html never rendered them, so truncation, context pressure, contract compliance, finish reason, and token utilization were invisible in the Agent Evals execution modal. They now render as a Run Signals row ahead of root causes with the bad values highlighted (truncation detected, finish_reason of length/error, high context pressure, violated/partial compliance, utilization at or above 80%), and the "What it returns" table in docs/features/agent-evals.md lists the field. The execution-modal button was labeled Diagnose via MCP but only opened a help modal; it is now How to Diagnose, and the help copy is rewritten for the operator (the prompt first, with a Copy button, then what the agent does and when the one approved save happens) with the file-path pointers folded into a maintainers-only disclosure. The saved-diagnosis views (GET .../evals/{id}/diagnosis, .../diagnoses, and the MCP list_eval_diagnoses / export_diagnosed_eval_bundles loaders) now share one load_saved_diagnoses helper in eval_diagnosis_service.py that orders runs by the persisted created_at stamp instead of file mtime, so a restore or copy of data/diagnoses/ no longer reorders run history; the MCP loaders report _source_file as a bare filename instead of an absolute host path; and get_eval_diagnosis_context reports execution_id / article_id as integers to match the save and list tools. .agents/skills/huntable-eval-diagnosis/SKILL.md (the canonical copy) had drifted to a shorter version than its .claude/skills/ compatibility copy and is synced to the fuller text. Two stale doc lines fixed: docs/guides/authentication.md still listed "LLM-powered bundle diagnosis" as an audited HTTP route, and docs/brand-style-guide.md still described source healing as automatic repair. Covered by tests/unit/test_eval_diagnosis_run_signals_render.py (template structure), a created_at-over-mtime ordering test in tests/api/test_eval_diagnosis_api.py, a bare-filename test in tests/unit/test_mcp_eval_tools.py, and an id-coercion test in tests/services/test_eval_diagnosis_service.py. Also removed the MLOps Control Center's "System Online" pulse indicator and its four-cell stats bar (mlops.html) -- all four cells were hardcoded (3 / 3, Ready, Monitoring, --), never wired to a real status source, and unrelated to this fix beyond sharing the file.
• The Hunter's Ledger source now ingests its /hunting-detections/ pages, and discovery-path articles keep their author (2026-09-02): the source only ingested /reports/ posts from the Atom feed, but the publisher's validated Sigma rules live on 56 /hunting-detections/<slug>-detections/ pages the feed never lists (a sample page carries 18 rules, every one with author: The Hunters Ledger; the matching report page carries none), so the SigmaGenerate AUTHOR PRESERVATION directive never fired on this source. Widening post_url_regex alone does nothing because the fetcher returns after a successful RSS tier unless archive_pages/max_archive_pages are set, and with no discovery.strategies URL discovery falls back to the homepage, which links to none of them (the site has no sitemap). The hunters_ledger stanza in config/sources.yaml now widens the regex, enables the archive supplement, and adds a single-page listing strategy over the detections index with a[href^="/hunting-detections/"][href!="/hunting-detections/"] (verified against the live index: 56 matches, no pagination). Index links omit the trailing slash while the page canonical includes it; the stored canonical_url comes from JSON-LD with the slash, and both the fetcher's RSS/archive merge and the processor's URL dedup strip trailing slashes, so no slash/no-slash duplicates are created. Two supporting fixes: ModernScraper._extract_article dropped authors, tags, and summary -- it extracted them from JSON-LD and selectors and then built the ArticleCreate without them, so every discovery-path article on every source was stored with no author; they are now carried through. And because the detections pages carry no author signal anywhere (JSON-LD BlogPosting without author, no meta author), the stanza adds meta[property='og:site_name']::attr(content) as a trailing author_selectors fallback so those articles match the feed-path attribution instead of being blank; report pages still get the person-level author from their TechArticle block. The live sources.id=42 row was updated with a targeted JSON merge rather than sync-sources, which rewrites the whole config and would have clobbered the operator's custom robots.user_agent. Known limit, unchanged by this work: the archive supplement re-fetches up to 50 discovered pages on every check (the _use_conditional flag is unused), which is the same behavior every archive_pages source already has.
• ContentCleaner keeps indentation inside <pre> blocks, and the processor no longer re-cleans already-clean text (2026-09-02): html_to_text folded block boundaries correctly, but normalize_whitespace_keep_newlines then left-trimmed every line, so an indented Sigma rule or code sample published in <pre> came out flush-left and stopped being parseable YAML (on The Hunter's Ledger detections pages all 18 author: The Hunters Ledger lines survived while selection: landed at column 0 under detection:). Downstream, the SigmaGenerate AUTHOR PRESERVATION directive assumes a verbatim rule in the content, so the model had to re-serialize instead of reproduce. html_to_text now stashes each <pre> block verbatim before normalization and restores it afterwards; block-level descendants (<br>, per-line <div>) still break lines, tabs and highlighter <span> wrappers are preserved, and prose outside <pre> is byte-identical to before (pinned by a captured-output test). Fixing the cleaner alone would have been inert: ContentProcessor ran clean_html a second time on the text every producer had already cleaned, and with no <pre> left to protect, that pass stripped the indentation again before the row was stored. The processor now skips that redundant pass when the content carries no HTML markup (ContentCleaner.looks_like_html), and still cleans content that really is HTML. Forward-only: existing rows keep their old content, and eval-article rows and config/eval_articles_data fixtures are untouched. Verified against the live Hunter's Ledger page: 18 embedded rules now come out of clean_html as parseable Sigma YAML with nested detection blocks. Covered by 8 new cleaner tests in tests/test_content_cleaner.py and 2 processor tests in tests/test_content_processor.py.
• JSON-LD article picker accepts TechArticle and prefers the block that names an author (2026-09-02): StructuredDataExtractor.find_article_jsonld (src/core/modern_scraper.py) returned the first JSON-LD block typed Article, BlogPosting, or NewsArticle and ignored every other block. The Hunter's Ledger report pages emit a bare BlogPosting followed by a TechArticle carrying author.name, so the person-level author was dropped on the scrape path (feed-path articles were unaffected because rss_parser uses the feed author). The picker now also accepts TechArticle, ScholarlyArticle, and Report, and when several article-typed blocks exist it prefers the one with a usable author name, then the one with articleBody, then document order -- so single-block pages behave exactly as before. An author field with no usable name (empty list, empty object, organization without a name) does not count. extract_from_jsonld is unchanged. Covered by 11 new cases in tests/test_modern_scraper.py, including the two-block Hunter's Ledger shape and the single-block regression guard.
• WF-CFG-PHANTOM-003 un-quarantined: autosave dropped a stored agent_models key whenever its own last-known value was blank (2026-09-02): performAutoSave() in config.js only fell back to the stored value for a key that read back as '' (a provider/model round-trip miss, or a catalog fetch still in flight) when that stored value was itself a non-empty string -- so a key whose currently-known value happened to be '' (a genuinely unconfigured agent, or one whose real value hadn't reached this page's currentConfig snapshot yet) was dropped from the PUT payload entirely rather than sent as ''. The backend's agent_models merge masked this in practice (an omitted key keeps whatever the DB already has), but it meant the save payload could not stand on its own -- exactly what [WF-CFG-PHANTOM-003] asserts. Fixed by keying the fallback on whether currentConfig.agent_models has the key at all, not on the stored value being non-empty. Root-caused via a from-scratch Postgres+Redis+uvicorn environment mirroring tests.yml's ui job: a freshly-seeded config has every *_model/*_provider key stored as '', which reproduced the exact "stored key missing from the sent payload" failure deterministically (no concurrency needed) and is also the condition an in-flight catalog/provider fetch reduces a real value to at collection time. Verified with the restored spec run 2-up alongside every other non-excluded workflow spec (expanded_prompt_editor_save.spec.ts included) and repeated within-file at --workers=2. Un-quarantined: quarantined_workflow_config_phantom_003.spec.ts merged back into workflow_config_no_phantom_save.spec.ts, and playwright.config.ts's quarantineProject.testMatch restored to empty.

Security

• Dependency pins bumped for pip-audit findings (2026-09-25): pip-audit -r requirements.txt --no-deps --disable-pip (a full resolve fails on a langchain-core conflict) flagged three pins. GitPython 3.1.58 -> 3.1.60 (PYSEC-2026-3785/3786/3787/3788/3837 fixed in 3.1.59, PYSEC-2026-3982/3984 fixed in 3.1.60; still zero import sites). mkdocs-material 9.7.6 -> 9.7.7 (PYSEC-2026-3864), in both requirements.txt and pyproject.toml. accelerate 1.13.0 -> 1.15.0 (PYSEC-2026-3804 / CVE-2026-69112 / GHSA-4j2p-28q2-5m79, path traversal via sharded-checkpoint weight_map entries in load_checkpoint_in_model / load_checkpoint_and_dispatch): the advisory lists no fixed version, but its text scopes it to "through 1.14.0" and PyPI reports no advisories against 1.15.0. A patched release existed, so the pin moved and no accepted-risk ignore was added. uv.lock regenerated with uv lock; only those three packages changed.