AiSOC v11.2.0

Added

• The detection engine runs 2,603 rules instead of 833, and every one of the
  1,770 added rules has been watched to fire. The library has held roughly
  7,000 ATT&CK-mapped rules for months while 833 executed, and the question
  that prompted this — why 800 and not 5,000 — turned out to have a single
  concrete answer rather than a missing feature.

  Windows telemetry was unreachable, and nothing said so. The engine
  flattens raw_event's top level into the namespace the matcher reads, and
  the matcher does a plain event.get(field) with no path traversal. A Windows
  event puts its payload one level below that: CommandLine lives at
  raw_event.EventData.CommandLine, EventID under System. Those are the
  two most-used fields in the public Sigma corpus — CommandLine in 2,173
  rules and Image in 2,300 — so every Windows rule read None on its first
  clause and could not fire however correctly it was written.
  windows_event.normalize() now lifts both containers, which is where the
  fix belongs: System and EventData are names from the Windows event
  schema, and the engine is shared by every connector. Nothing in the engine or
  the matcher changed, so the blast radius on the existing 833 is nil — proven
  by replaying all 1,756 committed fixtures and confirming every native rule
  keeps its verdict, now a standing test rather than a one-off check.

  scripts/sigma_compiler.py translates imported Sigma into match_when, or
  refuses. Refusal is the design. Sigma compares strings case-insensitively
  and several matcher operators do not, so equality and affix tests compile to
  anchored pattern_match_any rather than to endswith_any, which would have
  silently stopped matching \SvcHost.exe. Sigma's not filter is true when
  the field is missing, and only not_in and not_contains_any behave that
  way, so the other negations are refused rather than narrowed. Of 3,132
  imported rules, 1,770 ship and 1,362 are refused with a recorded reason — the
  largest being 556 whose log source no connector emits and 464 whose negation
  would flip on a missing field. The full taxonomy is in
  docs/detections/sigma-compilation.md.

  A rule ships only after it is seen to fire. check_detection_fields.py
  says in its own docstring that it over-approximates and that "a false pass is
  a rule this gate should have caught", so passing it is not evidence.
  scripts/compile_sigma_ruleset.py replays a vendor-shaped event through the
  real connector normalize() and the real DetectionEngine, and
  keeps the rule only if it produces a hit and stays silent on an empty event
  of the same shape. The proof is that the rule is reachable and fires on a
  well-formed event of its log source — not that it detects an attack, which
  no gate here claims. --prove-gate reverts the connector to its pre-fix
  behaviour and requires all 1,687 Windows rules to stop firing, so the proof
  is known to be capable of failing.

• Eight windowed aggregation rules, covering the detection family the
  quarantine index had been telling contributors to skip. Windows now
  contributes here too: failed-logon volume per host, password spray counted by
  distinct account rather than by attempt, service-install and scheduled-task
  bursts, remote-thread fan-out, and per-process DNS fan-out for tunnelling
  and domain-generation algorithms.

• scripts/measure_triage_reliability.py — the measurement behind the
  auto-triage reliability figures under Fixed below. It imports the
  production prompt, envelope and parser rather than restating them, and uses a
  different alert per attempt: production pins temperature=0.0, so asking
  one alert twenty times measures one reply twenty times, not a rate. Failed
  attempts record the model's own words, since "Expecting value: line 4 column
  16" cannot be diagnosed without the text it indexes into. With no model
  reachable it prints SKIPPED and says that nothing was measured, because a
  skip is not a pass.

Changed

• Nothing an operator has to do, but the detection surface is 3.1x wider.
  There is no configuration change, no migration and no API change in this
  release — the connector fix lifts the System and EventData containers
  into the namespace the matcher already read, and a connector-normalized key
  still wins on collision, so it only adds fields. What does change is that
  2,603 rules evaluate where 833 did, and 1,687 of the additions are Windows
  rules that could never fire before, so a deployment with Windows telemetry
  should expect more alerts from the same stream. Every imported rule carries
  its upstream_status onto the alert, so the 125 experimental ones can be
  filtered without disabling the rest.

• Upstream lifecycle status no longer decides whether an imported rule runs.
  The importer quarantined on SigmaHQ status, which put 2,844 test and 211
  experimental rules behind a flag. That conflated whether a rule can
  execute here with how confident its authors are in its content; only the
  first was ever the blocker. In SigmaHQ, test means reviewed and in
  community use — the normal state for most of the corpus, not a warning.
  Fireability now gates, status is carried onto the rule and the alert so it
  can still be filtered, and deprecated/unsupported are refused outright.

• Published counts lead with the library and always name the executable
  figure beside it — "6,991 rules on disk, 2,603 executable". The two travel
  together because a library figure on its own reads as coverage. 6,991 rather
  than the 7,016 files under detections/, because 25 of those are playbooks.

Fixed

• 76 rules were marked enabled and the engine had never heard of them. They
  counted as shipped coverage and detected nothing: 44 native rules authored as
  YAML with no Python spec (the YAML is a generated projection, so the engine
  never reads it), 31 imported Sigma rules, and 1 community rule. 46 of the
  original 122 became executable through the compiler; the rest are now
  enabled: false with a quarantine_reason naming what each would need. They
  were deliberately not translated: their fields have no known emitter, so
  making them load would have added rules that cannot fire, which is the defect
  being removed rather than progress against it.
  detection_truth_table.py --check now fails on any such rule.

• readme_gates.py misread a formatted figure. Its pattern stopped at the
  thousands separator, so "2,603 executable" was compared as 603 — a gate
  measuring a number the README never claimed.

• The quarantine index told translators to skip the largest untranslated
  family in it. Step 3 said | stats aggregations belong in the windowed
  engine, "which today has three hardcoded rules and no loader. Skip those
  until it has one." load_window_rules() reads windowed_ruleset.json and
  main.py wires it on by default, so the instruction had been stale for some
  time while continuing to steer people away from roughly half the Splunk
  quarantine. Corrected in scripts/build_quarantine_index.py, since the
  README is generated output and editing the file would have regenerated the
  error — the same shape as the detection-coverage generator that kept
  restoring a dead link after the page was fixed.

• Auto-triage never asked the provider for JSON, so a third of the bundled
  model's replies were thrown away. run_auto_triage parses the reply as a
  JSON object, but nothing in services/agents/app/llm/ ever set
  response_format, leaving the model free to emit prose shaped like JSON and
  the parser to correct it afterwards. Measured over 50 alerts from the
  committed synthetic corpus, through the LiteLLM gateway exactly as production
  routes, against the bundled llama3.2:3b-instruct-q4_K_M at the production
  temperature=0.0 / max_tokens=512: replies triage could use went from
  44 of 50 to 50 of 50 once the reply was constrained to a JSON object.

  The failure shape is worth recording, because it was not the one expected.
  Every one of the six failures carried a correct verdict and confidence
  and a malformed rationale — "rationale": The alert is... with the
  opening quote simply missing, a raw newline before an unquoted value, or an
  invalid \' escape. finish_reason was stop on all 50 calls, so
  truncation was not involved and max_tokens is exonerated. A model that had
  reasoned correctly was being discarded over its punctuation, which is
  precisely what constraining the grammar prevents and no amount of parsing
  after the fact can.

  The gateway ships drop_params: true, so a provider that does not support
  the parameter has it dropped rather than erroring. Requested through
  model_kwargs rather than as a top-level argument, because langchain warns
  on every construction otherwise.

• The published "7 of 19" reliability figure does not reproduce. Re-measured
  on the production call path at the sample sizes above, the pre-fix tree reads
  44 of 50. The original number's conditions are not recoverable — it was taken
  during a QA run whose host had exhausted its disk — so rather than restate it,
  scripts/measure_triage_reliability.py now produces the figure repeatably and
  records why each failed attempt failed. Every surface that published the old
  ratio now carries the new one and names the method.

• A valid verdict was discarded when one field had the wrong type.
  float(data.get("confidence", 0.5)) was unguarded, so "confidence": "high"
  raised ValueError, which the caller converts into an AutoTriageError and a
  deterministic fallback — losing a verdict and rationale that may have been
  perfectly good. _coerce_confidence now reads numeric strings and
  percentages and degrades to a neutral 0.5 otherwise. That is safe here and
  only here: auto-close requires confidence >= AUTO_CLOSE_THRESHOLD (0.85 by
  default), so an unread field escalates to a human instead of closing an
  alert. The verdict itself still fails closed. Not a cause of the measured
  failures, but reachable by any model on any alert.

• structured_output.py described itself as the single fail-closed parser
  while nothing in production imported it. Its docstring said each agent
  "had its own ad-hoc _parse_llm_response", past tense; all five still did,
  and its only callers were its own tests — a passing test on an uncalled
  function is indistinguishable from a working feature. auto_triage_agent now
  shares its extraction step, and the docstring records that cloud_agent,
  identity_agent, insider_threat_agent and phishing_agent still carry
  their own, rather than describing the consolidation as finished.

  Extracting is also now brace-balanced and string-aware. It trimmed prose
  before the opening { and nothing after the closing }, so a model that
  answered correctly and added "Hope that helps!" was scored unparseable — and
  a } inside a string value would have ended the scan early. An unclosed
  object is still handed to the JSON parser to report rather than silently
  trimmed to the last balanced point, because extraction may read what the
  model said and must not guess at what it meant.

• Three counters still classified rules by their directory, and disagreed
  with the engine by about 1,700 each. _quarantine/ stopped meaning "cannot
  run" when the Sigma compiler began translating rules where they sat, and the
  tools that had not been moved to the compiled ruleset kept reporting the old
  world while printing OK. validate_detections.py published "Quarantined
  (parsed-but-disabled): 5937" against a truth table saying 4,213 about the
  same tree; it now reports Executable (loaded by the engine): 2603; not loaded: 4388, read from the same artefact, so the two cannot diverge.
  curate_detections.py skipped every _quarantine/ file before scoring, so
  the published coverage page selected from 1,054 candidates out of 2,603
  executable rules and its "By category" list contained a category named
  _quarantine; candidates are now the rules the engine loads, and a rule
  that cannot fire is no longer eligible for a page that promises coverage.

• build_marketplace.py published 7,016 detections against a README and a
  truth table that both say 6,991. The 25-rule gap was detections/playbooks/
  — response playbooks with trigger: and steps: and no detection: block —
  walked as native detections because playbooks was missing from the skip set
  that detection_truth_table.py has always carried. They are indexed as
  playbooks now, so the marketplace keeps all 7,155 items, stats.detections
  reads 6,991, and the landing page's pack figure is unaffected: stats.playbooks
  counts every playbook and a new stats.playbook_packs counts the v1 pack
  alone, because 62 of the 87 are packs and calling all 87 packs would have
  traded one wrong number for another.

• Five refusal reasons rendered as broken table rows. The Sigma compilation
  report escapes the example it prints and not the reason, and five reasons
  name a Sigma modifier — |re, |cidr, |all, |base64, |fieldref —
  whose leading pipe is also a Markdown cell delimiter, so each opened an empty
  first column and shifted its row. The rows that rendered wrong were exactly
  the ones explaining the least obvious refusals.

• The DRL-1.1 attribution gap is now stated in the report rather than
  discoverable only from the JSON. provenance.author is empty on all 1,770
  compiled rules because the Sigma importer never read the upstream author:
  field. Attribution travels as repository, upstream rule id, upstream path and
  licence — enough to find the rule, not enough to name who wrote it. The engine
  builds its attribution sentence from what the block holds, so an alert reads
  short rather than crediting an author called "", but a licence obligation
  that is partly met is not met. Closing it needs a re-import.