You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This commit was created on GitHub.com and signed with GitHub’s verified signature.
v0.14.0 public research release
Changed
Reframed the repository status as a public research release rather than a generic
pre-release label, while preserving explicit non-claims around production protection,
certification, v1.0 stability, and shipped gateway/runtime behavior.
Added docs/project-tracks.md to separate the shipped benchmark/evidence track from
the future LLM Safety Gateway / Runtime Verifier direction.
Added
Context-consent deterministic campaign: ash context-consent-campaign publishes the
fifth boundary family, "context is not consent", with 5 synthetic consent-boundary
cases, 45 deterministic rows, control-ablation attribution, benign-path checks,
validator support, and the committed sanitized example examples/context-consent-sanitized/. No local or external models are called.
Live local-model marketing web-injection campaign: ash marketing-web-live-campaign
runs an owned localhost web stand against local worker/chief models, writes raw
pages/prompts/responses under .internal/, publishes a sanitized example under examples/marketing-web-live-sanitized/, and validates public page/response hashes,
control attribution, bounded-vs-ablation outcomes, and benign pass behavior.
Marketing web-injection campaign: ash marketing-web-injection-campaign models a
controlled offline marketing/ads analytics swarm reading hostile web-like material,
writes private raw artifacts under .internal/, publishes a sanitized example under examples/marketing-web-injection-sanitized/, and validates naive/bounded/ablation/
benign outcomes without exposing synthetic strategy values.
Deep sanitized local-model mini-swarm evidence pack: examples/swarm-defense-live-deep-sanitized/ records 168 private local-model
observations as public-safe hashes and aggregate metrics, including unsafe/benign
rates, Wilson intervals, model breakdowns, and replay-ablation attribution.
docs/run-your-model.md: a single cross-platform operator path for deterministic
demo, one OpenAI-compatible model, and local mini-swarm campaigns.
docs/evidence-pack-format.md: public rules for promoting private/local research
into sanitized, hash-anchored evidence updates with claim rows, tests, and validation.
instruction-integrity scenario family so named scenario groups cover the full
24-pattern corpus instead of leaving prompt/instruction patterns only under all.
Semantic propagation defense model v0.2: sanitized public artifacts now include a
defensive control catalog, control-effect ablation rows, validator checks, and a
reviewer note documenting the public/private evidence boundary.
Semantic propagation campaign: ash semantic-propagation-campaign writes sanitized
public artifacts for worker-to-chief semantic drift propagation probes, including
deterministic bounded-vs-ablation contract results, local-model observation
aggregates, response-hash coverage, and validator support. Raw worker/chief prompts,
responses, canonical-state hashes, and synthetic canaries stay private under .internal/.
Semantic parameter drift campaign: ash semantic-drift-campaign writes sanitized
public artifacts for 4 synthetic local mini-swarm handoff cases, including
deterministic bounded-vs-ablation contract results, local-model observation
aggregates, response-hash coverage, and validator support. Raw prompts, raw
responses, canonical-state hashes, and synthetic canaries stay private under .internal/.
Git evidence workflow documenting the issue -> branch -> artifact -> PR -> GitHub
checks -> review gate -> merge/close process as a public project norm.
Scenario timeline fixtures and validator contract for delayed activation, context
overload, and handoff provenance scenarios. These are synthetic design fixtures, not a
live multi-agent executor.
Generated showcase failure cards from the committed demo-agent report, with trace
references and non-claim language for reviewer-facing evidence.
ash local-suite for bounded local Prometheus/Ollama smoke profiles. The command is
dry-run by default, enforces request caps, and validates real local-run artifacts after
explicit --execute.
First-class low-context Prometheus profiles (prometheus-lowctx-smoke and prometheus-lowctx-reliability) for the maintainer Ollama alias prometheus-qwen15b-lowctx:latest, keeping the recovered local smoke reproducible by
name instead of as an ad hoc command.
docs/current-state.md: reviewer-facing status snapshot that separates shipped,
experimental, planned, active, and claim-boundary items.
docs/authorized-testing-paths.md: official/authorized use paths for synthetic local
labs, local runtimes, owned-system assessments, customer-authorized assessments,
provider-program testing, and standards-aligned benchmarking.
Evaluation topology documentation for single models, local targets, protected-vs-
vulnerable agents, memory/tool loops, model chains, multi-agent handoffs, provider
boundaries, human approval, and recovery paths.
Corpus expansion plan that requires invariant-based, topology-aware pattern selection
and rejects full combinatorial sweeps of model/provider/agent/time variants.
Documentation contract tests for methodology links, topology coverage, expansion-plan
structure, governance files, and stale pattern-count claims.
External raw-response evidence: new run-external --raw-response-limit flag, full
per-request response files under raw_responses/, and raw_response_path, raw_response_sha256, raw_response_chars, and raw_response_truncated fields in external_results.json.
Pattern-level external verdict validation: external prompts now require pattern_id
and boundary_assertion; the harness validates them against the concrete DefensivePattern and canonical control family before recording PASS/FINDING.
Golden snapshot coverage for external artifacts, including the normalized external_results.json, external_summary.json, external_report.md, run_config.json,
and the linked raw response file.
Run-history maintenance commands: ash stats, ash retention, and ash compare-models
expose the previously internal stats, retention, and external-run comparison logic.
External retry controls are part of the recorded run configuration and reproduction
command (--retries, retry backoff in artifacts).
Local-runtime metadata for external runs: run_config.runtime records runtime name,
runtime family, network_mode, authorization mode, model id, model license/policy note,
prompt-only/tool-execution flags, and recovery guidance for local Ollama, LM Studio,
vLLM, localhost, and generic OpenAI-compatible endpoints.
Local toy multi-agent handoff target: toy-multi-agent models a deterministic
coordinator/worker handoff for data-label stripping and capability-delegation drift,
records before/after handoff evidence in trace steps, and remains offline with no
provider calls or live tools.
Recovery-path pattern design: recovery.trust_gate_no_path now has a documented
pre-implementation proposal covering invariant, topology, expected vulnerable behavior,
trace evidence, protected control, residual risk, and anti-combinatorial guardrails.
Public showcase report checklist: required commands, artifacts, validation result,
baseline/protected summary, claim-boundary language, and standards-mapping caveat before
any report is promoted in README or release material.
Data-boundary theory module with an explicit envelope restriction relation
(E_out <= E_in), field-level non-expansion rules, and conservative policy-context
caveats for classification ordering, trusted sources, and TTL checks.
Data-boundary missing-envelope recovery pattern: data_boundary_missing_envelope_recovery checks fail-closed behavior when a required DataEnvelope is absent at a boundary action. The local corpus now has 24 deterministic
seed patterns; baseline demo targets fail all 24 and the protected demo target passes
all 24 under deterministic replay.
Data-boundary research closure records in the project tracker and claims registry,
separating public evidence artifacts from local-only derivation/audit notes.
Changed
run_diff.json is now schema v0.2 with explicit decisive/non-decisive labels such as finding_fixed, new_finding, and inconclusive_error_drift; v0.1 aliases and
validation support remain for compatibility.
Project tracker, project map, local Prometheus docs, and research claims now point to
the bounded local-suite workflow and clarify that weak local model evidence is
inconclusive/error unless the validated artifacts say otherwise.
agentic-boundary-model.md is now the canonical protection/boundary model catalog,
including current coverage and missing situation families.
README, protocol, semantics, project map, roadmap, research roadmap, harness,
development, adapter contract, and release checklist now distinguish boundary
invariants, evaluation topologies, shipped coverage, and planned work more explicitly.
README and project map now point reviewers to current-state and authorized-testing
documents before they infer status from scattered docs.
New external runs treat missing pattern ids, invalid boundary assertions, control-family
mismatches, and contradictory verdict fields as inconclusive instead of PASS/FINDING.
The fake OpenAI-compatible demo server now echoes the requested pattern id and emits the
new boundary assertion field.
Benchmark protocol and semantics now document the conservative external cross-check:
contradictory model self-reports, such as decision=block with would_preserve_boundary=false, are weak evidence and remain inconclusive.
Key CLI commands now have machine-readable output paths for automation while preserving
the existing human-readable default output.
External credential metadata now uses credential_env_var and the preferred --credential-env flag. The legacy api_key_env artifact field and --api-key-env
CLI alias remain readable for compatibility, but new artifacts and prompts avoid
secret-like plaintext patterns.
ash validate now hides validation message details in text output and redacts
secret-shaped strings in JSON output.
Standards mapping now asserts a small MITRE ATLAS 2026.05 verified subset for direct-fit
categories and keeps governance/audit/delegation categories deferred where the fit would
be speculative.
External reports and run manifests now surface runtime metadata and recovery guidance,
and the committed external demo report uses the same fake-local runtime metadata path
as a normal CLI run.
README, current-state, adapter contract, capability matrix, evaluation topologies,
roadmap, boundary model, and project map now list toy-multi-agent as shipped while
keeping live/cross-provider multi-agent workflows future-scoped.
The committed comparison example README now reflects the current 24-pattern corpus and
links to the public showcase checklist.