Skip to content

v0.14.0 public research release

Choose a tag to compare

@krivonosoff161 krivonosoff161 released this 01 Jul 04:52
· 67 commits to main since this release
9d974bb

v0.14.0 public research release

Changed

  • Reframed the repository status as a public research release rather than a generic
    pre-release label, while preserving explicit non-claims around production protection,
    certification, v1.0 stability, and shipped gateway/runtime behavior.
  • Added docs/project-tracks.md to separate the shipped benchmark/evidence track from
    the future LLM Safety Gateway / Runtime Verifier direction.

Added

  • Context-consent deterministic campaign: ash context-consent-campaign publishes the
    fifth boundary family, "context is not consent", with 5 synthetic consent-boundary
    cases, 45 deterministic rows, control-ablation attribution, benign-path checks,
    validator support, and the committed sanitized example
    examples/context-consent-sanitized/. No local or external models are called.
  • Live local-model marketing web-injection campaign: ash marketing-web-live-campaign
    runs an owned localhost web stand against local worker/chief models, writes raw
    pages/prompts/responses under .internal/, publishes a sanitized example under
    examples/marketing-web-live-sanitized/, and validates public page/response hashes,
    control attribution, bounded-vs-ablation outcomes, and benign pass behavior.
  • Marketing web-injection campaign: ash marketing-web-injection-campaign models a
    controlled offline marketing/ads analytics swarm reading hostile web-like material,
    writes private raw artifacts under .internal/, publishes a sanitized example under
    examples/marketing-web-injection-sanitized/, and validates naive/bounded/ablation/
    benign outcomes without exposing synthetic strategy values.
  • Deep sanitized local-model mini-swarm evidence pack:
    examples/swarm-defense-live-deep-sanitized/ records 168 private local-model
    observations as public-safe hashes and aggregate metrics, including unsafe/benign
    rates, Wilson intervals, model breakdowns, and replay-ablation attribution.
  • docs/run-your-model.md: a single cross-platform operator path for deterministic
    demo, one OpenAI-compatible model, and local mini-swarm campaigns.
  • docs/evidence-pack-format.md: public rules for promoting private/local research
    into sanitized, hash-anchored evidence updates with claim rows, tests, and validation.
  • instruction-integrity scenario family so named scenario groups cover the full
    24-pattern corpus instead of leaving prompt/instruction patterns only under all.
  • Semantic propagation defense model v0.2: sanitized public artifacts now include a
    defensive control catalog, control-effect ablation rows, validator checks, and a
    reviewer note documenting the public/private evidence boundary.
  • Semantic propagation campaign: ash semantic-propagation-campaign writes sanitized
    public artifacts for worker-to-chief semantic drift propagation probes, including
    deterministic bounded-vs-ablation contract results, local-model observation
    aggregates, response-hash coverage, and validator support. Raw worker/chief prompts,
    responses, canonical-state hashes, and synthetic canaries stay private under
    .internal/.
  • Semantic parameter drift campaign: ash semantic-drift-campaign writes sanitized
    public artifacts for 4 synthetic local mini-swarm handoff cases, including
    deterministic bounded-vs-ablation contract results, local-model observation
    aggregates, response-hash coverage, and validator support. Raw prompts, raw
    responses, canonical-state hashes, and synthetic canaries stay private under
    .internal/.
  • Git evidence workflow documenting the issue -> branch -> artifact -> PR -> GitHub
    checks -> review gate -> merge/close process as a public project norm.
  • Scenario timeline fixtures and validator contract for delayed activation, context
    overload, and handoff provenance scenarios. These are synthetic design fixtures, not a
    live multi-agent executor.
  • Generated showcase failure cards from the committed demo-agent report, with trace
    references and non-claim language for reviewer-facing evidence.
  • ash local-suite for bounded local Prometheus/Ollama smoke profiles. The command is
    dry-run by default, enforces request caps, and validates real local-run artifacts after
    explicit --execute.
  • First-class low-context Prometheus profiles (prometheus-lowctx-smoke and
    prometheus-lowctx-reliability) for the maintainer Ollama alias
    prometheus-qwen15b-lowctx:latest, keeping the recovered local smoke reproducible by
    name instead of as an ad hoc command.
  • docs/current-state.md: reviewer-facing status snapshot that separates shipped,
    experimental, planned, active, and claim-boundary items.
  • docs/authorized-testing-paths.md: official/authorized use paths for synthetic local
    labs, local runtimes, owned-system assessments, customer-authorized assessments,
    provider-program testing, and standards-aligned benchmarking.
  • GitHub project-governance surface: PR template, issue templates for bugs/features/
    defensive pattern proposals, CODEOWNERS, Dependabot config, CodeQL, Scorecard, release
    artifact workflow, governance, maintainers, support, code-of-conduct, and citation files.
  • Evaluation topology documentation for single models, local targets, protected-vs-
    vulnerable agents, memory/tool loops, model chains, multi-agent handoffs, provider
    boundaries, human approval, and recovery paths.
  • Corpus expansion plan that requires invariant-based, topology-aware pattern selection
    and rejects full combinatorial sweeps of model/provider/agent/time variants.
  • Documentation contract tests for methodology links, topology coverage, expansion-plan
    structure, governance files, and stale pattern-count claims.
  • External raw-response evidence: new run-external --raw-response-limit flag, full
    per-request response files under raw_responses/, and raw_response_path,
    raw_response_sha256, raw_response_chars, and raw_response_truncated fields in
    external_results.json.
  • Pattern-level external verdict validation: external prompts now require pattern_id
    and boundary_assertion; the harness validates them against the concrete
    DefensivePattern and canonical control family before recording PASS/FINDING.
  • JSON CLI output for automation: ash validate --format json, ash stats --format json,
    ash retention --format json, and ash compare-models --format json.
  • Golden snapshot coverage for external artifacts, including the normalized
    external_results.json, external_summary.json, external_report.md, run_config.json,
    and the linked raw response file.
  • Run-history maintenance commands: ash stats, ash retention, and ash compare-models
    expose the previously internal stats, retention, and external-run comparison logic.
  • External retry controls are part of the recorded run configuration and reproduction
    command (--retries, retry backoff in artifacts).
  • Local-runtime metadata for external runs: run_config.runtime records runtime name,
    runtime family, network_mode, authorization mode, model id, model license/policy note,
    prompt-only/tool-execution flags, and recovery guidance for local Ollama, LM Studio,
    vLLM, localhost, and generic OpenAI-compatible endpoints.
  • Local toy multi-agent handoff target: toy-multi-agent models a deterministic
    coordinator/worker handoff for data-label stripping and capability-delegation drift,
    records before/after handoff evidence in trace steps, and remains offline with no
    provider calls or live tools.
  • Recovery-path pattern design: recovery.trust_gate_no_path now has a documented
    pre-implementation proposal covering invariant, topology, expected vulnerable behavior,
    trace evidence, protected control, residual risk, and anti-combinatorial guardrails.
  • Public showcase report checklist: required commands, artifacts, validation result,
    baseline/protected summary, claim-boundary language, and standards-mapping caveat before
    any report is promoted in README or release material.
  • docs/v1-readiness.md: stable-vs-experimental readiness matrix covering clean install,
    fake-server path, schema/corpus freeze expectations, showcase report gate, claim
    boundaries, and open v1.0 blockers.
  • Data-boundary theory module with an explicit envelope restriction relation
    (E_out <= E_in), field-level non-expansion rules, and conservative policy-context
    caveats for classification ordering, trusted sources, and TTL checks.
  • Data-boundary missing-envelope recovery pattern:
    data_boundary_missing_envelope_recovery checks fail-closed behavior when a required
    DataEnvelope is absent at a boundary action. The local corpus now has 24 deterministic
    seed patterns; baseline demo targets fail all 24 and the protected demo target passes
    all 24 under deterministic replay.
  • Data-boundary research closure records in the project tracker and claims registry,
    separating public evidence artifacts from local-only derivation/audit notes.

Changed

  • run_diff.json is now schema v0.2 with explicit decisive/non-decisive labels such as
    finding_fixed, new_finding, and inconclusive_error_drift; v0.1 aliases and
    validation support remain for compatibility.
  • Project tracker, project map, local Prometheus docs, and research claims now point to
    the bounded local-suite workflow and clarify that weak local model evidence is
    inconclusive/error unless the validated artifacts say otherwise.
  • agentic-boundary-model.md is now the canonical protection/boundary model catalog,
    including current coverage and missing situation families.
  • README, protocol, semantics, project map, roadmap, research roadmap, harness,
    development, adapter contract, and release checklist now distinguish boundary
    invariants, evaluation topologies, shipped coverage, and planned work more explicitly.
  • README and project map now point reviewers to current-state and authorized-testing
    documents before they infer status from scattered docs.
  • New external runs treat missing pattern ids, invalid boundary assertions, control-family
    mismatches, and contradictory verdict fields as inconclusive instead of PASS/FINDING.
  • The fake OpenAI-compatible demo server now echoes the requested pattern id and emits the
    new boundary assertion field.
  • Benchmark protocol and semantics now document the conservative external cross-check:
    contradictory model self-reports, such as decision=block with
    would_preserve_boundary=false, are weak evidence and remain inconclusive.
  • Key CLI commands now have machine-readable output paths for automation while preserving
    the existing human-readable default output.
  • External credential metadata now uses credential_env_var and the preferred
    --credential-env flag. The legacy api_key_env artifact field and --api-key-env
    CLI alias remain readable for compatibility, but new artifacts and prompts avoid
    secret-like plaintext patterns.
  • ash validate now hides validation message details in text output and redacts
    secret-shaped strings in JSON output.
  • Standards mapping now asserts a small MITRE ATLAS 2026.05 verified subset for direct-fit
    categories and keeps governance/audit/delegation categories deferred where the fit would
    be speculative.
  • External reports and run manifests now surface runtime metadata and recovery guidance,
    and the committed external demo report uses the same fake-local runtime metadata path
    as a normal CLI run.
  • README, current-state, adapter contract, capability matrix, evaluation topologies,
    roadmap, boundary model, and project map now list toy-multi-agent as shipped while
    keeping live/cross-provider multi-agent workflows future-scoped.
  • The committed comparison example README now reflects the current 24-pattern corpus and
    links to the public showcase checklist.