Skip to content

Releases: IdoGol24/weir

v0.6.0: weir-scan 0.6.0 and weir-run 0.2.0

Choose a tag to compare

@IdoGol24 IdoGol24 released this 22 Oct 18:09
c21280f

weir-scan 0.6.0 and weir-run 0.2.0. Install: pip install weir-scan weir-run, then try weir canary judge --sample.

weir-scan 0.6.0

Added

  • The judge reports whether the network was isolated.
    canary-ledger/1 gains a network record (isolated, required) and a
    direct record (flow, protocol, target, attempts) for each
    connection attempt isolation blocked. Every report carries a network
    line after the coverage line, network isolated: N direct connections blocked or network not isolated: direct connections were not observed, and canary-report/1 a network object. Two incomplete
    reasons are new: network_not_isolated (isolation was required and
    missing) and direct_connection (a flow tried to reach a host around the
    mock). A ledger with no network record is reported as not isolated.
  • Outbound requests are a sink (weir canary judge --requests). The judge reads a request capture (canary-requests/1,
    one line per request the mock received: destination, method, path,
    query, headers and body, and whether it was served, refused or a
    CONNECT) and searches every request for every planted value in every
    encoded form. A value in a credential field of a request to the
    placement's own provider (a ledger placement's new optional providers
    endpoint ids and provider_host; each request record names its
    credential_fields) is request or egress evidence that it was
    exercised; anywhere else, the rest of that request included, it is a
    leak, reported with signal=requests,
    the request's sink (model, oauth, otlp, a declared api's id,
    unmocked, or <host> (blocked) for a refused plain-HTTP request) and
    the field (headers.<name>, query.<name>, body.<json path>,
    body.<form field>, path, host, method, or the whole query or
    body). A refused CONNECT has no visible content: it is listed as a
    blocked connection attempt and counted in the coverage, never a leak by
    itself. The report gains the hit field sink, a blocked list and the
    coverage counts request_files, requests, requests_refused and
    connections_blocked; the text coverage line gains a requests segment
    when a request capture is given. A request capture line that is not a
    request record, or a body the mock could not read in full, makes the run
    incomplete (unparsed_record). The schema artifact
    canary-requests.v1.schema.json is new; canary-ledger.v1 and
    canary-report.v1 gain the fields above. The judge stays offline and
    deterministic.

Changed

  • The printing rule withholds runs of 6 characters, down from 8. No
    report prints a run of 6 or more consecutive characters of any planted
    value, and a ledger id may not hold one.
  • The PyPI summary names every sink the judge searches. It reads
    "Leak tests for AI agents: deterministically find planted credentials
    in your agent's telemetry, model requests and outbound calls."

Corpus

  • The release commit regenerated the diffspec baselines for the package
    version, which the baselines embed: weir_version moved from 0.5.0 to
    0.6.0. That diff is version-only; no flow fact, node count or finding
    moved.

weir-run 0.2.0

Released beside weir-scan 0.6.0 and requiring weir-scan>=0.6.0.

Added

  • Flows run where the only network is loopback. On Linux,
    weir-run run and the pytest plugin run the mock and every flow in an
    unprivileged user and network namespace with no route out, so a client
    that ignores the proxy settings reaches nothing. Each blocked attempt is
    seen and reported (tcp and udp by address and port, dns by name),
    and a blocked datagram's payload, a DNS query included, is swept capture,
    so a canary in it is a leak. Where the namespace cannot be made (not
    Linux, user namespaces disabled or restricted), the run goes ahead
    unisolated and says why. --require-isolation
    (--canary-require-isolation) makes an unisolated run incomplete, exit
    3. weir-run now requires weir-scan>=0.6.0, whose judge reads the new
    ledger records.
  • Every request the mock receives is recorded. The mock
    writes each request, refused ones included, to capture/requests.jsonl
    before answering it, and the runner passes it to the judge with
    --requests and deletes it after the run with the rest of the capture.
    A placement names its own provider with the new provider key (model,
    oauth, otlp or a [[mock.endpoint]] id; provider= in the pytest
    plugin), so a key pasted into a prompt, sent to the model API as a
    credential, sent to the wrong one of the program's APIs or sent in a
    plain-HTTP request to another host is a leak. A token the mock's OAuth
    endpoints issue belongs to them and to every declared endpoint.
  • A warning for each placement without a provider. Before
    any flow runs, weir-run run and the pytest plugin print placement <id> has no provider: any request holding it counts as a leak to
    stderr for every placement with neither provider nor
    provider_host. It names the id, never a value.
  • An Anthropic Messages API mock. Model APIs can now be
    OpenAI-compatible or Anthropic Messages API. Beside the
    OpenAI-compatible APIs, the mock serves POST /v1/messages, plain and
    streaming (message_start, ping, content_block_start,
    content_block_delta, content_block_stop, message_delta,
    message_stop), on the same script: with tools offered, turn one
    answers with a tool_use block for every tool, and once the last user
    message carries tool_result blocks it answers model_reply. A query
    such as ?beta=true, body fields it does not know and system entries
    between turns are accepted, and the model API answers HEAD and GET /api/hello with 200. Flows get ANTHROPIC_BASE_URL pointing at the
    mock, the variable the Anthropic SDKs read their base URL from; a
    flow's env may no longer set it. The model API now always declares
    the x-api-key header as a credential field: a placement with
    provider = "model" sent there is exercised, the same value in a
    prompt or a tool result is a leak to model, and any other value in
    x-api-key is unexpected.model.x-api-key and makes the run
    incomplete.

Changed

  • A placement without provider belongs in no request. Any
    request that holds its value is now a leak, where a credential field
    holding it used to count as exercised wherever it was sent. Add
    provider to each placement that a flow legitimately sends to a mocked
    endpoint. The runner refuses a weir-scan whose judge has no --requests.
  • The mock no longer writes request or egress evidence.
    The judge derives both from the request capture, so the evidence and
    the leak verdict come from one place.
  • The PyPI summary describes the rig. It reads "Leak tests for AI
    agents: plant fake credentials, run your agent in an isolated rig, and
    capture everything it sends out."

v0.5.0: weir-scan 0.5.0 and weir-run 0.1.0

Choose a tag to compare

@IdoGol24 IdoGol24 released this 21 Oct 14:57
b8b4f3e

weir-scan 0.5.0 and weir-run 0.1.0. Install: pip install weir-scan weir-run, then try weir canary judge --sample.

weir-scan 0.5.0

Added

  • weir canary judge: did a planted fake credential reach the
    telemetry?
    weir canary judge --ledger FILE --otlp PATH [--json] [PATH...] reads a canary-ledger/1 ledger (placement ids plus the fake
    values planted there) and the captured telemetry, and reports every
    placement whose value, or an encoded form of it, appears anywhere in the
    capture, naming the placement, file, line, record and location, never a
    value or a fragment of one. The contract is docs/canary.md. This first
    version ships the placement kinds env, code and mock-oauth; the
    evidence types request, minted and egress; the forms raw, json,
    json2, url, base64, base64url (the base64 core at byte offsets 0,
    1 and 2) and hex; and the incomplete reasons required_not_exercised,
    flow_not_ok, unexpected_credential, not_scanned, unparsed_record
    and empty_capture. --otlp files get a structured walk of every span,
    log record and metric (every string, stringValue and decoded
    bytesValue) plus a raw byte sweep; other paths, such as a flow's stdout
    and stderr, get the raw sweep only. Exit 0 clean, 1 leak, 2 unusable
    input, 3 incomplete. --json emits the canary-report/1 report; both
    schemas are in packages/weir/schema_artifacts/. The judge opens no
    socket, starts no process and reads no clock.
  • weir canary judge --sample: a leak report in one command. It judges
    a bundled sample (a ledger of fake values and an OTLP/JSON capture with
    one planted value in a span attribute) with no other input, offline, and
    exits 1 naming the placement, span and attribute, never the value.
  • weir inventory (experimental): which credentials an agent could reach,
    by name only.
    Its output and JSON schema may change without notice. Four
    sources: a versioned list of standard credential locations under the given
    home (~/.aws, Google application default credentials,
    ~/.npmrc, ~/.kube/config, ~/.docker/config.json, ~/.netrc, git
    credentials, gh and more, including the variables that move them); the
    project tree, through Weir's catalog plus imported gitleaks or trufflehog
    results; the launch environment (--env-file or --process-env) with the
    shell startup files that re-export into it; and the approved map
    .weir/reach.yml. Environment variables are secrets by name or value
    shape, not secrets when the value is a number, boolean, path or plain
    hostname, and otherwise unknown: a count on screen, names in a file, a
    failure only with --strict, markable as fine in the map. Files a
    .gitignore covers are reported per route (hidden from Grep's directory
    search, still reachable by Read, cat and grep -r). Nothing prints, logs or
    stores a value; --store writes HMAC-SHA256 fingerprints under a local
    key plus byte offsets. --json emits the weir-inventory/1 report, whose
    schema and Python API are documented in docs/inventory.md. --home is
    required: there is no default home.

Corpus

  • The release commit regenerated the diffspec baselines for the package
    version, which the baselines embed: weir_version moved from 0.4.1 to
    0.5.0 (main never carried 0.4.2, which shipped from release/0.4.x). That
    diff is version-only; no flow fact, node count or finding moved.

weir-run 0.1.0

The canary runner is its own package, weir-run, released beside weir-scan
0.5.0 and requiring weir-scan>=0.5.0. Its versions are recorded here,
under weir-run headings.

Added

  • The canary runner, a separate package. It plants unique fake canaries
    (env and code placements), runs a program's flows under an empty
    environment plus an allowlist against a mock bound to 127.0.0.1 (an
    OpenAI-compatible model API with scripted tool calls, OAuth endpoints
    whose issued tokens become mock-oauth placements, an OTLP/HTTP receiver
    writing OTLP/JSON lines, the program's own declared APIs, and refusal of
    every proxied request to another host; the runner does not sandbox the
    network, so a client that ignores proxy settings can connect directly),
    relays flow output with every canary form redacted, and exits with the code of weir canary judge (docs/canary.md). Ships a
    subprocess-mode pytest plugin and a synthetic demo that is red with a
    leak and green without. weir-scan's commands stay network-free, and weir
    never imports the runner.

weir-run 0.1.0

Choose a tag to compare

@IdoGol24 IdoGol24 released this 21 Oct 15:30
114c53e

weir-run 0.1.0, the canary runner, released beside weir-scan 0.5.0 (already on PyPI). Install: pip install weir-scan weir-run.

The canary runner is its own package, weir-run, released beside weir-scan
0.5.0 and requiring weir-scan>=0.5.0. Its versions are recorded here,
under weir-run headings.

Added

  • The canary runner, a separate package. It plants unique fake canaries
    (env and code placements), runs a program's flows under an empty
    environment plus an allowlist against a mock bound to 127.0.0.1 (an
    OpenAI-compatible model API with scripted tool calls, OAuth endpoints
    whose issued tokens become mock-oauth placements, an OTLP/HTTP receiver
    writing OTLP/JSON lines, the program's own declared APIs, and refusal of
    every proxied request to another host; the runner does not sandbox the
    network, so a client that ignores proxy settings can connect directly),
    relays flow output with every canary form redacted, and exits with the code of weir canary judge (docs/canary.md). Ships a
    subprocess-mode pytest plugin and a synthetic demo that is red with a
    leak and green without. weir-scan's commands stay network-free, and weir
    never imports the runner.

weir-scan 0.4.2

Choose a tag to compare

@IdoGol24 IdoGol24 released this 20 Oct 15:36
a250acb

Fixed

  • The exposure display no longer shows characters of a key's body. A
    hit is shown as the class prefix, ... and the last four characters. The
    prefix was cut at the last - or _ in the first 16 characters, and real
    OpenAI and Google key bodies contain both, so the display could show up to
    8 body characters of an OpenAI project key or 12 of a Google API key. The
    prefix now stops at the class's literal head (sk-proj-, sk-ant-api03-,
    ghp_, AKIA, AIza and the like).

Corpus

  • The release commit regenerated the diffspec baselines for the package
    version, which the baselines embed: weir_version moved from 0.4.1 to
    0.4.2. That diff is version-only; no flow fact, node count or finding moved.

weir-scan 0.4.1

Choose a tag to compare

@IdoGol24 IdoGol24 released this 15 Oct 09:51
d35d18d

Fixed

  • The credential_field key=value detector now sees assignments after an
    escaped line break.
    Tool arguments arrive as JSON-encoded text, so line two
    of a written .env file or script sits right after a literal \n (or \t,
    \r), and that escape's letter defeated the key-name boundary: no
    assignment past the first line reached the review queue. A JSON \n, \t
    or \r escape now counts as a boundary on both sides: it starts a key name
    and ends the value before it, so a value is measured as decoded, the next
    line is no longer folded into it, and a masked API_KEY=******** on line two
    is still dropped. A key name glued to any other letter (mytoken=) still
    does not match.
  • Every key-shaped detector now sees a value after an escaped line break.
    github_token, openai_api_key, anthropic_api_key, aws_access_key_id,
    google_api_key and financial_account_identifier opened with a word
    boundary, which read the escape's letter and the value as one word: a key or
    IBAN on line two of JSON-encoded tool arguments or tool results was never
    labeled: no exposure-scan hit for a key, no taint finding for any class. Each
    now opens with (?:\b|(?<=\\[ntr])), a word boundary or a JSON \n, \t
    or \r escape; a value already ended at a following escape. A value glued
    to any other letter (xghp_...) still does not match, and sk-ant- still
    outranks sk- at one offset.

Corpus

  • The diffspec baselines were regenerated for the catalog digest only; no flow
    fact, node count or finding moved. The attribute-exposure capture gains a
    Write tool-call pair: a .env line-two assignment that must reach the review
    queue, and a glued-identifier near-miss that must not. It also gains a key
    file with one value of each key-shaped class after a \n, \t or \r
    escape, and the same values glued to a letter as the near-miss; the diffspec
    baselines moved with the catalog digest again, digest only.
  • The release commit also regenerated the diffspec baselines for the package
    version, which the baselines embed: weir_version moved from 0.4.0 to
    0.4.1. That diff is version-only; no flow fact, node count or finding moved.

weir-scan 0.4.0

Choose a tag to compare

@IdoGol24 IdoGol24 released this 07 Sep 16:34

Full Changelog: v0.3.0...v0.4.0

v0.3.0

Choose a tag to compare

@IdoGol24 IdoGol24 released this 31 Aug 20:50

Provenance evidence tier + execute_tool span-split fix, node-ordering fix. See CHANGELOG.

v0.2.1

Choose a tag to compare

@IdoGol24 IdoGol24 released this 31 Aug 19:58

Fixing tait bug. support for LangChain

Full Changelog: v0.2.0...v0.2.1

weir-scan 0.2.0

Choose a tag to compare

@IdoGol24 IdoGol24 released this 25 Aug 13:29

pre-lunching

weir-scan 0.1.0

Choose a tag to compare

@IdoGol24 IdoGol24 released this 20 Aug 08:21

First public cut. See CHANGELOG.md for what ships: the OTel GenAI adapter with its 18-row reject-vs-degrade contract, evidence-tiered joins, the coverage gauge with its capability ladder, weir scan with witness-path findings, and the calibrated corpora behind all of it.