Repository navigation
Releases: IdoGol24/weir
Release list
v0.6.0: weir-scan 0.6.0 and weir-run 0.2.0
weir-scan 0.6.0 and weir-run 0.2.0. Install: pip install weir-scan weir-run, then try weir canary judge --sample.
weir-scan 0.6.0
Added
- The judge reports whether the network was isolated.
canary-ledger/1gains anetworkrecord (isolated,required) and a
directrecord (flow,protocol,target,attempts) for each
connection attempt isolation blocked. Every report carries a network
line after the coverage line,network isolated: N direct connections blockedornetwork not isolated: direct connections were not observed, andcanary-report/1anetworkobject. Two incomplete
reasons are new:network_not_isolated(isolation was required and
missing) anddirect_connection(a flow tried to reach a host around the
mock). A ledger with nonetworkrecord is reported as not isolated. - Outbound requests are a sink (
weir canary judge --requests). The judge reads a request capture (canary-requests/1,
one line per request the mock received: destination, method, path,
query, headers and body, and whether it was served, refused or a
CONNECT) and searches every request for every planted value in every
encoded form. A value in a credential field of a request to the
placement's own provider (a ledger placement's new optionalproviders
endpoint ids andprovider_host; each request record names its
credential_fields) isrequestoregressevidence that it was
exercised; anywhere else, the rest of that request included, it is a
leak, reported withsignal=requests,
the request'ssink(model,oauth,otlp, a declared api's id,
unmocked, or<host> (blocked)for a refused plain-HTTP request) and
the field (headers.<name>,query.<name>,body.<json path>,
body.<form field>,path,host,method, or the wholequeryor
body). A refusedCONNECThas no visible content: it is listed as a
blocked connection attempt and counted in the coverage, never a leak by
itself. The report gains the hit fieldsink, ablockedlist and the
coverage countsrequest_files,requests,requests_refusedand
connections_blocked; the text coverage line gains a requests segment
when a request capture is given. A request capture line that is not a
request record, or a body the mock could not read in full, makes the run
incomplete (unparsed_record). The schema artifact
canary-requests.v1.schema.jsonis new;canary-ledger.v1and
canary-report.v1gain the fields above. The judge stays offline and
deterministic.
Changed
- The printing rule withholds runs of 6 characters, down from 8. No
report prints a run of 6 or more consecutive characters of any planted
value, and a ledger id may not hold one. - The PyPI summary names every sink the judge searches. It reads
"Leak tests for AI agents: deterministically find planted credentials
in your agent's telemetry, model requests and outbound calls."
Corpus
- The release commit regenerated the diffspec baselines for the package
version, which the baselines embed:weir_versionmoved from 0.5.0 to
0.6.0. That diff is version-only; no flow fact, node count or finding
moved.
weir-run 0.2.0
Released beside weir-scan 0.6.0 and requiring weir-scan>=0.6.0.
Added
- Flows run where the only network is loopback. On Linux,
weir-run runand the pytest plugin run the mock and every flow in an
unprivileged user and network namespace with no route out, so a client
that ignores the proxy settings reaches nothing. Each blocked attempt is
seen and reported (tcpandudpby address and port,dnsby name),
and a blocked datagram's payload, a DNS query included, is swept capture,
so a canary in it is a leak. Where the namespace cannot be made (not
Linux, user namespaces disabled or restricted), the run goes ahead
unisolated and says why.--require-isolation
(--canary-require-isolation) makes an unisolated run incomplete, exit
3. weir-run now requiresweir-scan>=0.6.0, whose judge reads the new
ledger records. - Every request the mock receives is recorded. The mock
writes each request, refused ones included, tocapture/requests.jsonl
before answering it, and the runner passes it to the judge with
--requestsand deletes it after the run with the rest of the capture.
A placement names its own provider with the newproviderkey (model,
oauth,otlpor a[[mock.endpoint]]id;provider=in the pytest
plugin), so a key pasted into a prompt, sent to the model API as a
credential, sent to the wrong one of the program's APIs or sent in a
plain-HTTP request to another host is a leak. A token the mock's OAuth
endpoints issue belongs to them and to every declared endpoint. - A warning for each placement without a provider. Before
any flow runs,weir-run runand the pytest plugin printplacement <id> has no provider: any request holding it counts as a leakto
stderr for every placement with neitherprovidernor
provider_host. It names the id, never a value. - An Anthropic Messages API mock. Model APIs can now be
OpenAI-compatible or Anthropic Messages API. Beside the
OpenAI-compatible APIs, the mock servesPOST /v1/messages, plain and
streaming (message_start,ping,content_block_start,
content_block_delta,content_block_stop,message_delta,
message_stop), on the same script: with tools offered, turn one
answers with atool_useblock for every tool, and once the last user
message carriestool_resultblocks it answersmodel_reply. A query
such as?beta=true, body fields it does not know andsystementries
between turns are accepted, and the model API answersHEADandGET /api/hellowith200. Flows getANTHROPIC_BASE_URLpointing at the
mock, the variable the Anthropic SDKs read their base URL from; a
flow'senvmay no longer set it. The model API now always declares
thex-api-keyheader as a credential field: a placement with
provider = "model"sent there is exercised, the same value in a
prompt or a tool result is a leak tomodel, and any other value in
x-api-keyisunexpected.model.x-api-keyand makes the run
incomplete.
Changed
- A placement without
providerbelongs in no request. Any
request that holds its value is now a leak, where a credential field
holding it used to count as exercised wherever it was sent. Add
providerto each placement that a flow legitimately sends to a mocked
endpoint. The runner refuses a weir-scan whose judge has no--requests. - The mock no longer writes
requestoregressevidence.
The judge derives both from the request capture, so the evidence and
the leak verdict come from one place. - The PyPI summary describes the rig. It reads "Leak tests for AI
agents: plant fake credentials, run your agent in an isolated rig, and
capture everything it sends out."
v0.5.0: weir-scan 0.5.0 and weir-run 0.1.0
weir-scan 0.5.0 and weir-run 0.1.0. Install: pip install weir-scan weir-run, then try weir canary judge --sample.
weir-scan 0.5.0
Added
weir canary judge: did a planted fake credential reach the
telemetry?weir canary judge --ledger FILE --otlp PATH [--json] [PATH...]reads acanary-ledger/1ledger (placement ids plus the fake
values planted there) and the captured telemetry, and reports every
placement whose value, or an encoded form of it, appears anywhere in the
capture, naming the placement, file, line, record and location, never a
value or a fragment of one. The contract isdocs/canary.md. This first
version ships the placement kindsenv,codeandmock-oauth; the
evidence typesrequest,mintedandegress; the formsraw,json,
json2,url,base64,base64url(the base64 core at byte offsets 0,
1 and 2) andhex; and the incomplete reasonsrequired_not_exercised,
flow_not_ok,unexpected_credential,not_scanned,unparsed_record
andempty_capture.--otlpfiles get a structured walk of every span,
log record and metric (every string,stringValueand decoded
bytesValue) plus a raw byte sweep; other paths, such as a flow's stdout
and stderr, get the raw sweep only. Exit 0 clean, 1 leak, 2 unusable
input, 3 incomplete.--jsonemits thecanary-report/1report; both
schemas are inpackages/weir/schema_artifacts/. The judge opens no
socket, starts no process and reads no clock.weir canary judge --sample: a leak report in one command. It judges
a bundled sample (a ledger of fake values and an OTLP/JSON capture with
one planted value in a span attribute) with no other input, offline, and
exits 1 naming the placement, span and attribute, never the value.weir inventory(experimental): which credentials an agent could reach,
by name only. Its output and JSON schema may change without notice. Four
sources: a versioned list of standard credential locations under the given
home (~/.aws, Google application default credentials,
~/.npmrc,~/.kube/config,~/.docker/config.json,~/.netrc, git
credentials, gh and more, including the variables that move them); the
project tree, through Weir's catalog plus imported gitleaks or trufflehog
results; the launch environment (--env-fileor--process-env) with the
shell startup files that re-export into it; and the approved map
.weir/reach.yml. Environment variables are secrets by name or value
shape, not secrets when the value is a number, boolean, path or plain
hostname, and otherwise unknown: a count on screen, names in a file, a
failure only with--strict, markable as fine in the map. Files a
.gitignorecovers are reported per route (hidden from Grep's directory
search, still reachable by Read, cat and grep -r). Nothing prints, logs or
stores a value;--storewrites HMAC-SHA256 fingerprints under a local
key plus byte offsets.--jsonemits theweir-inventory/1report, whose
schema and Python API are documented indocs/inventory.md.--homeis
required: there is no default home.
Corpus
- The release commit regenerated the diffspec baselines for the package
version, which the baselines embed:weir_versionmoved from 0.4.1 to
0.5.0 (main never carried 0.4.2, which shipped fromrelease/0.4.x). That
diff is version-only; no flow fact, node count or finding moved.
weir-run 0.1.0
The canary runner is its own package, weir-run, released beside weir-scan
0.5.0 and requiring weir-scan>=0.5.0. Its versions are recorded here,
under weir-run headings.
Added
- The canary runner, a separate package. It plants unique fake canaries
(envandcodeplacements), runs a program's flows under an empty
environment plus an allowlist against a mock bound to127.0.0.1(an
OpenAI-compatible model API with scripted tool calls, OAuth endpoints
whose issued tokens becomemock-oauthplacements, an OTLP/HTTP receiver
writing OTLP/JSON lines, the program's own declared APIs, and refusal of
every proxied request to another host; the runner does not sandbox the
network, so a client that ignores proxy settings can connect directly),
relays flow output with every canary form redacted, and exits with the code ofweir canary judge(docs/canary.md). Ships a
subprocess-mode pytest plugin and a synthetic demo that is red with a
leak and green without. weir-scan's commands stay network-free, and weir
never imports the runner.
weir-run 0.1.0
weir-run 0.1.0, the canary runner, released beside weir-scan 0.5.0 (already on PyPI). Install: pip install weir-scan weir-run.
The canary runner is its own package, weir-run, released beside weir-scan
0.5.0 and requiring weir-scan>=0.5.0. Its versions are recorded here,
under weir-run headings.
Added
- The canary runner, a separate package. It plants unique fake canaries
(envandcodeplacements), runs a program's flows under an empty
environment plus an allowlist against a mock bound to127.0.0.1(an
OpenAI-compatible model API with scripted tool calls, OAuth endpoints
whose issued tokens becomemock-oauthplacements, an OTLP/HTTP receiver
writing OTLP/JSON lines, the program's own declared APIs, and refusal of
every proxied request to another host; the runner does not sandbox the
network, so a client that ignores proxy settings can connect directly),
relays flow output with every canary form redacted, and exits with the code ofweir canary judge(docs/canary.md). Ships a
subprocess-mode pytest plugin and a synthetic demo that is red with a
leak and green without. weir-scan's commands stay network-free, and weir
never imports the runner.
weir-scan 0.4.2
Fixed
- The exposure display no longer shows characters of a key's body. A
hit is shown as the class prefix,...and the last four characters. The
prefix was cut at the last-or_in the first 16 characters, and real
OpenAI and Google key bodies contain both, so the display could show up to
8 body characters of an OpenAI project key or 12 of a Google API key. The
prefix now stops at the class's literal head (sk-proj-,sk-ant-api03-,
ghp_,AKIA,AIzaand the like).
Corpus
- The release commit regenerated the diffspec baselines for the package
version, which the baselines embed:weir_versionmoved from 0.4.1 to
0.4.2. That diff is version-only; no flow fact, node count or finding moved.
weir-scan 0.4.1
Fixed
- The
credential_fieldkey=value detector now sees assignments after an
escaped line break. Tool arguments arrive as JSON-encoded text, so line two
of a written.envfile or script sits right after a literal\n(or\t,
\r), and that escape's letter defeated the key-name boundary: no
assignment past the first line reached the review queue. A JSON\n,\t
or\rescape now counts as a boundary on both sides: it starts a key name
and ends the value before it, so a value is measured as decoded, the next
line is no longer folded into it, and a maskedAPI_KEY=********on line two
is still dropped. A key name glued to any other letter (mytoken=) still
does not match. - Every key-shaped detector now sees a value after an escaped line break.
github_token,openai_api_key,anthropic_api_key,aws_access_key_id,
google_api_keyandfinancial_account_identifieropened with a word
boundary, which read the escape's letter and the value as one word: a key or
IBAN on line two of JSON-encoded tool arguments or tool results was never
labeled: no exposure-scan hit for a key, no taint finding for any class. Each
now opens with(?:\b|(?<=\\[ntr])), a word boundary or a JSON\n,\t
or\rescape; a value already ended at a following escape. A value glued
to any other letter (xghp_...) still does not match, andsk-ant-still
outrankssk-at one offset.
Corpus
- The diffspec baselines were regenerated for the catalog digest only; no flow
fact, node count or finding moved. The attribute-exposure capture gains a
Write tool-call pair: a.envline-two assignment that must reach the review
queue, and a glued-identifier near-miss that must not. It also gains a key
file with one value of each key-shaped class after a\n,\tor\r
escape, and the same values glued to a letter as the near-miss; the diffspec
baselines moved with the catalog digest again, digest only. - The release commit also regenerated the diffspec baselines for the package
version, which the baselines embed:weir_versionmoved from 0.4.0 to
0.4.1. That diff is version-only; no flow fact, node count or finding moved.
weir-scan 0.4.0
v0.3.0
v0.2.1
Fixing tait bug. support for LangChain
Full Changelog: v0.2.0...v0.2.1
weir-scan 0.2.0
weir-scan 0.1.0
First public cut. See CHANGELOG.md for what ships: the OTel GenAI adapter with its 18-row reject-vs-degrade contract, evidence-tiered joins, the coverage gauge with its capability ladder, weir scan with witness-path findings, and the calibrated corpora behind all of it.