Replies: 1 comment
|
Thanks for checking before you publish. Your setup is fair, and all three reproduce on the 3.8.0 release too.
The loopback SSRF allowlist and the higher rate limit are fair edits. There's a fourth worth listing. You remap Happy to read the write-up before it goes out. |
Uh oh!
There was an error while loading. Please reload this page.
Hi, I maintain Bollard (github.com/tsushanth/bollard), an open-source egress firewall for AI agents. I built a small harness that replays 36 agent traces (20 benign, 16 prompt-injected) through real proxies on loopback, and I ran Pipelock through it. Before publishing anything I'd like you to check that I configured and read it fairly.
Setup: Built from pipelab-org/pipelock main at d97bee5 (version string 0.0.0-dev). I used the shipped strict and balanced presets with these edits: allowlist set to the test hosts, rate limit raised,
ssrf.ip_allowlistset to loopback. TLS interception is off except in one labelled pass. Pipelock receives no tool-provenance data, and each task gets a fresh process.Three results I'd like you to sanity-check (raw responses are in
bench/results/pipelock.json):authority_mismatch/taint_policy, even with no secret in the body. Is that intended for plain HTTP traffic?body_entropyblocks a plain 41-character JSON chat message, a 40-hex git sha and a UUID. Is there a recommended way to tune this for agent traffic?In the other direction, Pipelock's DLP caught URL, header and encoded-token cases that Bollard misses, and I'll publish those numbers too.
Reproduce: https://github.com/tsushanth/bollard/tree/bench-harness/bench (see "Drivers" in the README;
python3 -I bench/run_bench.py --driver pipelock --pipelock-bin <your build> --pipelock-src <your checkout>). I plan to publish in about two weeks and will correct anything you flag before then. PRs to my harness config are welcome. Thanks for building it.All reactions