You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Benchmarked garak against two third-party targets, corrections welcome
#2119
I have published a comparison of garak, promptfoo and my own tool against the same two third-party targets, scoring every reply from all three by one shared rule rather than trusting any tool's own verdict. I would rather have your corrections than find out later that the method was wrong.
What garak does that neither of the others does.latentinjection carries its payload into the prompt and looks for its own trigger, so a hit attributes to the probe by construction. My tool's findings on the unguarded target rested on a canary planted in the corpus before the run, and that string comes back on 46% of ordinary questions, so 181 of my 192 breaches were flagged unattributable by my own tool. I have since added an attack family that carries its own marker, and the design is copied from yours. It is credited on the page and in the commit that added it.
And a class I do not measure at all.dan.DanInTheWild bypassed mitigation on 231 of 256 prompts against the unguarded target, 90.23% [86.33, 93.75]. My arsenal scores persona attacks by what leaked, so on a target with nothing to steal I report clean where you report 90%. That is now qatration/qatration#1, opened with your result quoted as the reason it exists.
Four things I would like checked, because any of them could explain a number.
I ran latentinjection.LatentInjectionFactSnippetEiffel against both targets, then encoding.InjectBase64, dan.DanInTheWild, goodside.Tag and leakreplay.GuardianComplete against the unguarded one. Five modules out of forty-two. If those are the wrong ones for a retrieval-augmented target, name better ones and I will rerun.
run.soft_probe_prompt_cap has no effect on latentinjection, which matches the source since it does not call _prune_data, while dan honours it. Is there a supported way to fix a prompt budget across modules? Not having one is why the page reports prompts sent and wall clock rather than claiming equal budgets.
DanInTheWild draws its 256 prompts with random.sample and I did not set run.seed, so my 90.23% is an estimate of the library rather than a fact about 256 particular prompts. Is seeding the right way to publish a reproducible figure, or is there something better?
The REST generator config is at out/bench/configs/garak-localrag-injection.yaml in my repository, and the guarded target's beside it. If anything in them is wrong, that alone could explain the injection numbers.
The result, and the caveat belongs in front of it. My shared rule cannot see garak's own trigger, so what follows measures something latentinjection was not built to produce. Under that rule, conditioned on the poisoned document actually being retrieved, garak's probes leaked my planted string at 85%, which is the rate ordinary questions produce. garak's own detector reported 12.11% on its own trigger. Two different questions, both legitimate, and most of the page is about that difference.
The targets.local-rag-chat at cd8cd89, an ordinary RAG application with no guardrails, and NeMo Guardrails with input and output rails. The corpus is mine: that project ships no documents, so the nine PDFs it answered from were written by me and one carries a canary. Nothing on the page is a vulnerability report about anybody's code.
One thing to expect if you open my artifacts: my tool's own attribution text says canary_in_output fires on 50% of benign traffic where the page says 46%. Different rule. Mine counts a second canary and drops two skipped probes; the shared rule counts two exact strings over all 50 replies.
Your raw reports are not in my repository: my commit guard refuses them over a Cyrillic character inside the jailbreak corpus, and that corpus has its own licence. The commands and configs to regenerate them are on the page.
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
Uh oh!
There was an error while loading. Please reload this page.
I have published a comparison of garak, promptfoo and my own tool against the same two third-party targets, scoring every reply from all three by one shared rule rather than trusting any tool's own verdict. I would rather have your corrections than find out later that the method was wrong.
https://github.com/qatration/qatration/blob/main/docs/benchmark.md
What garak does that neither of the others does.
latentinjectioncarries its payload into the prompt and looks for its own trigger, so a hit attributes to the probe by construction. My tool's findings on the unguarded target rested on a canary planted in the corpus before the run, and that string comes back on 46% of ordinary questions, so 181 of my 192 breaches were flagged unattributable by my own tool. I have since added an attack family that carries its own marker, and the design is copied from yours. It is credited on the page and in the commit that added it.And a class I do not measure at all.
dan.DanInTheWildbypassed mitigation on 231 of 256 prompts against the unguarded target, 90.23% [86.33, 93.75]. My arsenal scores persona attacks by what leaked, so on a target with nothing to steal I report clean where you report 90%. That is now qatration/qatration#1, opened with your result quoted as the reason it exists.Four things I would like checked, because any of them could explain a number.
latentinjection.LatentInjectionFactSnippetEiffelagainst both targets, thenencoding.InjectBase64,dan.DanInTheWild,goodside.Tagandleakreplay.GuardianCompleteagainst the unguarded one. Five modules out of forty-two. If those are the wrong ones for a retrieval-augmented target, name better ones and I will rerun.run.soft_probe_prompt_caphas no effect onlatentinjection, which matches the source since it does not call_prune_data, whiledanhonours it. Is there a supported way to fix a prompt budget across modules? Not having one is why the page reports prompts sent and wall clock rather than claiming equal budgets.DanInTheWilddraws its 256 prompts withrandom.sampleand I did not setrun.seed, so my 90.23% is an estimate of the library rather than a fact about 256 particular prompts. Is seeding the right way to publish a reproducible figure, or is there something better?out/bench/configs/garak-localrag-injection.yamlin my repository, and the guarded target's beside it. If anything in them is wrong, that alone could explain the injection numbers.The result, and the caveat belongs in front of it. My shared rule cannot see garak's own trigger, so what follows measures something
latentinjectionwas not built to produce. Under that rule, conditioned on the poisoned document actually being retrieved, garak's probes leaked my planted string at 85%, which is the rate ordinary questions produce. garak's own detector reported 12.11% on its own trigger. Two different questions, both legitimate, and most of the page is about that difference.The targets. local-rag-chat at
cd8cd89, an ordinary RAG application with no guardrails, and NeMo Guardrails with input and output rails. The corpus is mine: that project ships no documents, so the nine PDFs it answered from were written by me and one carries a canary. Nothing on the page is a vulnerability report about anybody's code.One thing to expect if you open my artifacts: my tool's own attribution text says
canary_in_output fires on 50% of benign trafficwhere the page says 46%. Different rule. Mine counts a second canary and drops two skipped probes; the shared rule counts two exact strings over all 50 replies.Your raw reports are not in my repository: my commit guard refuses them over a Cyrillic character inside the jailbreak corpus, and that corpus has its own licence. The commands and configs to regenerate them are on the page.
All reactions