Add HarmBench classifier detectors - #1725
Conversation
Implements HarmBench (Mazeika et al., 2024) as garak detectors, following the same judge-layer pattern established by the StrongREJECT rubric detector (PR NVIDIA#1717). Refs NVIDIA#973, which calls out HarmBench alongside StrongREJECT. The classifier is given a target behavior and a candidate generation, and emits a single Yes/No token indicating whether the generation exhibits the behavior. Two classes ship together, both with active = False since the underlying models are too large to auto-load by default; users opt in via explicit --detectors: - HarmBenchLlama2 (canonical): cais/HarmBench-Llama-2-13b-cls, the paper-numbers classifier (~26GB VRAM in bfloat16) - HarmBenchMistral7b: cais/HarmBench-Mistral-7b-val-cls, the smaller validation classifier (~14GB VRAM, MIT all the way down) Both load locally via AutoModelForCausalLM and parse a single generated token. Behavior text is read from attempt.notes["harmbench_behavior"] when present, with fallbacks to attempt.notes["original_prompt"] and the last user message in attempt.prompt. The prompt templates are byte-equivalent copies of upstream HarmBench: eval_utils.py for the Llama-2 template, and the cais/HarmBench-Mistral-7b-val-cls model card for the Mistral template. The MIT license text is reproduced verbatim in the module docstring per typical attribution practice. The generic test_detector_detect parametric test instantiates every detector and would attempt to download 26GB on each CI run. That does not fit GitHub Actions' ~10GB per-repo cache cap, so HarmBench classes are added to a new HEAVY_HF_DETECTORS skip list in tests/detectors/test_detectors.py. test_detector_structure and test_detector_metadata still run for both classes; behavioral coverage lives in tests/detectors/test_detectors_harmbench.py (24 cases) using mocked transformers loaders, so CI never downloads either model. Signed-off-by: precognitivem0nk <rextedgorman@gmail.com>
immu4989
left a comment
immu4989
left a comment
There was a problem hiding this comment.
Reviewing this alongside #1717 since same author and same family. Different abstraction story than that one though: #1717 subclasses ModelAsJudge and loads a generator via _plugins.load_plugin, while this is HFDetector-shaped and loads a transformers AutoModel directly. So the "shared judge base class" point I raised over there doesn't apply here. They're genuinely different patterns and that's fine.
Behavior text source and the milestone-12 refactor. The fallback chain attempt.notes["harmbench_behavior"] → attempt.notes["original_prompt"] → attempt.prompt.last_message().text is sensible, and the info-level log when the canonical key is missing is the right escape hatch. Worth surfacing for @jmartin-tech though: he flagged on #992 that the technique-and-intent milestone (specifically the work in #1434) is going to give detectors clearer access to per-attempt targeted goals, which sounds like exactly the source-of-truth this detector needs. Is it worth a brief acknowledgement in this PR that harmbench_behavior is the v1 plumbing and the canonical field is expected to migrate post-M12? Not asking to block; just keeping the two threads visible to each other.
Dual classes vs one configurable class. HarmBenchLlama2 and HarmBenchMistral7b share most of their implementation, with the meaningful divergence being the prompt template and the model footprint. Was the choice to ship two sibling classes (rather than one HarmBench base with detector_model_path and _prompt_template via DEFAULT_PARAMS) driven by something I'm missing? My instinct would be the configurable-base shape so adding a third variant later (the multimodal one in your out-of-scope list, for example) is one DEFAULT_PARAMS entry rather than a third subclass. I can see arguments the other way: explicit class names show up cleaner in --detectors listings and are easier for users to discover.
HEAVY_HF_DETECTORS skip list. Solid problem to solve, but a string-name allow-list in test_detectors.py means every future heavy detector needs to remember to update it. A class-level attribute on the detector itself (something like auto_test = False or estimated_model_size_gb) checked by the parametrize collector would put the responsibility where the model size actually lives. Not blocking; worth a thought before more heavy detectors land.
Smaller notes:
- Bundling the Llama-2 prompt template (not weights) under garak's Apache-2.0 license is probably fine since prompt templates aren't derivative works of model weights, but worth a sanity check from @jmartin-tech.
- The "out of scope: optional OpenAI-compatible remote serving knob" is the real accessibility gap. The Mistral variant at 14GB is on the edge of consumer hardware; the Llama-2 at 26GB is firmly cloud-only. A follow-up making the classifier addressable via API would meaningfully widen the user base. Not a blocker for this PR.
Things done well that I would otherwise have flagged. Explicit active = False with the MustContradictNLI precedent cited, per-variant licensing inline in the docstring, byte-equality discipline with the SHA256 verification, and an info-log fallback chain that fails informatively rather than silently. The CI economics consideration (HEAVY_HF_DETECTORS workaround aside) shows attention most contributors don't bring.
Looking forward to seeing both this and #1717 land.
leondz
left a comment
leondz
left a comment
There was a problem hiding this comment.
some restructuring will make sense
| return results | ||
|
|
||
|
|
||
| class HarmBenchMistral7b(HarmBenchLlama2): |
There was a problem hiding this comment.
two options:
- have just one class where the model is changeable
- have two separate classes that inherit from Detector and also a HarmBench mixin
mistral 7b is not a variant of llama2 :)
| "graceful_fail": True, | ||
| } | ||
|
|
||
| active = False # heavy local HF model; opt in via explicit --detectors |
| ``attempt.notes["harmbench_behavior"]`` if present, with fallbacks to | ||
| ``attempt.notes["original_prompt"]`` and the last user message in |
There was a problem hiding this comment.
this binds very closely to attempt formation. nothing in garak forms attempts like this, meaning HarmBench is never consumed.
what's the rationale behind the fallback?
attempts come with a goal attrib - recommend using that, or similar probe data, to populate the target harmbench behaviour
| Precedence: ``attempt.notes["harmbench_behavior"]`` > | ||
| ``attempt.notes["original_prompt"]`` > last user message in | ||
| ``attempt.prompt``. An info-level message is logged when | ||
| ``harmbench_behavior`` is absent so probe authors can wire it in over | ||
| time without breaking existing probes. |
There was a problem hiding this comment.
can you explain the justification for operating like this? see previous similar comment
| orig_loglevel = transformers_logging.get_verbosity() | ||
| transformers_logging.set_verbosity_error() | ||
|
|
||
| model_kwargs = self._gather_hf_params( | ||
| hf_constructor=AutoModelForCausalLM.from_pretrained | ||
| ) | ||
| # we drive generation manually; strip generation-specific kwargs that | ||
| # may have been pulled in via pipeline-signature merging | ||
| for param in ("max_new_tokens", "temperature", "do_sample", "top_p", "top_k"): | ||
| model_kwargs.pop(param, None) | ||
|
|
||
| self.detector_model = AutoModelForCausalLM.from_pretrained( | ||
| self.detector_model_path, **model_kwargs | ||
| ) | ||
| self.detector_tokenizer = AutoTokenizer.from_pretrained( | ||
| self.detector_model_path | ||
| ) | ||
|
|
||
| transformers_logging.set_verbosity(orig_loglevel) |
| Answer: [/INST]""" | ||
|
|
||
|
|
||
| class HarmBenchLlama2(Detector, HFCompatible): |
There was a problem hiding this comment.
why not inhering detectors.base.HFDetector? this does a lot of the legwork in __init__ - and if it's insufficient, improvements may be made there
| return attempt.notes["original_prompt"] | ||
| return attempt.prompt.last_message().text | ||
|
|
||
| def _classify(self, behavior: str, generation: str) -> float | None: |
There was a problem hiding this comment.
why does this need its own method?
|
Two of @leondz's points line up with concerns I raised in my own review six days ago (the dual class question and the behavior text fallback chain interacting with attempt structure). That convergence suggests the abstraction question isn't really specific to this PR. It's about how the broader judge/classifier family is shaping up across #1717 StrongREJECT, #1725 here, and #1773 MulticlassJudge. @leondz @jmartin-tech if it would be useful, I'm happy to open a separate design discussion issue laying out the convergent design pressure across the three PRs (shared generator loading, divergent prompt template/parse layers, attempt data plumbing). That conversation seems better suited to a dedicated issue than to fragment across three PR threads. No expectation it goes anywhere; just offering before @precognitivem0nk has to make scoping decisions in isolation. Either way, holding off on further input here so @precognitivem0nk can respond to @leondz's specific points first. |
3 participants
Add
harmbench.HarmBenchLlama2andharmbench.HarmBenchMistral7b, the HarmBench classifier-based evaluators from Mazeika et al. (2024), implemented as a sibling pattern to the StrongREJECT rubric detector (#1717). The classifier is given a target behavior plus a candidate generation and emits a single Yes/No token; the parser matches upstream exactly (literalyes/noafter strip+lower → 1.0/0.0/None).HarmBenchLlama2(canonical variant) wrapscais/HarmBench-Llama-2-13b-cls, the paper-numbers classifier (~26GB VRAM in bfloat16).HarmBenchMistral7bis a sibling subclass overcais/HarmBench-Mistral-7b-val-cls, the upstream validation classifier (~14GB VRAM). The Mistral variant is MIT all the way down (Apache-2.0 base, MIT fine-tune); the Llama-2 variant inherits Meta's Llama 2 Community License at the base layer, documented inline in the module docstring.Both classes ship with
active = Falsesince the model footprints are too large to auto-load by default. Users opt in via explicit--detectors detectors.harmbench.HarmBenchLlama2(or the Mistral variant). Mirrors the precedent set bymisleading.MustContradictNLI("this one is slow, skip by default").Both prompt templates are byte-equivalent copies of upstream HarmBench:
eval_utils.pyfor the Llama-2 template, and thecais/HarmBench-Mistral-7b-val-clsmodel card for the Mistral template. Byte equality is preserved so scores are reproducible against published HarmBench numbers. The MIT license text is reproduced verbatim in the module docstring per typical attribution practice.Behavior text is read from
attempt.notes["harmbench_behavior"]if present, with fallbacks toattempt.notes["original_prompt"]and the last user message inattempt.prompt. Probes that mutate prompts (FITD, SATA) lose the canonical phrasing if they don't set the notes key; an info-level log fires when it's missing so probe authors can opt in over time. v1 ships without probe-side edits.Test infra note: the generic
tests/detectors/test_detectors.py::test_detector_detectparametrizes over every detector and would attempt to download 26GB on each CI run. That does not fit GitHub Actions' ~10GB per-repo cache cap. To opt out, a newHEAVY_HF_DETECTORSskip list is added totest_detectors.pyand the two HarmBench classes are listed there.test_detector_structureandtest_detector_metadatastill run for both classes; behavioral coverage lives in the newtests/detectors/test_detectors_harmbench.py(24 cases) using mocked transformers loaders.Out of scope (deferred to follow-ups):
attempt.notes["harmbench_behavior"]plumbing infitd.pyandsata.pycais/HarmBench-Llama-2-13b-cls-multimodal-behaviors)Refs #973
Verification
python -m pytest tests/detectors/test_detectors_harmbench.py -v(24 passed)python -m pytest tests/detectors/test_detectors.py -v -k "harmbench"(4 passed, 2 skipped via HEAVY_HF_DETECTORS)eval_utils.pyand the Mistral HF model card verified at commit time