Response Attack is one of the strongest jailbreak methods for LLMs to date.
If you are interested in our work, please star ⭐ our project, Thx 💕
Response Attack (RA) is a novel in-context jailbreak method that exploits contextual priming in large language models (LLMs). Inspired by psychological priming, it injects mildly harmful responses into earlier conversation turns. This primes the model to produce unsafe outputs at the final step—even when the actual malicious query would have been rejected on its own.
Figure: Overview of the Response Attack (RA) Method.
-
[2025/7/7] Released the priming dialogues used to attack HarmBench and AdvBench-50, along with the evaluation code.
Full attack generation and code will be released upon paper acceptance. -
[2025/7/8] We've released our paper: https://arxiv.org/abs/2507.05248.
-
[2025/10/18] If you need RA’s results on JailbreakBench, or the full RA code to use as a baseline for your paper, please contact miaoziqi@pjlab.org.cn.
- Contextual Priming Vulnerability: Demonstrates that prior assistant responses can unconsciously influence subsequent unsafe completions.
- High Success Rate: Achieves up to 94.8% ASR (Attack Success Rate), outperforming strong baselines like ActorAttack, CodeAttack, and PAIR.
- Low Interaction Cost: Requires only a single final query after the priming context is injected.
- Context-Aware Safety Dataset: Released 3k+ fine-tuning samples that effectively reduce jailbreak risk while preserving model utility.
📝 Note: The results below are obtained by running each harmful query with *three different priming dialogues per attack.
If you're comparing under a single-interaction setting (i.e., only one prompt per query), you can refer to the one-shot results in Table 6 of our paper, or simply evaluate using the first priming dialogue provided for each query in this repo.
| Metric | RA-DRI (Ours) | RA-SRI (Ours) | ActorAttack | CodeAttack | ReNeLLM |
|---|---|---|---|---|---|
| Avg. ASR (%) | 94.8 | 89.1 | 82.8 | 69.8 | 63.4 |
Table: Average Attack Success Rate (%) on HarmBench across GPT-4, Gemini-2.5, LLaMA-3-70B, DeepSeek, QwQ, etc.
| Method | GPT-4o | LLaMA-3-8B | LLaMA-3-70B | Gemini-2.5-Flash |
|---|---|---|---|---|
| RA-DRI | 98.0 | 92.0 | 90.0 | 100.0 |
| RA-SRI | 96.0 | 72.0 | 72.0 | 100.0 |
Table: Attack Success Rate (%) of RA on AdvBench-50 across representative models.
This section provides a brief overview of how to run attacks and evaluate results using our released code and data.
We will release the code to generate RA (DRI/SRI) priming dialogues soon.
# Coming soon...
“Inference” refers to injecting a crafted priming dialogue into the target model, followed by a final malicious query. The model's response is used to assess whether the attack succeeded.
- For closed-source models (e.g., GPT-4, Gemini), we use the official OpenAI API.
- For open-source models, we use VLLM for efficient batch inference.
You can run inference with the following script:
python generate_model_response.py \
--input_dir data/dialogues/harmbench/dri \
--model_name gpt-4o \
--max_workers 10 \
--include_v2v3--input_dir: Directory containing prompt JSON files (e.g., one per harmful query).--model_name: Name of the target model (e.g.,gpt-4o,gemini-2.0-flash, or any VLLM-hosted model).--max_workers: Number of concurrent workers for parallel inference.--include_v2v3: If set, runs attacks on up to three priming dialogues per query (.json,_v2.json,_v3.json) if available.
If not set or alternative versions are missing, only the base.jsonfile will be used.
We follow the evaluation protocol described in the paper.
To run evaluation:
bash eval/example.shIf you find our project is helpful, please cite our paper as
comming soon...