BadReasoner

This repository is for our new work: "BadReasoner: Planting Tunable Overthinking Backdoors into Large Reasoning Models for Fun or Profit", feel free to propose your issues!! 😎

Abstract

Large reasoning models (LRMs) have emerged as a significant advancement in artificial intelligence, representing a specialized class of large language models (LLMs) designed to tackle complex reasoning tasks. The defining characteristic of LRMs lies in their extensive chain-of-thought (CoT) reasoning capabilities. In this paper, we identify a previously unexplored attack vector against LRMs, which we term "overthinking backdoors". We advance this concept by proposing a novel tunable backdoor, which moves beyond simple on/off attacks to one where an attacker can precisely control the extent of the model's reasoning verbosity. Our attack is implemented through a novel data poisoning methodology. It pairs a tunable trigger—where the number of repetitions signals the desired intensity—with a correspondingly verbose CoT response. These responses are programmatically generated by instructing a teacher LLM to inject a controlled number of redundant refinement steps into a correct reasoning process. The approach preserves output correctness, which ensures stealth and establishes the attack as a pure resource-consumption vector. Extensive empirical results on various LRMs demonstrate that our method can reliably trigger a controllable, multi-fold increase in the length of the reasoning process, without degrading the final answer's correctness.

Getting Started

Environment

requirements.txt is available:

pip install -r requirements.txt 

Preparing Datasets

We use the released datasets on huggingface, you can get to know the detailed information in the datasets directory. The overthink samples we use are available in the sample-cot directory. Main trigger strength is set as 1 or 2. You can get your own overthink samples using sample.py in dos-sample directory.

cd dos-sample
python sample.py -num 3

Tunable Backdoor

We will introduce the tunable backdoor. (Framework of BadReasoner is shown below.)

Supervised Fine-Tuning

Put the json file with overthink samples into the sample-cot directory. Then, you can finetune the model

bash sft.sh

The parameters can be adjusted according to the requirements.

Model Test after Fine-Tuning

You can use the clean dataset or the poisoned dataset for fine-tuning and conduct tests to verify the behavior of the tunable backdoor. Specific test parameters can be directly set in the script.

python dos_test.py

Judgement via LLMs

After obtaining the test samples with overthink behavior, they can be copied to the dos-sample directory for judgement vis LLMs.

python judge.py --file_path yourfile

❤️Acknowledgments

Our code of fine tuning is based on the work of BoT.