DyCon: Dynamic Reasoning Control via Evolving Difficulty Modeling

Tengyao Tu^1,2*,  Yulin Li^1*,  Huiling Zhen³,  Libo Qin¹,  Zhoujun Wei⁴,  Jinghua Piao^2,5,  Zhuotao Tian^1,4†, 
Yong Li^2,5,  Min Zhang^1,4
1 Harbin Institute of Technology (Shenzhen)     ² Zhongguancun Academy     ³ Huawei Noah's Ark Lab
⁴ Shenzhen Loop Area Institute     ⁵ Tsinghua University    
^*Equal Contribution     ^†Corresponding Author
     

📚 TABLE OF CONTENTS

1. News
2. Why DyCon
3. Motivation
4. Method
5. TODO
6. Quick Start
7. Acknowledgements
8. Citation

🎉 News

• [2026.06.08] We release the DyCon regressors for DeepSeek-R1-Distill-Qwen-7B, Qwen3-4B-Thinking-2507, Qwen3-14B, and QwQ-32B under outputs/*/Math_Regress/remain_reg/.
• [2026.05.18] Our paper has been accepted by ICML 2026🎖️.

🏆 Why DyCon

1. Difficulty modeling should align with trajectories rather than solely the initial question. Existing methods typically rely on static difficulty estimations, handcrafted confidence heuristics, or external evaluators. In contrast, DyCon adaptively monitors the model's evolving internal state at each reasoning step, enabling simpler trajectories to terminate early and allowing more complex ones to continue as needed.
2. Training-free efficiency with accuracy preserved. DyCon fits a lightweight difficulty estimator from hidden step embeddings and keeps the original LRM parameters untouched. Across four backbones from 4B to 32B and twelve benchmarks spanning math, scientific QA, commonsense QA, knowledge QA, and coding, DyCon reduces redundant tokens while maintaining or even improving answer accuracy.

🎯 Motivation

1. Problem difficulty evolves dynamically during reasoning. Initially challenging problems may become simpler as the model decomposes them or conversely grow more complex if the reasoning trajectory deviates. Our analysis shows an overall decreasing trend in self-assessed difficulty, with fluctuations during the reasoning process across multiple model families, indicating that a static pre-generation difficulty score is insufficient.
2. LRMs already encode the evolving difficulty. Step embeddings at reasoning boundaries linearly predict normalized remaining reasoning length with strong held-out $R^2$ scores.

🌈 Method

1. Explicit modeling of evolving difficulty. DyCon first conducts offline reasoning on a small seen dataset to extract hidden states at each reasoning step boundary and records the remaining reasoning length. This remaining length is log-transformed and min-max normalized to yield a bounded, step-level difficulty target. A Ridge regression then maps step embeddings from a selected layer to this normalized difficulty, where automatic layer selection is achieved using validation ($R^2$).
2. Difficulty-aware dynamic reasoning control. During inference, DyCon predicts the current step difficulty online and uses it to softly adjust reflection-trigger logits. Low predicted difficulty strengthens suppression and encourages convergence; high predicted difficulty weakens suppression so the model can continue necessary reflection. This is a soft intervention rather than hard termination, preserving normal decoding behavior on challenging samples.

🔥 TODO

• Initialize Project.
• Release the DyCon regressors.

🚀 Quick Start

Easy Reproduction with Released Regressors

To quickly run DyCon without fitting a new estimator, reuse one of the released regressor checkpoints:

DyCon/
├── transformer_inference_regressor_soft_center.py
├── outputs/
│   ├── DeepSeek-R1-Distill-Qwen-7B/Math_Regress/remain_reg/
│   │   ├── remain_reg.joblib
│   │   └── remain_reg_meta.json
│   ├── Qwen3-4B-Thinking-2507/Math_Regress/remain_reg/
│   │   ├── remain_reg.joblib
│   │   └── remain_reg_meta.json
│   ├── Qwen3-14B/Math_Regress/remain_reg/
│   │   ├── remain_reg.joblib
│   │   └── remain_reg_meta.json
│   └── QwQ-32B/Math_Regress/remain_reg/
│       ├── remain_reg.joblib
│       └── remain_reg_meta.json
└── Data/

Step 1. Run DyCon inference

Replace --model_name_or_path with the corresponding local checkpoint path or Hugging Face model id in your environment.

CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 \
python -u transformer_inference_regressor_soft_center.py \
  --model_name_or_path "Qwen3-4B-Thinking-2507" \
  --dataset_dir "./Data" \
  --dataset "Math_AIME2025" \
  --output_path "./outputs_dycon" \
  --num_gpus 8 \
  --trust_remote_code \
  --max_new_tokens 81920 \
  --reg ./outputs/Qwen3-4B-Thinking-2507/Math_Regress/remain_reg/remain_reg.joblib \
  --reg_meta ./outputs/Qwen3-4B-Thinking-2507/Math_Regress/remain_reg/remain_reg_meta.json

Step 2. Merge multi-GPU shards

python merge_shards.py \
  --dir ./outputs_dycon/Qwen3-4B-Thinking-2507/Math_AIME2025 \
  --base 'results'

Step 3. Evaluate merged outputs

python check.py \
  --model_name_or_path "Qwen3-4B-Thinking-2507" \
  --data_name "Math_AIME2025" \
  --generation_path "./outputs_dycon/Qwen3-4B-Thinking-2507/Math_AIME2025/results.merged.jsonl"

Fit Your Own Difficulty Estimator

DyCon can fit a new estimator from a small seen dataset. The default example below uses Math_Regress, but the same pipeline can be applied to another dataset folder following the repository's Data/<dataset>/test.jsonl layout.

Step 1. Extract Hidden States and Remaining Thinking Length

This step runs inference with the thinking model, extracts hidden states at reasoning step boundaries, and records the remaining thinking length after each step. The saved hidden states and remain_length values are used to construct the regression dataset for later fitting.

CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 \
python -u transformer_inference_dp_save_all_steps.py \
  --model_name_or_path 'Qwen3-4B-Thinking-2507' \
  --dataset_dir './Data/' \
  --dataset 'Math_Regress' \
  --output_path './outputs' \
  --num_gpus 8 \
  --trust_remote_code \
  --save_step_hs \
  --max_new_tokens 81920 \
  --hs_device auto

Step 2. Fit the DyCon Remaining Thinking Length Regressor

This step fits the DyCon regressor, which predicts the remaining thinking length from the hidden states extracted in Step 1. Each saved reasoning step is treated as one fitting sample: the input is the hidden state at a reasoning step boundary, and the target is the corresponding remain_length. With --k -1, all extracted reasoning steps are used; with --layer -1, the script automatically searches over all available layers and selects the best layer based on validation performance. The --norm_mode log option applies log1p transformation followed by min-max normalization, making the remaining-length target smoother and easier for the Ridge regressor to fit.

python train_remain_length_reg_auto_layer.py \
  --folder "./outputs/Qwen3-4B-Thinking-2507/Math_Regress/" \
  --k -1 \
  --layer -1 \
  --norm_mode log

The fitted estimator is saved as:

outputs/Qwen3-4B-Thinking-2507/Math_Regress/remain_reg/
├── remain_reg.joblib
└── remain_reg_meta.json

Step 3. Run DyCon with the Fitted Regressor

This step runs DyCon inference using the remaining-thinking-length regressor fitted in Step 2. During generation, DyCon reads the hidden state at each reasoning step boundary, predicts the normalized remaining thinking length, and uses this prediction as a dynamic control signal to softly adjust the model's reasoning behavior. Intuitively, when the regressor predicts that the model is still far from finishing, DyCon allows more reasoning; when the predicted remaining thinking length becomes small, DyCon gradually encourages termination and reduces redundant overthinking.

The --reg argument specifies the fitted regressor checkpoint, while --reg_meta provides the metadata needed to correctly load the regressor, including the selected layer and normalization mode. The dataset can be changed to any evaluation split, such as Math_AIME2024, Math_AIME2025, or Math500, while reusing the same regressor fitted on Math_Regress.

CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 \
python -u transformer_inference_regressor_soft_center.py \
  --model_name_or_path "Qwen3-4B-Thinking-2507" \
  --dataset_dir "./Data" \
  --dataset "Math_AIME2025" \
  --output_path "./outputs_dycon" \
  --num_gpus 8 \
  --trust_remote_code \
  --max_new_tokens 81920 \
  --reg ./outputs/Qwen3-4B-Thinking-2507/Math_Regress/remain_reg/remain_reg.joblib \
  --reg_meta ./outputs/Qwen3-4B-Thinking-2507/Math_Regress/remain_reg/remain_reg_meta.json

Step 4. Merge multi-GPU shards

python merge_shards.py \
  --dir ./outputs_dycon/Qwen3-4B-Thinking-2507/Math_AIME2025 \
  --base 'results'

Step 5. Evaluate results

python check.py \
  --model_name_or_path "Qwen3-4B-Thinking-2507" \
  --data_name "Math_AIME2025" \
  --generation_path "./outputs_dycon/Qwen3-4B-Thinking-2507/Math_AIME2025/results.merged.jsonl"

❤️ Acknowledgements

Our work builds upon the codebase of ReBalance, DeepSeek-R1-Distill-Qwen, Qwen3, and QwQ. We sincerely thank the authors for their remarkable contributions.

🙏 Citation

If you find DyCon useful in your research, please cite our paper:

@inproceedings{tu2026dycon,
  title={DyCon: Dynamic Reasoning Control via Evolving Difficulty Modeling},
  author={Tu, Tengyao and Li, Yulin and Zhen, Huiling and Qin, Libo and Wei, Zhoujun and Piao, Jinghua and Tian, Zhuotao and Li, Yong and Zhang, Min},
  booktitle={Proceedings of the International Conference on Machine Learning},
  year={2026}
}