Skip to content

v0.3.22

Latest

Choose a tag to compare

@codelion codelion released this 18 Jul 12:38
· 2 commits to main since this release
c8cbdde

What's Changed

Fix: autothink, thinkdeeper and deepconf for transformers >= 5 (#320)

Since transformers 5, apply_chat_template(return_tensors="pt") returns a BatchEncoding rather than a plain tensor, so passing the result straight to model(input_ids=...) raised. The tensor is now unwrapped via .input_ids:

  • optillm/thinkdeeper.py
  • optillm/autothink/processor.py
  • optillm/deepconf/processor.py

The unwrap is chained with the existing device transfer (.input_ids.to(self.model.device)) so tokens still land on the model's device — Tensor.to() returns a new tensor rather than mutating in place, so the assignment matters on CUDA/MPS.

Thanks to @jacquerie for diagnosing and reporting this.

README: dropped the GitHub stars badge and switched the downloads badge to pepy.tech, which reports actual download counts.

Full Changelog: v0.3.21...v0.3.22