Skip to content

fix: switch torch_dynamic and quanto saves to save_before_apply#696

Merged
begumcig merged 3 commits into
mainfrom
fix/switch-torch_dynamic-save-function-from-pickle-to-reapply
Jun 22, 2026
Merged

fix: switch torch_dynamic and quanto saves to save_before_apply#696
begumcig merged 3 commits into
mainfrom
fix/switch-torch_dynamic-save-function-from-pickle-to-reapply

Conversation

@gsprochette

@gsprochette gsprochette commented Jun 20, 2026

Copy link
Copy Markdown
Collaborator

Description

In transformers 5, llama architectures are not picklable because of the choice of kernel functions.
Both quanto and torch_dynamic were facing saving problems, and are switched to save_before_apply to preserve compatibility with (among other) compilation algorithms (compilation algorithms use save_before_apply -> the quantized model's save functions is called -> for quanto or torch_dynamic that would be torch.save -> error).

Related Issue

Fixes #(issue number)

Type of Change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Refactor (no functional change)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation update

Testing

  • I added or updated tests covering my changes
  • Existing tests pass locally (uv run pytest -m "cpu and not slow")

For full setup and testing instructions, see the Contributing Guide.

Checklist

  • My code follows the style guidelines of this project
  • I have performed a self-review of my code, especially for agent-assisted changes
  • I updated the documentation where necessary

Thanks for contributing to Pruna! We're excited to review your work.

New to contributing? Check out our Contributing Guide for everything you need to get started.

Note:

  • Draft PRs or PRs without a clear and detailed overview may be delayed.
  • Please mark your PR as Ready for Review and ensure the sections above are filled out.
  • Contributions that are entirely AI-generated without meaningful human review are discouraged.

@gsprochette gsprochette requested review from begumcig and removed request for begumcig June 20, 2026 11:12
@gsprochette gsprochette changed the title fix: switch torch_dynamic save function from pickle to reapply fix: switch torch_dynamic and quanto saves to save_before_apply Jun 20, 2026
@gsprochette gsprochette requested a review from begumcig June 20, 2026 13:54

@begumcig begumcig left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!!! looks super good to me! :)

@begumcig begumcig merged commit ba157a0 into main Jun 22, 2026
11 checks passed
@begumcig begumcig deleted the fix/switch-torch_dynamic-save-function-from-pickle-to-reapply branch June 22, 2026 08:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants