Skip to content

add torch-ffn and unsloth-ffn gradient checkpointing modes for speed-up - #2911

Merged
bghira merged 5 commits into
mainfrom
feature/ffn-checkpointing-mode
Jul 27, 2026
Merged

add torch-ffn and unsloth-ffn gradient checkpointing modes for speed-up#2911
bghira merged 5 commits into
mainfrom
feature/ffn-checkpointing-mode

Conversation

@bghira

@bghira bghira commented Jul 27, 2026

Copy link
Copy Markdown
Owner

This pull request expands and clarifies the documentation for the --gradient_checkpointing_backend option across all supported languages, and introduces new experimental guides explaining the tradeoffs and usage of Unsloth-style checkpointing. The main changes include adding new backend options, updating descriptions to reflect these, and providing detailed, language-specific guides for advanced checkpointing strategies.

Documentation improvements:

  • The --gradient_checkpointing_backend option now documents four choices: torch, torch-ffn, unsloth, and unsloth-ffn, with clear explanations of each backend's behavior, requirements, and supported model scopes. [1] [2] [3] [4] [5] [6]

Experimental guides:

  • Added detailed experimental documentation for Unsloth-style checkpointing in Spanish, Hindi, and Japanese, covering practical usage advice, memory and speed tradeoffs, and best practices for choosing checkpointing strategies. [1] [2] [3]

Internationalization:

  • All changes to the --gradient_checkpointing_backend option and the new guides are provided in English, Spanish, Hindi, Japanese, Portuguese, and Chinese, ensuring consistent and clear guidance for users in all supported languages. [1] [2] [3] [4] [5] [6] [7] [8] [9]

These updates ensure that users understand the available checkpointing backends, their tradeoffs, and how to apply them effectively in different training scenarios.

@bghira
bghira merged commit 9ee1360 into main Jul 27, 2026
4 checks passed
@bghira
bghira deleted the feature/ffn-checkpointing-mode branch July 27, 2026 03:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant