10 Pull requests merged by 3 people

• use max of 32 dataset processes if not explicit

  #2403 merged Mar 11, 2025

• pass additional info for fix untrained tokens when using distributed + offloading

  #2388 merged Mar 11, 2025

• fix(modal): add git pull when getting branch files

  #2399 merged Mar 10, 2025

• include iproute2 and nvtop in cloud image

  #2393 merged Mar 10, 2025

• fix: create mount folder on modal if not exist

  #2390 merged Mar 10, 2025

• Use Latest Cut Cross Entropy

  #2392 merged Mar 10, 2025

• chore(doc): add faq when having no default chat_template

  #2398 merged Mar 10, 2025

• feat(doc): add more info on RewardModel datasets

  #2391 merged Mar 10, 2025

• refactor: trl grpo configs to have descriptions

  #2386 merged Mar 7, 2025

• remove lion-pytorch as it's already handled upstream

  #2389 merged Mar 7, 2025

7 Pull requests opened by 3 people

• grab sys prompt too from dataset

  #2397 opened Mar 8, 2025

• Feat: minor docs improvements for RLHF and faq on embeddings

  #2401 opened Mar 11, 2025

• Sequential sample packing

  #2404 opened Mar 11, 2025

• Feat: Add support for gemma3 and add e2e for gemma2

  #2406 opened Mar 12, 2025

• fixes against upstream main branches

  #2407 opened Mar 12, 2025

• only validate hf user token on rank 0

  #2408 opened Mar 12, 2025

• chore(docs): add cookbook/blog link to docs

  #2410 opened Mar 13, 2025

1 Issue closed by 1 person

• Add an example for reward model chat template in docs

  #2240 closed Mar 7, 2025

5 Issues opened by 5 people

• "CUDA error: invalid argument" with FSDP + QLora finetuning

  #2409 opened Mar 12, 2025

• Process hanged when using cpu offloading

  #2405 opened Mar 11, 2025

• FutureWarning: FSDP.state_dict_type() and FSDP.set_state_dict_type() are being deprecated. Please use APIs, get_state_dict() and set_state_dict(), which can support different parallelisms, FSDP1, FSDP2, DDP.

  #2402 opened Mar 11, 2025

• EXTREMELY SLOW (unusable) towards end of tokenization of dataset with long multi turn conversations

  #2396 opened Mar 7, 2025

• LoRA example from quickstart guide not working with Docker container

  #2395 opened Mar 7, 2025

8 Unresolved conversations

Sometimes conversations happen on old items that aren’t yet closed. Here is a list of all the Issues and Pull Requests with unresolved conversations.

• rebased Flex attention support

  #2363 commented on Mar 13, 2025 • 2 new comments

• Add CAME Optimizer

  #2385 commented on Mar 11, 2025 • 1 new comment

• Show sample batch content

  #2145 commented on Mar 7, 2025 • 0 new comments

• ImportError: cannot import name 'shard_checkpoint' from 'transformers.modeling_utils' (transformers 4.49.0)

  #2387 commented on Mar 12, 2025 • 0 new comments

• DPO Prompt Strategies only support single-turn and will fail silently on multi-turn datasets

  #1645 commented on Mar 12, 2025 • 0 new comments

• Update README.md

  #2360 commented on Mar 12, 2025 • 0 new comments

• feat: add eos_tokens and train_on_eot for chat_template EOT parsing

  #2364 commented on Mar 7, 2025 • 0 new comments

• fix(test): replace jackfram llama with smollm

  #2370 commented on Mar 7, 2025 • 0 new comments