Release v0.2.7

Changes

• Added support for vLLM-v0.4.2 @hijkzzz
• Added support for Jamba-v0.1 (Incompatible with vLLM-v0.4.2 now) @hijkzzz
• Added LoRA configs (--lora_dropout, --target_modules) @hijkzzz

Release v0.2.6

Changes

• Upgraded vLLM to v0.4.1 @mgerstgrasser @wuxibin89 @hijkzzz
• Upgraded Transformers to v4.40.1 and DeepSpeed to v0.14.0 @hijkzzz
• Fixed typo in train_ppo_ray.py @mickelliu
• Fixed mismatch size output_state_dict(148) and state_dict(149) in model saving @hijkzzz
• Added support for --colocate_actor_ref and --colocate_critic_reward in train_ppo_ray.py @hijkzzz
• Added support for Ray PPO reward ref models offloading @hijkzzz

Release v0.2.5

Changes

• Added Chinese README.md @khazic
• Added KD Trainer and Loss @ifromeast
• Fixed num_training_steps @wuxibin89
• Updated requirements.txt @kfertakis
• Fixed error due to 'margin' variable type being list in rm_trainer.py @StwayneXG

Release v0.2.4

Changes

• Fixed DPO masked loss function @hijkzzz
• Fixed Yi-34B tokenizer (--disable_fast_tokenizer) #240 @hijkzzz
• Supported wandb.login() (--wandb True) #231 @mgerstgrasser

Release v0.2.3

Changes

• Fixed #191 "deepspeed.zero.Init causes very strange spikes in PPO policy_loss" @hijkzzz
• Added dockerfile for vLLM @hijkzzz

Release v0.2.2

Changes

• Fixed LlamaRotaryEmbedding for Transformers v4.38.1 @hijkzzz
• Use lazy vLLM engine @wuxibin89
• Added Chinese PR docs @catqaq
• Fixed tensor shape docs @Thecats-Jfm

Release v0.2.1

Changes

• Fixed position_ids for left padding #217 @hijkzzz
• Supported input_key for custom dataset @hijkzzz

Release v0.2.0

Changes

• Supported vLLM 0.3.1 @wuxibin89

Release v0.1.10

Changes

• Fixed save_models for named_buffer @wuxibin89
• Fixed vLLM generation hang bug (requires vLLM<0.2.7) @hijkzzz

Release v0.1.9

Changes

• Supported input_template #203 @rbao2018
• Supported KTO #201 @Dylancer1998
• Upgrade HuggingFace Transformers to 4.37.1