Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Feature] Add PPO support #854

Closed
wants to merge 28 commits into from
Closed

[Feature] Add PPO support #854

wants to merge 28 commits into from

Conversation

wheresmyhair
Copy link
Collaborator

Description

Add PPO Support

Pipeline Tests

WIP

wheresmyhair and others added 28 commits June 9, 2024 20:33
@wheresmyhair
[Feature] PPO init
a94cf11
[Feature] PPO pipeline major dev
56dd266
[Typo] Shebang fix
50e14ac
[Feature] PPO shell args update
d18aad4
[Tokenization] RM textonly and text2text data support
58ec809
[Usability] PPO exp wandb naming fix
59873b3
[Usability] HFModelMixin will add pad token if not specified in model…
777ca4d 
… cfg
[Usability] disable use_cache when gradient_checkpointing
bf61d0b
[Usability] Add rollout arg in shell
c1df625
@wheresmyhair
[Bug] Fix padding side
ecdb1e1
@wheresmyhair
[Feature] PPO wrap-up
d12be51
[Typo] Shebang typo fix
82ce25e
[Usability] Add padding side
9ae0327
[Usability] Add text_only and text2text data support
911bbb6
[Usability] Disable use_cache when gradient_checkpointing
15002f6
[Usability] Add padding side
4fb2e0a
@wheresmyhair
[Usability] Add padding_side to dataset fingerprint
f9d99e1
@wheresmyhair
Merge pull request #857 from OptimalScale/yizhenjia-maintenance
d7055e3 
Usability update
@wheresmyhair
[Feature] PPO init
d4afba7
@wheresmyhair
[Feature] PPO pipeline major dev
ab1816c
@wheresmyhair
[Typo] Shebang fix
77ba7cc
@wheresmyhair
[Feature] PPO shell args update
9a12977
@wheresmyhair
[Tokenization] RM textonly and text2text data support
efbb3f1
@wheresmyhair
[Usability] PPO exp wandb naming fix
029fa46
@wheresmyhair
[Usability] HFModelMixin will add pad token if not specified in model…
94e6c99 
… cfg
@wheresmyhair
[Usability] Add rollout arg in shell
0728f07
@wheresmyhair
[Feature] PPO wrap-up
8098d62
@wheresmyhair
Merge branch 'yizhenjia-ppo-v1' of https://github.com/OptimalScale/LM…
5d48e33 
…Flow into yizhenjia-ppo-v1
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
1 participant
@wheresmyhair