Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix(off-policy): fix log when not update #206

Merged
merged 77 commits into from
Apr 6, 2023
Merged

fix(off-policy): fix log when not update #206

merged 77 commits into from
Apr 6, 2023

Conversation

Gaiejj
Copy link
Member

@Gaiejj Gaiejj commented Apr 6, 2023

Description

fix(off-policy): fix log when not update

Types of changes

What types of changes does your code introduce? Put an x in all the boxes that apply:

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds core functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Documentation (update in the documentation)

Checklist

Go over all the following points, and put an x in all the boxes that apply.
If you are unsure about any of these, don't hesitate to ask. We are here to help!

  • I have read the CONTRIBUTION guide. (required)
  • My change requires a change to the documentation.
  • I have updated the tests accordingly. (required for a bug fix or a new feature)
  • I have updated the documentation accordingly.
  • I have reformatted the code using make format. (required)
  • I have checked the code using make lint. (required)
  • I have ensured make test pass. (required)

rockmagma02 and others added 30 commits March 15, 2023 02:53
@rockmagma02 @zmsn-2077
refactor: change architecture of omnisafe (PKU-Alignment#121)
4987e80
@rockmagma02 @zmsn-2077
fix(algo): fix no return in algo_wrapper::learn (PKU-Alignment#122)
90f7549
@Gaiejj @zmsn-2077
chore: support num_thread setting (PKU-Alignment#124)
16be6a8
@zmsn-2077
feat: update architecture of config.yaml (PKU-Alignment#126)
51e7f96
@Gaiejj @zmsn-2077
fix(logger, wrapper): support csv file and velocity tasks (PKU-Alignm…
2aa1417 
…ent#131)
@1Asan @zmsn-2077
chore: fix typo. (PKU-Alignment#134)
04477eb
@Gaiejj @zmsn-2077
fix(ppo): fix entropy loss (PKU-Alignment#135)
fe686bb
@rockmagma02 @zmsn-2077
test: add more test case, and fix bugs (PKU-Alignment#136)
63dc035
@muchvo @pre-commit-ci @zmsn-2077
fix: support new config for exp_grid (PKU-Alignment#142)
8b32d73 
Co-authored-by: borong <borongzh@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
@Gaiejj @rockmagma02 @zmsn-2077
feat: support policy evaluation (PKU-Alignment#137)
06dcbdb 
Co-authored-by: ruiyang sun <rockmagma02@gmail.com>
@Gaiejj @pre-commit-ci
@rockmagma02 @zmsn-2077
feat(off-policy): add DDPG, TD3 SAC (PKU-Alignment#128)
c232fcf 
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: ruiyang sun <rockmagma02@gmail.com>
@muchvo @zmsn-2077 @friedmainfunction
fix(rollout, exp_grid): fix logdir path conflict (PKU-Alignment#145)
78ec6bd 
Co-authored-by: zmsn-2077 <73586554+zmsn-2077@users.noreply.github.com>
Co-authored-by: borong <borongzh@gmail.com>
Co-authored-by: friedmainfunction <73703265+friedmainfunction@users.noreply.github.com>
@Gaiejj @pre-commit-ci @zmsn-2077
fix(on-policy): fix the second order algorithms performance (PKU-Alig…
817edf9 
…nment#147)

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
@muchvo @Gaiejj
@pre-commit-ci @zmsn-2077
chore(on-policy): update benchmark performance for first-order algori…
d7233ce 
…thms (PKU-Alignment#148)

Co-authored-by: borong <borongzh@gmail.com>
Co-authored-by: Gaiejj <524339208@qq.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: zmsn-2077 <jiamg.ji@gmail.com>
@Gaiejj @zmsn-2077
docs: update README.md (PKU-Alignment#149)
cb633d6
@zmsn-2077
docs: update changelog and release v0.1.0
ba48512
@Gaiejj
feat: support off-policy lagrangian algorithms
3e8de48
@Gaiejj
chore: update yaml (PKU-Alignment#155)
8463ca5
@Gaiejj
chore: update github workflow (PKU-Alignment#156)
5068262
@Gaiejj
feat: support cuda
6c1ed1d
@muchvo @pre-commit-ci
@rockmagma02 @zmsn-2077 @friedmainfunction @Gaiejj @1Asan
feat: support command line interfaces for omnisafe (PKU-Alignment#144)
3c2daf0 
Co-authored-by: borong <borongzh@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: ruiyang sun <rockmagma02@gmail.com>
Co-authored-by: zmsn-2077 <73586554+zmsn-2077@users.noreply.github.com>
Co-authored-by: friedmainfunction <73703265+friedmainfunction@users.noreply.github.com>
Co-authored-by: Gaiejj <524339208@qq.com>
Co-authored-by: zmsn-2077 <jiamg.ji@gmail.com>
Co-authored-by: Ruiyang Sun <rockmagma02@gmail.com>
Co-authored-by: Jiayi Zhou <108712610+Gaiejj@users.noreply.github.com>
Co-authored-by: 1Asan <99461435+1Asan@users.noreply.github.com>
fix(algo): fix no return in algo_wrapper::learn (PKU-Alignment#122)
fix(logger, wrapper): support csv file and velocity tasks (PKU-Alignment#131)
fix typo. (PKU-Alignment#134)
fix(ppo): fix entropy loss (PKU-Alignment#135)
fix bugs (PKU-Alignment#136)
fix: support new config for exp_grid (PKU-Alignment#142)
fix(rollout, exp_grid): fix logdir path conflict (PKU-Alignment#145)
fix(on-policy): fix the second order algorithms performance (PKU-Alignment#147)
@Gaiejj
feat: support cuda exp grid
3380d75
@Gaiejj
chore: merge dev branch
f43f32c
@Dtrc2207 @pre-commit-ci @muchvo
fix(onpolicy_adapter): fix the calculation of last state value (PKU-A…
973b363 
…lignment#162)

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: borong <borongzh@gmail.com>
@Gaiejj
Merge branch 'dev' into dev-action
cb1544c
@pre-commit-ci
fix: [pre-commit.ci] auto fixes [...]
2a294a4
@Gaiejj
fix: disable test cuda
065e22d
@Gaiejj @zmsn-2077
chore: update yaml (PKU-Alignment#155)
60c901e
@Gaiejj @zmsn-2077
chore: update github workflow (PKU-Alignment#156)
d955dd8
@muchvo @pre-commit-ci
@rockmagma02 @zmsn-2077 @friedmainfunction @Gaiejj @1Asan
feat: support command line interfaces for omnisafe (PKU-Alignment#144)
12dbced 
Co-authored-by: borong <borongzh@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: ruiyang sun <rockmagma02@gmail.com>
Co-authored-by: zmsn-2077 <73586554+zmsn-2077@users.noreply.github.com>
Co-authored-by: friedmainfunction <73703265+friedmainfunction@users.noreply.github.com>
Co-authored-by: Gaiejj <524339208@qq.com>
Co-authored-by: zmsn-2077 <jiamg.ji@gmail.com>
Co-authored-by: Ruiyang Sun <rockmagma02@gmail.com>
Co-authored-by: Jiayi Zhou <108712610+Gaiejj@users.noreply.github.com>
Co-authored-by: 1Asan <99461435+1Asan@users.noreply.github.com>
fix(algo): fix no return in algo_wrapper::learn (PKU-Alignment#122)
fix(logger, wrapper): support csv file and velocity tasks (PKU-Alignment#131)
fix typo. (PKU-Alignment#134)
fix(ppo): fix entropy loss (PKU-Alignment#135)
fix bugs (PKU-Alignment#136)
fix: support new config for exp_grid (PKU-Alignment#142)
fix(rollout, exp_grid): fix logdir path conflict (PKU-Alignment#145)
fix(on-policy): fix the second order algorithms performance (PKU-Alignment#147)
XuehaiPan and others added 23 commits March 28, 2023 23:45
@XuehaiPan
feat: add ruff and codespell integration (PKU-Alignment#186)
8cf2687
@zmsn-2077
docs: update changelog.md (PKU-Alignment#187)
989e11e
@muchvo
deps(torch): pin the version of pytorch (PKU-Alignment#184)
1574db6
@Gaiejj
chore: merge dev
a420d39
@Gaiejj
fix: fix off-policy lag
07b4aac
@muchvo
feat: perfecting training workflow (PKU-Alignment#185)
14f5493
@Gaiejj
fix: fix off_policy algorithm
e121a8b
@Gaiejj
fix: fix cvpo
03103be
@Gaiejj
refactor: refactor off policy lag
c6b9e2e
@Gaiejj
chore: merge dev
3d6e7c5
@Gaiejj
chore: update configs
2b34c94
@Gaiejj
chore: rebase dev
26fa1c7
@Gaiejj
chore: clean the code
5bd5e21
@Gaiejj
chore: clean the code
d05f0ce
@Gaiejj
chore: update yaml
2a7d7a8
@Gaiejj
chore: clean the code
b800909
@Gaiejj
chore: update env version
8c5a8f2
@Gaiejj
chore: clean the code
c60aedf
@Gaiejj
chore: merge dev
f31a3e1
@Gaiejj
fix: fix device error
6e71cee
@Gaiejj
docs(adapters): update comment
746371f
@Gaiejj
fix(off_policy): fix log when not update
e4a9d0a
@Gaiejj
chroe: merge dev
bc10dc5
@Gaiejj Gaiejj added bug Something isn't working algorithm Some issues about algorithm labels Apr 6, 2023
@Gaiejj Gaiejj marked this pull request as ready for review April 6, 2023 14:57
muchvo
muchvo approved these changes Apr 6, 2023
View reviewed changes
Copy link
Collaborator

@muchvo muchvo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

@Gaiejj Gaiejj merged commit 55b9db1 into PKU-Alignment:dev Apr 6, 2023
4 checks passed
@Gaiejj Gaiejj deleted the dev-off branch April 13, 2023 10:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@zmsn-2077 zmsn-2077 zmsn-2077 approved these changes

@muchvo muchvo muchvo approved these changes

Assignees
No one assigned
Labels
algorithm Some issues about algorithm bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
7 participants
@Gaiejj @zmsn-2077 @muchvo @rockmagma02 @1Asan @Dtrc2207 @XuehaiPan