feat(saute, simmer): support saute rl and clean the code #209

zmsn-2077 · 2023-04-08T11:00:18Z

Description

feat(saute, simmer): support saute rl and clean the code

Types of changes

What types of changes does your code introduce? Put an x in all the boxes that apply:

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds core functionality)
Breaking change (fix or feature that would cause existing functionality to change)
Documentation (update in the documentation)

Checklist

Go over all the following points, and put an x in all the boxes that apply.
If you are unsure about any of these, don't hesitate to ask. We are here to help!

I have read the CONTRIBUTION guide. (required)
My change requires a change to the documentation.
I have updated the tests accordingly. (required for a bug fix or a new feature)
I have updated the documentation accordingly.
I have reformatted the code using make format. (required)
I have checked the code using make lint. (required)
I have ensured make test pass. (required)

…ent#131)

Co-authored-by: borong <borongzh@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

Co-authored-by: ruiyang sun <rockmagma02@gmail.com>

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: ruiyang sun <rockmagma02@gmail.com>

Co-authored-by: zmsn-2077 <73586554+zmsn-2077@users.noreply.github.com> Co-authored-by: borong <borongzh@gmail.com> Co-authored-by: friedmainfunction <73703265+friedmainfunction@users.noreply.github.com>

…nment#147) Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

…thms (PKU-Alignment#148) Co-authored-by: borong <borongzh@gmail.com> Co-authored-by: Gaiejj <524339208@qq.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: zmsn-2077 <jiamg.ji@gmail.com>

Co-authored-by: borong <borongzh@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: ruiyang sun <rockmagma02@gmail.com> Co-authored-by: zmsn-2077 <73586554+zmsn-2077@users.noreply.github.com> Co-authored-by: friedmainfunction <73703265+friedmainfunction@users.noreply.github.com> Co-authored-by: Gaiejj <524339208@qq.com> Co-authored-by: zmsn-2077 <jiamg.ji@gmail.com> Co-authored-by: Ruiyang Sun <rockmagma02@gmail.com> Co-authored-by: Jiayi Zhou <108712610+Gaiejj@users.noreply.github.com> Co-authored-by: 1Asan <99461435+1Asan@users.noreply.github.com> fix(algo): fix no return in algo_wrapper::learn (PKU-Alignment#122) fix(logger, wrapper): support csv file and velocity tasks (PKU-Alignment#131) fix typo. (PKU-Alignment#134) fix(ppo): fix entropy loss (PKU-Alignment#135) fix bugs (PKU-Alignment#136) fix: support new config for exp_grid (PKU-Alignment#142) fix(rollout, exp_grid): fix logdir path conflict (PKU-Alignment#145) fix(on-policy): fix the second order algorithms performance (PKU-Alignment#147)

…lignment#162) Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: borong <borongzh@gmail.com>

Co-authored-by: borong <borongzh@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: ruiyang sun <rockmagma02@gmail.com> Co-authored-by: zmsn-2077 <73586554+zmsn-2077@users.noreply.github.com> Co-authored-by: friedmainfunction <73703265+friedmainfunction@users.noreply.github.com> Co-authored-by: Gaiejj <524339208@qq.com> Co-authored-by: zmsn-2077 <jiamg.ji@gmail.com> Co-authored-by: Ruiyang Sun <rockmagma02@gmail.com> Co-authored-by: Jiayi Zhou <108712610+Gaiejj@users.noreply.github.com> Co-authored-by: 1Asan <99461435+1Asan@users.noreply.github.com> fix(algo): fix no return in algo_wrapper::learn (PKU-Alignment#122) fix(logger, wrapper): support csv file and velocity tasks (PKU-Alignment#131) fix typo. (PKU-Alignment#134) fix(ppo): fix entropy loss (PKU-Alignment#135) fix bugs (PKU-Alignment#136) fix: support new config for exp_grid (PKU-Alignment#142) fix(rollout, exp_grid): fix logdir path conflict (PKU-Alignment#145) fix(on-policy): fix the second order algorithms performance (PKU-Alignment#147)

muchvo

LGTM.

Gaiejj

LGTM.

rockmagma02 and others added 30 commits March 15, 2023 02:53

refactor: change architecture of omnisafe (PKU-Alignment#121)

4987e80

fix(algo): fix no return in algo_wrapper::learn (PKU-Alignment#122)

90f7549

chore: support num_thread setting (PKU-Alignment#124)

16be6a8

feat: update architecture of config.yaml (PKU-Alignment#126)

51e7f96

fix(logger, wrapper): support csv file and velocity tasks (PKU-Alignm…

2aa1417

…ent#131)

chore: fix typo. (PKU-Alignment#134)

04477eb

fix(ppo): fix entropy loss (PKU-Alignment#135)

fe686bb

test: add more test case, and fix bugs (PKU-Alignment#136)

63dc035

fix: support new config for exp_grid (PKU-Alignment#142)

8b32d73

Co-authored-by: borong <borongzh@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

feat: support policy evaluation (PKU-Alignment#137)

06dcbdb

Co-authored-by: ruiyang sun <rockmagma02@gmail.com>

feat(off-policy): add DDPG, TD3 SAC (PKU-Alignment#128)

c232fcf

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: ruiyang sun <rockmagma02@gmail.com>

fix(rollout, exp_grid): fix logdir path conflict (PKU-Alignment#145)

78ec6bd

Co-authored-by: zmsn-2077 <73586554+zmsn-2077@users.noreply.github.com> Co-authored-by: borong <borongzh@gmail.com> Co-authored-by: friedmainfunction <73703265+friedmainfunction@users.noreply.github.com>

fix(on-policy): fix the second order algorithms performance (PKU-Alig…

817edf9

…nment#147) Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

docs: update README.md (PKU-Alignment#149)

cb633d6

docs: update changelog and release v0.1.0

ba48512

feat: support off-policy lagrangian algorithms

3e8de48

chore: update yaml (PKU-Alignment#155)

8463ca5

chore: update github workflow (PKU-Alignment#156)

5068262

feat: support cuda

6c1ed1d

feat: support cuda exp grid

3380d75

chore: merge dev branch

f43f32c

fix(onpolicy_adapter): fix the calculation of last state value (PKU-A…

973b363

…lignment#162) Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: borong <borongzh@gmail.com>

Merge branch 'dev' into dev-action

cb1544c

fix: [pre-commit.ci] auto fixes [...]

2a294a4

fix: disable test cuda

065e22d

chore: update yaml (PKU-Alignment#155)

60c901e

chore: update github workflow (PKU-Alignment#156)

d955dd8

Gaiejj and others added 22 commits April 1, 2023 13:14

fix: fix cvpo

03103be

chore: merge dev

0815991

feat: support saute

0c036e5

feat: support simmer

7362b3e

refactor: refactor off policy lag

c6b9e2e

chore: merge dev

3d6e7c5

chore: update configs

2b34c94

chore: rebase dev

26fa1c7

chore: clean the code

5bd5e21

chore: clean the code

d05f0ce

chore: update yaml

2a7d7a8

chore: clean the code

b800909

chore: update env version

8c5a8f2

chore: clean the code

c60aedf

feat: support simmer controller

633acb5

chore: merge dev

f31a3e1

chore: merge dev

31adb38

feat: support simmer

1fddc2a

refactor: clean the code

0e8e698

chore: merge dev

06b0132

refactor: clean the code

96356d4

style: clean the code

ac5c4ea

zmsn-2077 marked this pull request as draft April 8, 2023 11:01

style: clean the code

5745b9d

muchvo approved these changes Apr 8, 2023

View reviewed changes

fix: fix plotter

864785e

Gaiejj marked this pull request as ready for review April 8, 2023 19:23

Gaiejj approved these changes Apr 8, 2023

View reviewed changes

Gaiejj merged commit b0a0699 into PKU-Alignment:dev Apr 8, 2023
4 checks passed

Gaiejj deleted the dev-saute branch April 8, 2023 19:29

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(saute, simmer): support saute rl and clean the code #209

feat(saute, simmer): support saute rl and clean the code #209

zmsn-2077 commented Apr 8, 2023

muchvo left a comment

Gaiejj left a comment

feat(saute, simmer): support saute rl and clean the code #209

feat(saute, simmer): support saute rl and clean the code #209

Conversation

zmsn-2077 commented Apr 8, 2023

Description

Types of changes

Checklist

muchvo left a comment

Choose a reason for hiding this comment

Gaiejj left a comment

Choose a reason for hiding this comment