Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Refine RL workflow & tune RL models under GYM #577

Merged
merged 49 commits into from
Feb 17, 2023
Merged

Refine RL workflow & tune RL models under GYM #577

merged 49 commits into from
Feb 17, 2023

Commits on Apr 24, 2022

  1. PPO, SAC, DDPG passed

    @lihuoran
    lihuoran committed Apr 24, 2022
    Configuration menu
    Copy the full SHA
    6bc7f0b View commit details
    Browse the repository at this point in the history

  2. Explore in SAC

    @lihuoran
    lihuoran committed Apr 24, 2022
    Configuration menu
    Copy the full SHA
    b3f5aef View commit details
    Browse the repository at this point in the history

  3. Test GYM on server

    @lihuoran
    lihuoran committed Apr 24, 2022
    Configuration menu
    Copy the full SHA
    5dab711 View commit details
    Browse the repository at this point in the history

Commits on Jan 17, 2023

  1. Sync server changes

    @lihuoran
    lihuoran committed Jan 17, 2023
    Configuration menu
    Copy the full SHA
    211c06f View commit details
    Browse the repository at this point in the history

  2. Merge branch 'v0.3' into rl_benchmark_debug

    @lihuoran
    lihuoran committed Jan 17, 2023
    Configuration menu
    Copy the full SHA
    f92f7f1 View commit details
    Browse the repository at this point in the history

  3. pre-commit

    @lihuoran
    lihuoran committed Jan 17, 2023
    Configuration menu
    Copy the full SHA
    514250a View commit details
    Browse the repository at this point in the history

  4. Ready to try on server

    @lihuoran
    lihuoran committed Jan 17, 2023
    Configuration menu
    Copy the full SHA
    fc0c02d View commit details
    Browse the repository at this point in the history

  5. .

    @lihuoran
    lihuoran committed Jan 17, 2023
    Configuration menu
    Copy the full SHA
    9fcdf42 View commit details
    Browse the repository at this point in the history

  6. .

    @lihuoran
    lihuoran committed Jan 17, 2023
    Configuration menu
    Copy the full SHA
    01b5a94 View commit details
    Browse the repository at this point in the history

  7. .

    @lihuoran
    lihuoran committed Jan 17, 2023
    Configuration menu
    Copy the full SHA
    dd27eed View commit details
    Browse the repository at this point in the history

  8. .

    @lihuoran
    lihuoran committed Jan 17, 2023
    Configuration menu
    Copy the full SHA
    1c8f258 View commit details
    Browse the repository at this point in the history

  9. .

    @lihuoran
    lihuoran committed Jan 17, 2023
    Configuration menu
    Copy the full SHA
    1aa1085 View commit details
    Browse the repository at this point in the history

Commits on Jan 18, 2023

  1. Performance OK

    @lihuoran
    lihuoran committed Jan 18, 2023
    Configuration menu
    Copy the full SHA
    148af38 View commit details
    Browse the repository at this point in the history

  2. Move to tests

    @lihuoran
    lihuoran committed Jan 18, 2023
    Configuration menu
    Copy the full SHA
    99ff7b9 View commit details
    Browse the repository at this point in the history

  3. Remove old versions

    @lihuoran
    lihuoran committed Jan 18, 2023
    Configuration menu
    Copy the full SHA
    65ba1a1 View commit details
    Browse the repository at this point in the history

  4. PPO done

    @lihuoran
    lihuoran committed Jan 18, 2023
    Configuration menu
    Copy the full SHA
    f4a85b8 View commit details
    Browse the repository at this point in the history

  5. Start to test AC

    @lihuoran
    lihuoran committed Jan 18, 2023
    Configuration menu
    Copy the full SHA
    2349191 View commit details
    Browse the repository at this point in the history

  6. Start to test SAC

    @lihuoran
    lihuoran committed Jan 18, 2023
    Configuration menu
    Copy the full SHA
    f6f7dae View commit details
    Browse the repository at this point in the history

Commits on Jan 28, 2023

  1. SAC test passed

    @lihuoran
    lihuoran committed Jan 28, 2023
    Configuration menu
    Copy the full SHA
    110fec4 View commit details
    Browse the repository at this point in the history

  2. Multiple round in evaluation

    @lihuoran
    lihuoran committed Jan 28, 2023
    Configuration menu
    Copy the full SHA
    2a1ccd5 View commit details
    Browse the repository at this point in the history

  3. Modify config.yml

    @lihuoran
    lihuoran committed Jan 28, 2023
    Configuration menu
    Copy the full SHA
    c371220 View commit details
    Browse the repository at this point in the history

  4. Add Callbacks

    @lihuoran
    lihuoran committed Jan 28, 2023
    Configuration menu
    Copy the full SHA
    a65d902 View commit details
    Browse the repository at this point in the history

Commits on Jan 30, 2023

  1. [wip] SAC performance not good

    @lihuoran
    lihuoran committed Jan 30, 2023
    Configuration menu
    Copy the full SHA
    aa484f8 View commit details
    Browse the repository at this point in the history

  2. [wip] still not good

    @lihuoran
    lihuoran committed Jan 30, 2023
    Configuration menu
    Copy the full SHA
    84ec6e6 View commit details
    Browse the repository at this point in the history

Commits on Jan 31, 2023

  1. update for some PR comments; Add a MARKDOWN file (#576) 

    Co-authored-by: Jinyu Wang <wang.jinyu@microsoft.com>
    @Jinyu-W
    Jinyu-W and Jinyu Wang committed Jan 31, 2023
    Configuration menu
    Copy the full SHA
    0ceaac4 View commit details
    Browse the repository at this point in the history

  2. Use FullyConnected to replace mlp

    @lihuoran
    lihuoran committed Jan 31, 2023
    Configuration menu
    Copy the full SHA
    aad41d9 View commit details
    Browse the repository at this point in the history

  3. Update action bound

    @lihuoran
    lihuoran committed Jan 31, 2023
    Configuration menu
    Copy the full SHA
    8884231 View commit details
    Browse the repository at this point in the history

  4. Merge branch 'rl_benchmark_debug' into rl_workflow_refine

    @lihuoran
    lihuoran committed Jan 31, 2023
    Configuration menu
    Copy the full SHA
    0a01fb1 View commit details
    Browse the repository at this point in the history

  5. ???

    @lihuoran
    lihuoran committed Jan 31, 2023
    Configuration menu
    Copy the full SHA
    0bd25ca View commit details
    Browse the repository at this point in the history

  6. Change gym env wrapper metrics logci

    @lihuoran
    lihuoran committed Jan 31, 2023
    Configuration menu
    Copy the full SHA
    8781dd6 View commit details
    Browse the repository at this point in the history

  7. Change gym env wrapper metrics logci

    @lihuoran
    lihuoran committed Jan 31, 2023
    Configuration menu
    Copy the full SHA
    7b9b698 View commit details
    Browse the repository at this point in the history

Commits on Feb 1, 2023

  1. refine env_sampler.sample under step mode

    @lihuoran
    lihuoran committed Feb 1, 2023
    Configuration menu
    Copy the full SHA
    52b4d1d View commit details
    Browse the repository at this point in the history

  2. Add DDPG. Performance not good...

    @lihuoran
    lihuoran committed Feb 1, 2023
    Configuration menu
    Copy the full SHA
    a3fea0d View commit details
    Browse the repository at this point in the history

  3. Add DDPG. Performance not good...

    @lihuoran
    lihuoran committed Feb 1, 2023
    Configuration menu
    Copy the full SHA
    23f39d1 View commit details
    Browse the repository at this point in the history

  4. wip

    @lihuoran
    lihuoran committed Feb 1, 2023
    Configuration menu
    Copy the full SHA
    9da8b90 View commit details
    Browse the repository at this point in the history

  5. Sounds like sac works

    @lihuoran
    lihuoran committed Feb 1, 2023
    Configuration menu
    Copy the full SHA
    fb11c31 View commit details
    Browse the repository at this point in the history

  6. Refactor file structure

    @lihuoran
    lihuoran committed Feb 1, 2023
    Configuration menu
    Copy the full SHA
    d7d3282 View commit details
    Browse the repository at this point in the history

  7. Refactor file structure

    @lihuoran
    lihuoran committed Feb 1, 2023
    Configuration menu
    Copy the full SHA
    ea26275 View commit details
    Browse the repository at this point in the history

  8. Refactor file structure

    @lihuoran
    lihuoran committed Feb 1, 2023
    Configuration menu
    Copy the full SHA
    8881a1c View commit details
    Browse the repository at this point in the history

Commits on Feb 6, 2023

  1. Pre-commit

    @lihuoran
    lihuoran committed Feb 6, 2023
    Configuration menu
    Copy the full SHA
    b4db842 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'rl_benchmark_debug' into rl_workflow_refine

    @lihuoran
    lihuoran committed Feb 6, 2023
    Configuration menu
    Copy the full SHA
    8874a65 View commit details
    Browse the repository at this point in the history

  3. Merge branch 'v0.3' into rl_workflow_refine

    @lihuoran
    lihuoran committed Feb 6, 2023
    Configuration menu
    Copy the full SHA
    2a7334b View commit details
    Browse the repository at this point in the history

  4. Pre commit

    @lihuoran
    lihuoran committed Feb 6, 2023
    Configuration menu
    Copy the full SHA
    eb7ae9b View commit details
    Browse the repository at this point in the history

Commits on Feb 8, 2023

  1. Minor refinement of CIM RL

    @lihuoran
    lihuoran committed Feb 8, 2023
    Configuration menu
    Copy the full SHA
    627b7d1 View commit details
    Browse the repository at this point in the history

  2. Jinyu/rl workflow refine (#578) 

    * remove useless files; add device mapping; update pdoc

* add default checkpoint path; fix distributed worker log path issue; update example log path

* update performance doc

* remove tests/rl/algorithms folder
    @Jinyu-W
    Jinyu-W committed Feb 8, 2023
    Configuration menu
    Copy the full SHA
    8386312 View commit details
    Browse the repository at this point in the history

Commits on Feb 9, 2023

  1. Resolve PR comments

    @lihuoran
    lihuoran committed Feb 9, 2023
    Configuration menu
    Copy the full SHA
    b05c849 View commit details
    Browse the repository at this point in the history

  2. Compare PPO with spinning up (#579) 

    * [wip] compare PPO

* PPO matching

* Revert unnecessary changes

* Minor

* Minor
    @lihuoran
    lihuoran committed Feb 9, 2023
    Configuration menu
    Copy the full SHA
    ab5e675 View commit details
    Browse the repository at this point in the history

Commits on Feb 13, 2023

  1. SAC Test parameters update (#580) 

    * fix sac to_device issue; update sac gym test parameters

* add rl test performance plot func

* update sac eval interval config

* update sac checkpoint interval config

* fix callback issue

* update plot func

* update plot func

* update plot func

* update performance doc; upload performance images

* Minor fix in callbacks; refine plot.py format.

* Add n_interactions. Use n_interactions to plot curves.

* pre-commit

---------

Co-authored-by: Huoran Li <huo53926@126.com>
Co-authored-by: Huoran Li <huoranli@microsoft.com>
    @Jinyu-W @lihuoran
    3 people committed Feb 13, 2023
    Configuration menu
    Copy the full SHA
    e180f10 View commit details
    Browse the repository at this point in the history

Commits on Feb 17, 2023

  1. Episode truncation & early stopping (#581) 

    * Add truncated logic

* (To be tested) early stop

* Early stop test passed

* Test passed

* Random action. To be tested.

* Warmup OK

* Pre-commit

* random seed

* Revert pre-commit config
    @lihuoran
    lihuoran committed Feb 17, 2023
    Configuration menu
    Copy the full SHA
    9371949 View commit details
    Browse the repository at this point in the history