Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[RLlib] ConnectorV2: Disseminate default connector pieces, fix framestacking for MA, fix bugs in prev-a/prev-r for MA #43491

Merged
merged 40 commits into from
Mar 3, 2024
Merged

[RLlib] ConnectorV2: Disseminate default connector pieces, fix framestacking for MA, fix bugs in prev-a/prev-r for MA #43491

merged 40 commits into from
Mar 3, 2024

Commits on Feb 23, 2024

  1. wip 

    Signed-off-by: sven1977 <svenmika1977@gmail.com>
    @sven1977
    sven1977 committed Feb 23, 2024
    Configuration menu
    Copy the full SHA
    dcf3da9 View commit details
    Browse the repository at this point in the history

Commits on Feb 24, 2024

  1. wip 

    Signed-off-by: sven1977 <svenmika1977@gmail.com>
    @sven1977
    sven1977 committed Feb 24, 2024
    Configuration menu
    Copy the full SHA
    65e6efc View commit details
    Browse the repository at this point in the history

Commits on Feb 25, 2024

  1. wip 

    Signed-off-by: sven1977 <svenmika1977@gmail.com>
    @sven1977
    sven1977 committed Feb 25, 2024
    Configuration menu
    Copy the full SHA
    c0f43fb View commit details
    Browse the repository at this point in the history

Commits on Feb 26, 2024

  1. single-agent framestacking example working 

    Signed-off-by: sven1977 <svenmika1977@gmail.com>
    @sven1977
    sven1977 committed Feb 26, 2024
    Configuration menu
    Copy the full SHA
    2efe039 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'master' of https://github.com/ray-project/ray into conn… 

    …ector_v2_fix_framestacking_for_multi_agent
    @sven1977
    sven1977 committed Feb 26, 2024
    Configuration menu
    Copy the full SHA
    6628cce View commit details
    Browse the repository at this point in the history

  3. multi-agent AND single-agent framestacking example working 

    Signed-off-by: sven1977 <svenmika1977@gmail.com>
    @sven1977
    sven1977 committed Feb 26, 2024
    Configuration menu
    Copy the full SHA
    a1559be View commit details
    Browse the repository at this point in the history

Commits on Feb 27, 2024

  1. multi-agent AND single-agent nested dict example working 

    Signed-off-by: sven1977 <svenmika1977@gmail.com>
    @sven1977
    sven1977 committed Feb 27, 2024
    Configuration menu
    Copy the full SHA
    3343d6c View commit details
    Browse the repository at this point in the history

  2. Merge branch 'master' of https://github.com/ray-project/ray into conn… 

    …ector_v2_fix_framestacking_for_multi_agent
    @sven1977
    sven1977 committed Feb 27, 2024
    Configuration menu
    Copy the full SHA
    4e358c3 View commit details
    Browse the repository at this point in the history

  3. strange state: multi-agent nested obs learning, but single-agent NOT! 

    Signed-off-by: sven1977 <svenmika1977@gmail.com>
    @sven1977
    sven1977 committed Feb 27, 2024
    Configuration menu
    Copy the full SHA
    e96188c View commit details
    Browse the repository at this point in the history

Commits on Feb 28, 2024

  1. all example scripts running in sa and ma mode, but sa not learning(???) 

    Signed-off-by: sven1977 <svenmika1977@gmail.com>
    @sven1977
    sven1977 committed Feb 28, 2024
    Configuration menu
    Copy the full SHA
    2936879 View commit details
    Browse the repository at this point in the history

  2. all example scripts (framestack, nested obs, prev-a/prev-r) running/l… 

    …earning in SA and MA mode.

Signed-off-by: sven1977 <svenmika1977@gmail.com>
    @sven1977
    sven1977 committed Feb 28, 2024
    Configuration menu
    Copy the full SHA
    096e933 View commit details
    Browse the repository at this point in the history

  3. LINT 

    Signed-off-by: sven1977 <svenmika1977@gmail.com>
    @sven1977
    sven1977 committed Feb 28, 2024
    Configuration menu
    Copy the full SHA
    aa2d1cb View commit details
    Browse the repository at this point in the history

  4. Merge branch 'master' of https://github.com/ray-project/ray into conn… 

    …ector_v2_fix_framestacking_for_multi_agent
    @sven1977
    sven1977 committed Feb 28, 2024
    Configuration menu
    Copy the full SHA
    833d4ec View commit details
    Browse the repository at this point in the history

  5. wip 

    Signed-off-by: sven1977 <svenmika1977@gmail.com>
    @sven1977
    sven1977 committed Feb 28, 2024
    Configuration menu
    Copy the full SHA
    5768abe View commit details
    Browse the repository at this point in the history

  6. wip 

    Signed-off-by: sven1977 <svenmika1977@gmail.com>
    @sven1977
    sven1977 committed Feb 28, 2024
    Configuration menu
    Copy the full SHA
    b111365 View commit details
    Browse the repository at this point in the history

Commits on Mar 1, 2024

  1. Merge branch 'master' of https://github.com/ray-project/ray into conn… 

    …ector_v2_fix_framestacking_for_multi_agent
    @sven1977
    sven1977 committed Mar 1, 2024
    Configuration menu
    Copy the full SHA
    9c49e31 View commit details
    Browse the repository at this point in the history

  2. wip 

    Signed-off-by: sven1977 <svenmika1977@gmail.com>
    @sven1977
    sven1977 committed Mar 1, 2024
    Configuration menu
    Copy the full SHA
    a0f6b1e View commit details
    Browse the repository at this point in the history

  3. wip 

    Signed-off-by: sven1977 <svenmika1977@gmail.com>
    @sven1977
    sven1977 committed Mar 1, 2024
    Configuration menu
    Copy the full SHA
    e189dc5 View commit details
    Browse the repository at this point in the history

  4. fixes 

    Signed-off-by: sven1977 <svenmika1977@gmail.com>
    @sven1977
    sven1977 committed Mar 1, 2024
    Configuration menu
    Copy the full SHA
    1dafdcc View commit details
    Browse the repository at this point in the history

  5. fixes 

    Signed-off-by: sven1977 <svenmika1977@gmail.com>
    @sven1977
    sven1977 committed Mar 1, 2024
    Configuration menu
    Copy the full SHA
    668dbde View commit details
    Browse the repository at this point in the history

  6. fixes 

    Signed-off-by: sven1977 <svenmika1977@gmail.com>
    @sven1977
    sven1977 committed Mar 1, 2024
    Configuration menu
    Copy the full SHA
    9785db3 View commit details
    Browse the repository at this point in the history

  7. merge 

    Signed-off-by: sven1977 <svenmika1977@gmail.com>
    @sven1977
    sven1977 committed Mar 1, 2024
    Configuration menu
    Copy the full SHA
    c832bce View commit details
    Browse the repository at this point in the history

  8. LINT 

    Signed-off-by: sven1977 <svenmika1977@gmail.com>
    @sven1977
    sven1977 committed Mar 1, 2024
    Configuration menu
    Copy the full SHA
    a12ad55 View commit details
    Browse the repository at this point in the history

  9. fix 

    Signed-off-by: sven1977 <svenmika1977@gmail.com>
    @sven1977
    sven1977 committed Mar 1, 2024
    Configuration menu
    Copy the full SHA
    0a4ef8d View commit details
    Browse the repository at this point in the history

  10. fix 

    Signed-off-by: sven1977 <svenmika1977@gmail.com>
    @sven1977
    sven1977 committed Mar 1, 2024
    Configuration menu
    Copy the full SHA
    d1b2dd3 View commit details
    Browse the repository at this point in the history

Commits on Mar 2, 2024

  1. Merge branch 'master' of https://github.com/ray-project/ray into conn… 

    …ector_v2_fix_framestacking_for_multi_agent
    @sven1977
    sven1977 committed Mar 2, 2024
    Configuration menu
    Copy the full SHA
    5883e3f View commit details
    Browse the repository at this point in the history

  2. Fixed cont. actions problem with normalized/clipped actions being sto… 

    …red in episode (in episode, we should only store those actions actually coming from the dist sample process!).

BUT nested_actions example (single-agent) seems to be >10% slower now than on master.

Signed-off-by: sven1977 <svenmika1977@gmail.com>
    @sven1977
    sven1977 committed Mar 2, 2024
    Configuration menu
    Copy the full SHA
    bd9d1f0 View commit details
    Browse the repository at this point in the history

  3. LINT 

    Signed-off-by: sven1977 <svenmika1977@gmail.com>
    @sven1977
    sven1977 committed Mar 2, 2024
    Configuration menu
    Copy the full SHA
    9908bc1 View commit details
    Browse the repository at this point in the history

  4. noticed that multi-agent mean std filtering w/ pendulum is actually o… 

    …k, single-agent is not ??

* code simplifications
* better nested action space test via adding lopsided action bounds

Signed-off-by: sven1977 <svenmika1977@gmail.com>
    @sven1977
    sven1977 committed Mar 2, 2024
    Configuration menu
    Copy the full SHA
    ef26e62 View commit details
    Browse the repository at this point in the history

  5. fixed simple agent problem. Still a sorted in listify connector (di… 

    …dn't matter for MA, b/c MA is NOT vectorized, only a sub-env, sorting doesn't effect this).

Signed-off-by: sven1977 <svenmika1977@gmail.com>
    @sven1977
    sven1977 committed Mar 2, 2024
    Configuration menu
    Copy the full SHA
    91c0827 View commit details
    Browse the repository at this point in the history

  6. LINT 

    Signed-off-by: sven1977 <svenmika1977@gmail.com>
    @sven1977
    sven1977 committed Mar 2, 2024
    Configuration menu
    Copy the full SHA
    c012f11 View commit details
    Browse the repository at this point in the history

  7. fixes 

    Signed-off-by: sven1977 <svenmika1977@gmail.com>
    @sven1977
    sven1977 committed Mar 2, 2024
    Configuration menu
    Copy the full SHA
    360dbc2 View commit details
    Browse the repository at this point in the history

  8. docstrings 

    Signed-off-by: sven1977 <svenmika1977@gmail.com>
    @sven1977
    sven1977 committed Mar 2, 2024
    Configuration menu
    Copy the full SHA
    16051d5 View commit details
    Browse the repository at this point in the history

  9. Merge branch 'master' of https://github.com/ray-project/ray into conn… 

    …ector_v2_fix_framestacking_for_multi_agent
    @sven1977
    sven1977 committed Mar 2, 2024
    Configuration menu
    Copy the full SHA
    de74ab4 View commit details
    Browse the repository at this point in the history

Commits on Mar 3, 2024

  1. docstrings 

    Signed-off-by: sven1977 <svenmika1977@gmail.com>
    @sven1977
    sven1977 committed Mar 3, 2024
    Configuration menu
    Copy the full SHA
    2498d1e View commit details
    Browse the repository at this point in the history

  2. docstrings 

    Signed-off-by: sven1977 <svenmika1977@gmail.com>
    @sven1977
    sven1977 committed Mar 3, 2024
    Configuration menu
    Copy the full SHA
    d0040bd View commit details
    Browse the repository at this point in the history

  3. docstrings 

    Signed-off-by: sven1977 <svenmika1977@gmail.com>
    @sven1977
    sven1977 committed Mar 3, 2024
    Configuration menu
    Copy the full SHA
    6a8b117 View commit details
    Browse the repository at this point in the history

  4. LINT 

    Signed-off-by: sven1977 <svenmika1977@gmail.com>
    @sven1977
    sven1977 committed Mar 3, 2024
    Configuration menu
    Copy the full SHA
    f4c29d0 View commit details
    Browse the repository at this point in the history

  5. wip 

    Signed-off-by: sven1977 <svenmika1977@gmail.com>
    @sven1977
    sven1977 committed Mar 3, 2024
    Configuration menu
    Copy the full SHA
    d80a0a5 View commit details
    Browse the repository at this point in the history

  6. wip 

    Signed-off-by: sven1977 <svenmika1977@gmail.com>
    @sven1977
    sven1977 committed Mar 3, 2024
    Configuration menu
    Copy the full SHA
    ca07d17 View commit details
    Browse the repository at this point in the history