Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[RLlib] - Using new SingleAgentEnvRunner with "MeanStdFilter" results in error. #42772

Closed
simonsays1980 opened this issue Jan 27, 2024 · 1 comment
Labels
bug Something that is supposed to be working; but isn't rllib RLlib related issues triage Needs triage (eg: priority, bug/not-bug, and owning component)

Comments

@simonsays1980
Copy link
Collaborator

What happened + What you expected to happen

What happened

Using the new SingleAgentEnvRunner with the MeanStdFilter resulted in an error:

Traceback (most recent call last):
  File "/home/simon/git-projects/experiments/.venv-3-11-0/lib/python3.11/site-packages/ray/air/execution/_internal/event_manager.py", line 110, in resolve_future
    result = ray.get(future)
             ^^^^^^^^^^^^^^^
  File "/home/simon/git-projects/experiments/.venv-3-11-0/lib/python3.11/site-packages/ray/_private/auto_init_hook.py", line 22, in auto_init_wrapper
    return fn(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^
  File "/home/simon/git-projects/experiments/.venv-3-11-0/lib/python3.11/site-packages/ray/_private/client_mode_hook.py", line 103, in wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/simon/git-projects/experiments/.venv-3-11-0/lib/python3.11/site-packages/ray/_private/worker.py", line 2624, in get
    raise value.as_instanceof_cause()
ray.exceptions.RayTaskError(AttributeError): ray::PPO.train() (pid=513252, ip=192.168.1.111, actor_id=09f1ac7874004ae681f1c2a501000000, repr=PPO)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/simon/git-projects/experiments/.venv-3-11-0/lib/python3.11/site-packages/ray/tune/trainable/trainable.py", line 342, in train
    raise skipped from exception_cause(skipped)
  File "/home/simon/git-projects/experiments/.venv-3-11-0/lib/python3.11/site-packages/ray/tune/trainable/trainable.py", line 339, in train
    result = self.step()
             ^^^^^^^^^^^
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/simon/git-projects/experiments/.venv-3-11-0/lib/python3.11/site-packages/ray/rllib/algorithms/algorithm.py", line 868, in step
    self._sync_filters_if_needed(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/simon/git-projects/experiments/.venv-3-11-0/lib/python3.11/site-packages/ray/rllib/algorithms/algorithm.py", line 2579, in _sync_filters_if_needed
    central_worker.filters,
    ^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'SingleAgentEnvRunner' object has no attribute 'filters'

What you expected to happen

Either the new runner uses the corresponding connector for mean-std-filtering or that the MeanStdFilter was used.

Versions / Dependencies

Ray 2.9.1
Python 3.11.0
Fedora 37

Reproduction script

Use any RLlib example with ``SingleAgentEnvRunnerandMeanStdFilter`.

Issue Severity

High: It blocks me from completing my task.

@simonsays1980 simonsays1980 added bug Something that is supposed to be working; but isn't triage Needs triage (eg: priority, bug/not-bug, and owning component) labels Jan 27, 2024
@anyscalesam anyscalesam added the rllib RLlib related issues label Jan 29, 2024
@simonsays1980
Copy link
Collaborator Author

This is solved with the new connector MeanStdFilter.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something that is supposed to be working; but isn't rllib RLlib related issues triage Needs triage (eg: priority, bug/not-bug, and owning component)
Projects
None yet
Development

No branches or pull requests

2 participants