[RLlib] For new API stack: Move SampleBatch column names (e.g. SampleBatch.OBS) into new class (Columns).
#43665
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
sven1977
requested review from
avnishn,
ArturNiederfahrenhorst,
maxpumperla,
kouroshHakha and
simonsays1980
as code owners
simonsays1980
approved these changes
Mar 4, 2024
| @@ -37,6 +37,7 @@ | |||
| from ray.rllib.algorithms.algorithm_config import AlgorithmConfig | |||
| from ray.rllib.algorithms.registry import ALGORITHMS_CLASS_TO_NAME as ALL_ALGORITHMS | |||
| from ray.rllib.connectors.agent.obs_preproc import ObsPreprocessorConnector | |||
| from ray.rllib.core.columns import Columns | |||
There was a problem hiding this comment.
Already the PR title made my heart beat faster :D
| @@ -241,7 +241,7 @@ def additional_update_for_module( | |||
| @OverrideToImplementCustomLogic | |||
| def _compute_values( | |||
| self, | |||
| batch_for_vf: Union[MultiAgentBatch, NestedDict], | |||
| batch_for_vf: Dict[str, Any], | |||
There was a problem hiding this comment.
I see we swtich now to the plain dictionary. I remember the ActorCriticEncoder uses indeed a two level NestedDict - is this also planned for these classes?
There was a problem hiding this comment.
Not yet, but yes, I would like to simplify things into using pure dicts.
This will be a separate PR, in which I'll try for the new API stack to not depend on these classes anymore: SampleBatch, MultiAgentBatch, NestedDict.
Signed-off-by: Sven Mika <svenmika1977@gmail.com>
…_sample_batch_fields_to_data Signed-off-by: sven1977 <svenmika1977@gmail.com> # Conflicts: # rllib/env/multi_agent_env_runner.py # rllib/env/single_agent_env_runner.py
…' into move_sample_batch_fields_to_data
simonsays1980
approved these changes
Mar 4, 2024
…_sample_batch_fields_to_data
…_sample_batch_fields_to_data
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
For new API stack: Move SampleBatch column names (e.g. SampleBatch.OBS) into new class (
Columns).Background: We are trying to not rely anymore on SampleBatch or MultiAgentBatch in the new API stack (even though migration away from these has not been completed yet). Instead, we are aiming at simply using python dicts.
Same is true for NestedDicts.
Why are these changes needed?
Related issue number
Checks
git commit -s) in this PR.scripts/format.shto lint the changes in this PR.method in Tune, I've added it in
doc/source/tune/api/under thecorresponding
.rstfile.