Signed-off-by: Simon Zehnder <simon.zehnder@gmail.com>

Signed-off-by: Simon Zehnder <simon.zehnder@gmail.com>

…ization, timestep mapping and class data.

Signed-off-by: Simon Zehnder <simon.zehnder@gmail.com>

…ule states will only be stored in the 'SingleAgentEpisode's.

Signed-off-by: Simon Zehnder <simon.zehnder@gmail.com>

…et_return' and '__len__'. Moved episode files into 'rllib/env'.

Signed-off-by: Simon Zehnder <simon.zehnder@gmail.com>

…one. Furthermore moved 'SingleAgentEpisode' and 'MultiAgentEpisode' towards 'rllib/env'. I also added unit testing for 'SingleAgentEpisode'.

Signed-off-by: Simon Zehnder <simon.zehnder@gmail.com>

Signed-off-by: Simon Zehnder <simon.zehnder@gmail.com>

Signed-off-by: Simon Zehnder <simon.zehnder@gmail.com>

Signed-off-by: Simon Zehnder <simon.zehnder@gmail.com>

Signed-off-by: Simon Zehnder <simon.zehnder@gmail.com>

Signed-off-by: Simon Zehnder <simon.zehnder@gmail.com>

…de into branch episode.

Signed-off-by: Simon Zehnder <simon.zehnder@gmail.com>

…AgentEpisode'.

Signed-off-by: Simon Zehnder <simon.zehnder@gmail.com>

Signed-off-by: Simon Zehnder <simon.zehnder@gmail.com>

Signed-off-by: Simon Zehnder <simon.zehnder@gmail.com>

Signed-off-by: Simon Zehnder <simon.zehnder@gmail.com>

Signed-off-by: Simon Zehnder <simon.zehnder@gmail.com>

Signed-off-by: Simon Zehnder <simon.zehnder@gmail.com>

Signed-off-by: Simon Zehnder <simon.zehnder@gmail.com>

Signed-off-by: Simon Zehnder <simon.zehnder@gmail.com>

…r-mae

Signed-off-by: Simon Zehnder <simon.zehnder@gmail.com>

Signed-off-by: Simon Zehnder <simon.zehnder@gmail.com>

Signed-off-by: Simon Zehnder <simon.zehnder@gmail.com>

Signed-off-by: Simon Zehnder <simon.zehnder@gmail.com>

Signed-off-by: Simon Zehnder <simon.zehnder@gmail.com>

Signed-off-by: Simon Zehnder <simon.zehnder@gmail.com>

… 'add_timestep()'.

Signed-off-by: Simon Zehnder <simon.zehnder@gmail.com>

… and agents that terminate before stepping first time.

Signed-off-by: Simon Zehnder <simon.zehnder@gmail.com>

…le of functionalitites in 'MultiAgentEPisode'.

Signed-off-by: Simon Zehnder <simon.zehnder@gmail.com>

Signed-off-by: Simon Zehnder <simon.zehnder@gmail.com>

…n the generation of the global timestep mapping.

Signed-off-by: Simon Zehnder <simon.zehnder@gmail.com>

…timestep' to deal directly with 'done' agents and also to skip partial testing for is_terminated/is_truncated. Furthermore, adjusted all functions to deal with 'terminated' and Ätruncated' dictionaries that contain only ready agents.

Signed-off-by: Simon Zehnder <simon.zehnder@gmail.com>

…bugs in 'add_timestep()' and '__init__()' methods in 'MultiAgentEpisode' to handle agents that never stepped and to handle the '__all__' in 'terminated' and 'truncated'. Changed '_agent_ids' to 'set' instead of 'list'.

Signed-off-by: Simon Zehnder <simon.zehnder@gmail.com>

…rack of 'orphane' rewards in multi-agent environemnts that occur in between an agent's action and the nex tobservation or before the initial observation. Furthermore, changed data extraction for single agents to take care of siutations in which collected data contains 'orphane' actions, i.e. actions that miss their subsequent observation. Orphane actions, states, and extra model outputs are put into the buffer. Ensured that tests are making use of these functionalities.

Signed-off-by: Simon Zehnder <simon.zehnder@gmail.com>

…ntense testing led to further changes in the 'MultiAgentEpisode', specifically as we do need for the successor results from 'get_observations()', 'get_infos', etc. in type 'List[MultiAgentDict]' and not 'MultiAgentDict'. Furthermore, 'terminateds' and 'truncateds' hat to be provided with a getter. Needs more testing.

Signed-off-by: Simon Zehnder <simon.zehnder@gmail.com>

…tEpisode's. Also added corresponding tests. Rnamed 'global_rewards' to 'partial_rewards'. This is an intermediate commit to have a safe state to return to.

Signed-off-by: Simon Zehnder <simon.zehnder@gmail.com>

…encies in the use of the ' partial_rewards'. Has to be fixed before stepping forward.

Signed-off-by: Simon Zehnder <simon.zehnder@gmail.com>

…ere removed from 'SingleAgentEpisode by @sven1977. Added test for getters. Needed to change '_getattr_by_index' as using buffered actions is non-trivial. Had to add a 'global_actions_t' timestep_mapping for the actions as they could be buffered and the original timestep would get lost. Testing is not finished.

Signed-off-by: Simon Zehnder <simon.zehnder@gmail.com>

… of using buffered actions. Thereby refactored '_getattr_by_index()' extensively to be used more generically. Furthermore, wrote corresponding tests. Fixed a bug in the 'create_successor()' method. All tests run.

Signed-off-by: Simon Zehnder <simon.zehnder@gmail.com>

…ards, but with receiving either 'MultiAgentDict' or 'List[MutliAgentDict]'. Made minor changes to '_IndexMapping' and '_getattr_by_index()'.

Signed-off-by: Simon Zehnder <simon.zehnder@gmail.com>

…ded buffered rewards to the 'get_rewards' method.

Signed-off-by: Simon Zehnder <simon.zehnder@gmail.com>

…wards' is complete now and needs to be tested more.

Signed-off-by: Simon Zehnder <simon.zehnder@gmail.com>

…e for some days now.

Signed-off-by: Simon Zehnder <simon.zehnder@gmail.com>

…le testing.

Signed-off-by: Simon Zehnder <simon.zehnder@gmail.com>

…t_episode()' to use the corresponding 'SingleAgentEpisode''s methods, to contain as initial observation in the successor always the last observation of an agent. Adjusted tests accordingly and added multiple new ones.

Signed-off-by: Simon Zehnder <simon.zehnder@gmail.com>

…method to 'MultiAgentEpisode' for immutable copying of buffers between episodes. Refined the test file.

Signed-off-by: Simon Zehnder <simon.zehnder@gmail.com>

…fied these functions to account for empty episodes and agents that are done.'

Signed-off-by: Simon Zehnder <simon.zehnder@gmail.com>

Signed-off-by: sven1977 <svenmika1977@gmail.com>

Signed-off-by: sven1977 <svenmika1977@gmail.com>

Signed-off-by: sven1977 <svenmika1977@gmail.com>

…ing-multi-agent-episode

Signed-off-by: sven1977 <svenmika1977@gmail.com>