How to set black_death to true

[Axelskool]

Hi, I have a custom pettingzoo Parallel environment and want to train a shared policy on multiple agents. I want to be able to have a few of my agents inactive during a training-episode under certain conditions. I also want to bring back agents during an episode. I call the petting_zoo_env_to_vec_env_v1 and concat_vec_envs_v1 on my environment to allow for shared policy and training on copies of the environment.

I also use the wrappers black_death_v3, pad_observations_v0 and pad_action_space_v0 to allow for agent death but I still get an error message "environment has agent death. Not allowed for pettingzoo_env_to_vec_env_v1 unless black_death is true". How do I set this? So in principle is it possible to have self.possible_agents as a sort of agent pool to take agents from and then have self.agents as the active agents during an episode?

best regards

Axel

[Axelskool]

Alternatively can we somehow set the reward/observation to some value for the agents that have some inactive attribute set, in order to avoid using these agents for the learning step while they are inactive?

[jjshoots]

environment has agent death. Not allowed for pettingzoo_env_to_vec_env_v1 unless black_death is true

I believe the order of wrappers matters in this case. Are you wrappingn it in black death before doing pettingzoo_env_to_vec_env, and then only doing concat_vec_envs_v1?

[jjshoots]

Alternatively can we somehow set the reward/observation to some value for the agents that have some inactive attribute set, in order to avoid using these agents for the learning step while they are inactive?

In general no, at least not without modifying the environment some manner or form I believe. Since you're the one writing the environment, that should be doable.

In general though, this sort of behaviour is the whole idea of the AEC environment.

[Axelskool]

I am first wrapping the parallel petting zoo environment with black death, then petting_zoo_env_to_vec_env then concat_vec_envs_v1. The problem for our environment is that the problem is most easily formulated as a parallel enviroment and the rewards are difficult to calculate without knowing all the other agents actions at once. We are applying marl to manage creation of tracks in a multi-object tracking system.

[Axelskool]

And if I convert this environment to an AEC, does vectorize_aec_env_v0(aec_env, num_envs, num_cpus=x) allow me to have parameter sharing if I use stable-baseline3 and ppo to train the agent.

Thank you again for your time. It's very appreciated!

[jjshoots]

I think @RedTachyon is a better person to answer this question.

[benblack769]

This black death error is often a symtom hiding the underlying problem. Some ideas:

1. Did you check that your paralell env passes the parallel_api_test? https://www.pettingzoo.ml/environment_creation#parallel-api-test
2. Does your environment generate new agents after reset is called? (the environment vectorization does not support this)
3. Could you share your code snippet of how your environment is wrapped? We might be able to help debug this.

Unfortunately, vectorize_aec_env_v0 is not compatable with SB3.

[Axelskool]

We generate new agents after reset is called in the sense that we reset the self.agents list and populate it once more. However, we do not change the the possible agents, list. We will try your suggestions! Thanks again!

[benblack769]

Do you repopulate the agents outside of reset (i.e. in step?). This is what is not supported.

[Axelskool]

Yes this is what we did. Thanks for the reply.

[jjshoots]

Hi @Axelskool, has this issue been solved? If so would it be possible to close it?

[Axelskool]

Yes close it. Thank you for your time! Den ons 11 maj 2022 12:42Jet ***@***.***> skrev:

…

Hi @Axelskool <https://github.com/Axelskool>, has this issue been solved? If so would it be possible to close it? — Reply to this email directly, view it on GitHub <#661 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AN3BHZYHXBIQJWJQ5MI3DDTVJOFJXANCNFSM5T6O2RHA> . You are receiving this because you were mentioned.Message ID: ***@***.***>