[Bug Fix] Fix crash if spawn is delayed in multi-env

[vincentpierre]

Proposed change(s)

In an environment with 2 behaviors, if one behavior randomly spawns after some time, training with multiple environments can cause a crash.
The reason is that we try to get the behavior specs from the first environment and there is a chance the later behavior is not yet spawned in this environment. I modified the code to take an aggreate of the behavior specs of all of the environments. I tested my fix on a custom environment.

I do not know how to write a test for this bug since it involves multi environment, multi behavior with random spawning of behaviors. I am open to suggestions.

Useful links (Github issues, JIRA tickets, ML-Agents forum threads etc.)

Types of change(s)

• Bug fix
• New feature
• Code refactor
• Breaking change
• Documentation update
• Other (please describe)

Checklist

• Added tests that prove my fix is effective or that my feature works
• Updated the changelog (if applicable)
• Updated the documentation (if applicable)
• Updated the migration guide (if applicable)

Other comments

[ervteng]

I think for a test - you could do a test just for training_behaviors, something very similar to the test_reset_collects_results_from_all_envs in test_subprocess_env_manager.py. It just tests that it calls the EnvironmentCommand for all workers. We have unit tests that test that training_behaviors is called, so we just need to test that calling it does the right thing.

[ervteng]

🚢 with test