Does it also support Vector Observation and Visual Observations?

[Dhaizei]

Does this mlagents support both vector observations and visual observations simultaneously? After checking, it seems that the available convolutional models are only Match3, CNN, and ResNet. If both vector and visual observations are involved, can these models still be used?In the demos, I also only saw support for vector or visual observations, but none that support both simultaneously.

[github-actions]

This issue is stale because it has been open for 30 days with no activity.

[github-actions]

This issue was closed because it has been inactive for 14 days since being marked as stale. Please open a new issue for related bugs.

[IliaTrofimov]

Yes. Just try train agent with vector and vis observations, then view resulted onnx file (no matter if it has trained well or not). Visual observations are being processed with conv layers (depends on your setting), than they being concatenated with regular vector observations. After that last dense layer (you configure it with num_layers and hidden_units) is used to create actions.

See https://github.com/mbaske/grid-sensor/blob/master/Assets/Examples/Driver/Agent/Policy/Driver.onnx as example. This agent contains 2 visual observations and one vector.