Rendering control environments

[neale]

In dm_control there was the ability to render control tasks like cartpole swingup, even though the environment only had a dynamics-based observation space. It would be nice to have that ability here, especially since the environment is from dm_control.
It looks like the rendering function in the Gym wrapper just returns the last observation (in both human and rgb_array mode), which doesn't really work for a lot of tasks in bsuite when the observation is not an rgb_array.

Is there some way to see grab RGB output for bsuite environments?

For now, I hacked in the cartpole specific viewer from Gym to cartpole.py and cartpole_swingup.py. It works and it doesn't look horrible, but it's not exactly ideal.

[aslanides]

Thanks for the bug report. Yes, this would be a nice-to-have.

[iosband]

+1 - this would be nice!

However, I don't think this is our plan for now.
We want to keep the barrier-to-entry for new experiments to be low, and don't want to force experiments to implement RBG output.

That said, it might be nice to build a generic "Numpy Visualizer" which we could offer to people who want to inspect the output.