Getting the GLFW context of "Simulate" in Python

[Balint-H]

Hello, following up on #741, is it possible to get the GLFW context/window of the interactive viewer in python? This is for the purposes of registering keyboard/mouse events. I tried using glfw.get_current_context() at different stages, but the pointer it returns cannot be used by the the callback functions of mujoco.glfw without an access violation exception.

If we had access to it then it would be a really convenient way to extend the functionality of the viewer (e.g. by allowing user controls with unused keys like on the numpad, or overlaying graphs in real time).

I know these are all possible with other rendering methods of MuJoCo, but I do enjoy the easy setup and rich GUI experience of the viewer.

Thanks!

[saran-t]

This is on the roadmap, but won't make it to the upcoming release next week unfortunately.

[ripkea]

Are there any updates on this? I want to access the context while using viewer.launch_passive, but I can't seem to do that. Thanks!

[saran-t]

I'm about to make a commit that exposes a keypress callback in launch_passive, which seems to be the main use case requested via this issue.

The commit will mark this issue as Fixed. If this isn't sufficient for you, please open new issues with specific requests. Providing direct access to the actual GLFW window is tricky especially on macOS where there are threading concerns, however if you let me know exactly what you need to achieve we can possibly find alternative means of going about it.