How to visualize planning results (as videos)

[synsin0]

Thanks for your great benchmark. Now I can run_transfuser.sh on miniset. I'd like to ask how to visualize planning results as videos for each scenario. Great thanks if you would help!

[DanielDauner]

Hi @synsin0, thanks for your interest in our project!

We have some methods to visualize sensor data and agent trajectories in tutorial_visualization.ipynb. We might add more interactive visualization tools in the future but do not have a timeline yet.

I hope that helps for now!

[synsin0]

I find that during training stage the validation results of transfuser are written into tensorboard. I wonder whether this visualization also applies to evaluation stage in run_transfuser.sh.

[DanielDauner]

The visualization of transfuser in the tensorboard is done as a callback in transfuser_callback.py. This visualization operates differently, as it mainly visualizes the input-output tensors of TransFuser. There might be options to integrate these visualizations during evaluation, but this has not been implemented yet.

Given a token (e.g. from a failed scene during evaluation), you can still visualize the environment and agent output in tutorial_visualization.ipynb.

[synsin0]

I adapt the transfuser callback in evaluation stage and successfully draw similar pictures compared with training stage. Thanks for your help!

[Hongbo123467]

I have the same question, could you please share how to adapt the callback in evaluation stage@synsin0