How to visualize Fig.4 in your paper?

[synsin0]

Thanks for your great work! I am interested in visualizing occupancies behind the scenes as depicted in Fig.4 in your paper. I succeed in training in KITTI-360 and generate depth sequences, but the transition to BEV are not very successful. Do you use the script: python scripts/videos/gen_vid_transition.py to get similar results as Fig. 4?

[Brummi]

Hi Yining,
thank you for your interest in our project!
Great that you managed to train on KITTI-360.

The script you mentioned is mainly meant for transitions in videos (e.g. the one in the readme).
The result is a top-down depth map.

The visualizations in Fig. 4 are not depth map based, but directly visualize density. To generate these images, you can use the gen_imgs.py or gen_img_custom.py script. In this code, these top-down visualizations are referred to as profile.

Do let me know if you have any further questions.

[synsin0]

I have found that and generated the video on density fields! Thanks for your timely response. By the way, you provide some waymo apis, will you support waymo dataset in the future?

[Brummi]

I briefly experimented with Waymo, but we found several issues with the dataset. (namely too narrow FoV, poor image quality due to rolling shutter, inconsistent exposure time and strange lens effects, too much traffic).
we didn't pursue it any further, but think that our method could definitely be extended to Waymo with a little extra work.

while we don't plan to do it ourselves for now, I left the dataloaders in the repo to make it easier for others to extend our method.
best,
felix

[synsin0]

I may extend the experiments to other datasets. I have one last question. In function 'render_profile', 'alphas' is the probability for voxel. the line 'profile = (alpha_sum<=8).....'. Why choose 8 in this 384x384x64 voxels? Is it an empirical value?

[Brummi]

Hi!
Yes, you are right about the alphas.
the value 8 is empirically chosen (1/8th of 64 evaluations), as it produces clean results while retaining details.
best,
Felix