Code release plan

[Vilonge]

Great work! Is there an open source plan for the code? About what time?

[exiawsh]

@Vilonge
Hello, thank you for your attention! I expect to release the code of data generatition samples (using GPT-4) before 6.15. I will release the training code before 7.1.
I integrated the code of llava into mmdet3d 1.0 during the development, but the mmdet3d 1.0 version does not support deepspeed, resulting in higher training memory.
So I plan to refactor on mmdet3d 1.1 to support deep speed.

[Vilonge]

Thanks for your reply. Looking forward to open source.

[exiawsh]

We release the caption & conversation data. Hope it will help you.

[Vilonge]

Thanks. Indeed.

[Grace0413]

Hi, where is the caption & conversation data? I could only find omnidrive_data.zip, where there is only annotation json files, but I do not know how the .json is connected to the actual image.

[exiawsh]

@Grace0413 Hi, please refer to this issue:
#2 (comment)
The name of json files are the sample_tokens in nuscenes dataset.

[Grace0413]

Thanks for your reply! I have read the code you provided:

sample_idx=info['token'] # ('sample_token' for nusc)
with open(vqa_path+results['sample_idx']+".json", 'r') as f:
data_qa = json.load(f)

Maybe this is a stupid question: what is info and what is results here? I'm sorry but I have no idea how to obtain info['token'] and results['sample_idx'] without knowing what info and results are.

[exiawsh]

@Grace0413
Do you have any experience with the nuscenes api? For each timestamp, you will have sample['token'], which you can use to index the corresponding image.
info['token'] and results['sample_idx'] = sample['token'] here.
You can refer to the StreamPETR dataset section from my previous work.

[ltp1995]

Hi, thanks for your great work! Could you provide the evaluation codes in advance, especially how to calculate the metrics such as Collision && Intersection? Maybe some essential files (e.g., gt_fut_trajs, gt_occ, results of ominidrive) are also needed.

[Grace0413]

Thanks, I will try it out!