Implementation Method of NLQ

[krkrkrrk]

Hello,

The report mentions that the NLQ is based on GroundNLQ, but could you explain specifically how it is implemented?
Are you replacing the InternVideo features with EgoVideo features?
If so, I would like to conduct a replication experiment using the GroundNLQ code. Could you explain the steps for replacing InternVideo features with EgoVideo features?

[ayiyayi]

Hello krkrkrrk,

Thank you for your question! I’m happy to clarify.

1. We use EgoVideo features for both pretraining and fine-tuning. To replace the features, you’ll need to update the paths in the configuration file. Specifically, modify the following entries:

dataset:
  video_feat_dir: your_feat_dir,
  text_feat_dir: your_text_feat_dir,
  val_text_feat_dir: your_val_text_feat_dir

2. Additionally, you will need to adjust the _get_query_feat_by_qid() and _get_video_feat_by_vid() functions in libs/datasets/ego4d_loader.py to read the data from the .pt format, which is used for EgoVideo features.

[krkrkrrk]

Thank you very much.

Can the accuracy of the NLQ in the paper be reproduced using the publicly available 4-frame model checkpoint? Or is the 16-frame model necessary?

If the 16-frame model is required to reproduce the accuracy, do you have plans to release the checkpoint for the 16-frame model? Additionally, if I want to train the model myself, is it sufficient to simply change the num_frames in backbone/model/configs.py to 16? Are there any other modifications needed?