CSV file error in generating tapvid kinetic data.

[songtianhui]

I follow the steps in https://github.com/deepmind/tapnet/blob/main/data/README.md to generate tapvid kinetic dataset.
When processing the clips, ie. run the generate_tapvid.py scipt, I encounter AssertiongError, message is shown below:

  File "generate_tapvid.py", line 177, in main
    videos = csv_to_dataset(FLAGS.csv_path, videos_path)
  File "generate_tapvid.py", line 73, in csv_to_dataset
    assert len(row) == 3 + 3 * 250, f"{len(row)}"
AssertionError: 711

It looks like that the file tapvip_kinetic.csv contains some invalid format, it is extracted from https://storage.googleapis.com/dm-tapnet/tapvid_kinetics.zip.
Can you check it?
Thanks!

[yangyi02]

Hi, we just updated our method of visualizing tapvid_kinetics.

Could you download the latest tapvid_kinetics.csv and kinetics videos, follow the updated tapnet/data/README.md and try again?

Thanks.

[songtianhui]

Thanks for replying!
The script can run well now.
But the visualization video for kinetic looks not correct, say, the points track inaccurately. Videos for davis and rgb-stacking are accurate.
Is it possiple that the raw videos are different? I follow the repository https://github.com/cvdfoundation/kinetics-dataset to download kinetic700-2020 val split videos. And the videos are in the name pattern of f'{youtube_id}_{start_time_sec:06}_{end_time_sec:06}.mp4', not described in https://github.com/deepmind/tapnet/tree/main/data as f'{youtube_id}.mp4'.
Besides, I check the sha256sum of both csv files before and after updating, the are the same.

[songtianhui]

Thanks for replying! The script can run well now. But the visualization video for kinetic looks not correct, say, the points track inaccurately. Videos for davis and rgb-stacking are accurate. Is it possiple that the raw videos are different? I follow the repository https://github.com/cvdfoundation/kinetics-dataset to download kinetic700-2020 val split videos. And the videos are in the name pattern of f'{youtube_id}_{start_time_sec:06}_{end_time_sec:06}.mp4', not described in https://github.com/deepmind/tapnet/tree/main/data as f'{youtube_id}.mp4'. Besides, I check the sha256sum of both csv files before and after updating, the are the same.

I think I find the reason. When reading the video using media.read_video(), the number of frames is usually not exact 250, while the point labels are 250 frames. This results in painting unmatch.
How do you resolve it?

[HarryHsing]

Hi, we just updated our method of visualizing tapvid_kinetics.

Could you download the latest tapvid_kinetics.csv and kinetics videos, follow the updated tapnet/data/README.md and try again?

Thanks.

Hi yangyi, thanks a lot for your great work! Maybe you could upload the corresponding Kinetic videos for us to download directly, that would be much appreciated!

[yangyi02]

Thanks for your feedback, we start to check the video mismatch issue now, will keep you updated.

[yangyi02]

Apologize for the inconvenience, we have spotted the problem and update the README and codebase for kinetics data processing and visualization. Please check again and let us know if there is still issue.

Thanks for your patience.

[songtianhui]

Apologize for the inconvenience, we have spotted the problem and update the README and codebase for kinetics data processing and visualization. Please check again and let us know if there is still issue.

Thanks for your patience.

It works well now!
Appreciating for your awesome job!

[yangyi02]

Great! Will close it now, but please let us know if further issues.