Error in class VlogSet

[ananyahjha93]

The default videoLen is set to 4 in the config. Now there are videos where the number of frames is less than 4.

Within the file models/dataset/vlog_train.py, line 124 tries to read at least 4 frames for each video and the dataloader crashes for the case where the number of frames is less than 4.

Crash at line 132 while reading the image

img = load_image(img_path)

Error:

File "/beegfs/ahj265/self_supervised_tracking/models/dataset/vlog_train.py", line 175, in __getitem__
    img = load_image(img_path)  # CxHxW
  File "/beegfs/ahj265/self_supervised_tracking/utils/imutils2.py", line 23, in load_image
    img = img.astype(np.float32)
AttributeError: 'NoneType' object has no attribute 'astype'

Is there a preprocessing step I am missing where you filter out such videos?

[ananyahjha93]

I found that the ffmpeg module is reducing the number of frames present in a video, when it resizes it to 256. Is this the expected behaviour? Which version of ffmpeg was used to preprocess the datasets?

[xiaolonw]

I don't think the number of frames will be reduced after resizing the frames.

I installed ffmpeg directly from anaconda, the version is:
ffmpeg version 4.0.2 Copyright (c) 2000-2018 the FFmpeg developers

[ananyahjha93]

Using the latest version of ffmpeg solved the issue of reduction in frame count. Also, the scripts were not able to generate any data after that.

Simply changing lines 53-54 to

inname = folder_path + videoname + 'clip.mp4'
outname = output_path + videoname

in both 'downscale_video_joblib.py' and 'extract_jpegs_256.py' works for the problem of no data generation.

[Kuldeep-Attri]

Hey @ananyahjha93 @xiaolonw, I am still getting the same error(even after using the latest version of ffmpeg(4.1.3) and changing the line 53-54 in both files). All the preprocess steps are running without any error but I get the exact same error(as @ananyahjha93 carshes at line 123 in vlog_train.py) while running the "python3 train_video_cylce_simple.py ...".

[ananyahjha93]

@Kuldeep-Attri You will need to check if all data is being generated by the scripts. What happened with me was that more than half of the videos were not being converted into individual frames due to the HPC disk running out of i-nodes. The code does not give any exception when a video fails to convert into images.

The solution to the i-node problem for me was to store all images into a single hdf5 file and use that to load images from the data loader.

[yangyuke001]

@Kuldeep-Attri @xiaolonw @ananyahjha93
changing line 7 in genvloglist.py using full path :
foldername = '/home/usrname/TimeCycle/dataset/vlog_frames_12fps/'

instead of relative path:
foldername = '../dataset/vlog_frames_12fps/'

And you will get a new vlog_frames_12fps.txt ,it work for me cause in file imutils2.py line 23:

img = cv2.imread(img_path)

img_path got '../dataset/vlog_frames_12fps/...'
can not find the picture files

[ly-zhu]

I had the similar problem of

File "/beegfs/ahj265/self_supervised_tracking/models/dataset/vlog_train.py", line 175, in getitem
img = load_image(img_path) # CxHxW
File "/beegfs/ahj265/self_supervised_tracking/utils/imutils2.py", line 23, in load_image
img = img.astype(np.float32)
AttributeError: 'NoneType' object has no attribute 'astype'

there are around 15 empty folders in the extracted frames dataset.
solution is:
add script in "preprocess/genvloglist.py" to only add the folder path (which is not empty) to vlog_frames_12fps.txt