data preparation step

[justlovebarbecue]

Hi Tengda, thanks for sharing your code. I found CoCLR has no data preparation instruction and could you please provide some details about data preprocessing from the raw data? I found similar instruction in DPC and MemDPC, are they feasible for CoCLR?

[TengdaHan]

The instructions in DPC and MemDPC are still feasible. Basically what you need to do is just extract video frames and optical flow (code) and prepare a dataloader.
In this repo, there is one difference that I pack the entire frame dataset into lmdb (code). But this step is not essential, as long as the dataloader returns video tensors in the correct format.

[justlovebarbecue]

Hi Tengda, thanks for your reply. I found that there is a dataloader in your code, which is for the LMDB format. Hence, if I choose the LMDB format, I can directly use your code, while if I choose my own data format, I need to write corresponding dataloader. Am I right? Thanks!

[TengdaHan]

Yes, that's right.
If you choose to read from video frames, you can still refer to https://github.com/TengdaHan/MemDPC/blob/master/memdpc/dataset.py, with some modifications in the sampling method (i.e. sampling twice for two random augmentations etc.)
If you choose other ways to load data, you have to ensure your dataloader supports data augmentations & returns tensors with the same shape.

[justlovebarbecue]

Hi Tengda, thanks for your detailed reply. I will try it based on your code.
Thanks!

[farleylai]

While several related works simply download saved jpeg frames from the two-stream repo, CoCLR seems to have different frame saving parameters (resolution and quality) which retains the original video format better.
Does CoCLR extract the frames following the procedure in the DPC repo as follows? That is resize to 240 with the aspect ratio preserved and jpeg quality set to 80 using OpenCV?

https://github.com/TengdaHan/DPC/blob/master/process_data/src/extract_frame.py