Change default preprocessing to shortedge 256

docs/data_preparation.md

@@ -68,6 +68,7 @@ The recommended practice is
 
 1. set `$OUT_FOLDER` to be a folder located in SSD.
 2. symlink the link `$OUT_FOLDER` to `$MMACTION2/data/$DATASET/rawframes`.
+3. set `new-short` instead of using `new-width` and `new-height`.
 
 ```shell
 ln -s ${YOUR_FOLDER} $MMACTION2/data/$DATASET/rawframes

docs/faq.md

@@ -29,6 +29,10 @@ If the contents here do not cover your issue, please create an issue using the [
     If users encounter FileNotFound error for the first or last frame of the data, there is a need to check the files begin with offset 0 or 1,
     that is `xxx_00000.jpg` or `xxx_00001.jpg`, and then change the `start_index` value of data pipeline in configs.
 
+- **How should we preprocess the videos in the dataset? Resizing them to a fix size(all videos with the same height-width ratio) like `340x256`(1) or resizing them so that the short edges of all videos are of the same length (256px or 320px)**
+
+    We have tried both preprocessing approaches and found (2) is a better solution in general, so we use (2) with short edge length 256px as the default preprocessing setting. We benchmarked these preprocessing approaches and you may find the results in [TSN Data Benchmark](https://github.com/open-mmlab/mmaction2/tree/master/configs/recognition/tsn) and [SlowOnly Data Benchmark](https://github.com/open-mmlab/mmaction2/tree/master/configs/recognition/tsn).
+
 ## Training
 
 - **How to just use trained recognizer models for backbone pre-training ?**

tools/data/activitynet/README.md

@@ -66,8 +66,7 @@ Use following scripts to extract both RGB and Flow.
 bash extract_frames.sh
 ```
 
-These three commands above can generate images with size 340x256, if you want to generate images with short edge 320 (320p),
-you can change the args `--new-width 340 --new-height 256` to `--new-short 320`.
+The command above can generate images with new short edge 256. If you want to generate images with short edge 320 (320p), or with fix size 340x256, you can change the args `--new-short 256` to `--new-short 320` or `--new-width 340 --new-height 256`.
 More details can be found in [data_preparation](/docs/data_preparation.md)
 
 ### Step 4. Generate File List for ActivityNet Finetuning

tools/data/activitynet/extract_frames.sh

@@ -1,6 +1,6 @@
 #!/usr/bin/env bash
 cd ../
-python build_rawframes.py ../../data/ActivityNet/videos/ ../../data/ActivityNet/rawframes/ --level 1 --flow-type tvl1 --ext mp4 --task both  --new-width 340 --new-height 256
+python build_rawframes.py ../../data/ActivityNet/videos/ ../../data/ActivityNet/rawframes/ --level 1 --flow-type tvl1 --ext mp4 --task both  --new-short 256
 echo "Raw frames (RGB and tv-l1) Generated for train set"
 
 cd activitynet/

tools/data/kinetics/README.md

@@ -88,8 +88,7 @@ If both are required, run the following script to extract frames.
 bash extract_frames.sh ${DATASET}
 ```
 
-These three commands above can generate images with size 340x256, if you want to generate images with short edge 320 (320p),
-you can change the args `--new-width 340 --new-height 256` to `--new-short 320`.
+The commands above can generate images with new short edge 256. If you want to generate images with short edge 320 (320p), or with fix size 340x256, you can change the args `--new-short 256` to `--new-short 320` or `--new-width 340 --new-height 256`.
 More details can be found in [data_preparation](/docs/data_preparation.md)
 
 ## Step 4. Generate File List

tools/data/kinetics/extract_frames.sh

@@ -9,10 +9,10 @@ else
 fi
 
 cd ../
-python build_rawframes.py ../../data/${DATASET}/videos_train/ ../../data/${DATASET}/rawframes_train/ --level 2 --flow-type tvl1 --ext mp4 --task both  --new-width 340 --new-height 256
+python build_rawframes.py ../../data/${DATASET}/videos_train/ ../../data/${DATASET}/rawframes_train/ --level 2 --flow-type tvl1 --ext mp4 --task both  --new-short 256
 echo "Raw frames (RGB and tv-l1) Generated for train set"
 
-python build_rawframes.py ../../data/${DATASET}/videos_val/ ../../data/${DATASET}/rawframes_val/ --level 2 --flow-type tvl1 --ext mp4 --task both  --new-width 340 --new-height 256
+python build_rawframes.py ../../data/${DATASET}/videos_val/ ../../data/${DATASET}/rawframes_val/ --level 2 --flow-type tvl1 --ext mp4 --task both  --new-short 256
 echo "Raw frames (RGB and tv-l1) Generated for val set"
 
 cd ${DATASET}/

tools/data/kinetics/extract_rgb_frames.sh

@@ -9,10 +9,10 @@ else
 fi
 
 cd ../
-python build_rawframes.py ../../data/${DATASET}/videos_train/ ../../data/${DATASET}/rawframes_train/ --level 2  --ext mp4 --task rgb  --new-width 340 --new-height 256
+python build_rawframes.py ../../data/${DATASET}/videos_train/ ../../data/${DATASET}/rawframes_train/ --level 2  --ext mp4 --task rgb  --new-short 256
 echo "Raw frames (RGB only) generated for train set"
 
-python build_rawframes.py ../../data/${DATASET}/videos_val/ ../../data/${DATASET}/rawframes_val/ --level 2 --ext mp4 --task rgb  --new-width 340 --new-height 256
+python build_rawframes.py ../../data/${DATASET}/videos_val/ ../../data/${DATASET}/rawframes_val/ --level 2 --ext mp4 --task rgb  --new-short 256
 echo "Raw frames (RGB only) generated for val set"
 
 cd ${DATASET}/

tools/data/kinetics/extract_rgb_frames_opencv.sh

@@ -9,10 +9,10 @@ else
 fi
 
 cd ../
-python build_rawframes.py ../../data/${DATASET}/videos_train/ ../../data/${DATASET}/rawframes_train/ --level 2  --ext mp4 --task rgb --new-width 340 --new-height 256 --use-opencv
+python build_rawframes.py ../../data/${DATASET}/videos_train/ ../../data/${DATASET}/rawframes_train/ --level 2  --ext mp4 --task rgb --new-short 256 --use-opencv
 echo "Raw frames (RGB only) generated for train set"
 
-python build_rawframes.py ../../data/${DATASET}/videos_val/ ../../data/${DATASET}/rawframes_val/ --level 2 --ext mp4 --task rgb --new-width 340 --new-height 256 --use-opencv
+python build_rawframes.py ../../data/${DATASET}/videos_val/ ../../data/${DATASET}/rawframes_val/ --level 2 --ext mp4 --task rgb --new-short 256 --use-opencv
 echo "Raw frames (RGB only) generated for val set"
 
 cd ${DATASET}/