No-reference video quality assessment with guidance of keyframe extraction
Video quality assessment (VQA) is a foundational research in computer vision that aims to simulate the human visual system to evaluate video quality and determine its quality level. The most fundamental factor affecting the accuracy of the final score predicted by the video quality assessment model is the extracted frames. Most existing methods prioritize the study of refining feature extraction to improve performance while ignoring improvements in initial frame extraction. To solve these problems, we propose a no-reference video quality assessment with the guidance of keyframe extraction(KFVQA), which aims to improve model performance through keyframe extraction. Specifically, we propose a keyframe extraction module to ensure that the extracted frames present diversity and greater representativeness in content, and avoid excessive similarity and repetition rates. Secondly, we utilize the self-attention for feature focusing module and weighted feature fusion module in KFVQA to better extract spatial features and more accurately focus and emphasize key motion features and spatial feature parts to improve the accuracy and robustness of quality assessment.
pytorch
opencv
scipy
pandas
torchvision
torchvideo
- Extract video frames
python extract_frame.py- Extract motion features
python extract_SlowFast_features_VQA.py - Train the model
python train.pyTest on the public VQA database
python test_on_pretrained_model.pyTest on a single video
python test_demo.py