”dis" and "ref" in dataset json file

[ymxcywl]

Very grateful the release of DVQA.
I have several questions.

1. The dis yuv and corresponding ref(has the same index in json file) yuv are yuv files clipped from one source video, and must have the same size when coverted to tensor. Am i right?
2. What is the meaning of ref and dis(if i have a mp4 video to predict mos, according to what rule can I create ref and dis file)? Please tell me so i can create my own dis and ref yuv file, thank you.

[tommyhq]

Hi @ymxcywl ,

1. The ref and dis must have same size as input. The ref, an abbreviation of reference, is the lossless version of a vide. Then dis, an abbreviation of distorted, is corresponding distorted version of aforementioned ref.

2. Originally we have lossless ref.yuv, then we compress it to dis.mp4, and one would want to predict the MOS of dis.mp4.

3. If you just want to evaluate on your own video, simply create dataset.json with the following format. Probably, you need to conduct subjective test to get the MOS, i.e. ground truth, of your videos.

{
    "test": {
        "dis": ["dis_1.yuv", "dis_2.yuv"],
        "ref": ["ref_1.yuv", "ref_2.yuv"],
        "fps": [30, 24],
        "mos": [94.2, 55.8],
        "height": [1080, 720],
        "width": [1920, 1280]
    },
    "train": {
        "dis": ["dis_3.yuv", "dis_4.yuv"],
        "ref": ["ref_3.yuv", "ref_4.yuv"],
        "fps": [50, 24],
        "mos": [85.2, 51.8],
        "height": [320, 720],
        "width": [640, 1280]
    }
}

[ymxcywl]

Thank you so much.
So, is DVQA a full reference assessment?
If it is difficult to get orignal lossless version of a video, is it reasonable to use DVQA to predict mos ?(eg: dis is the video of low bps, while ref is the original mp4 video)

[tommyhq]

Yes. DVQA is a FR metric.

If you have the original.mp4 video in hand, and suppose it was losslessly compressed from origihal.yuv, all you need to do is decode it back to yuv format. A decoding script is provided at decode_stream.py