a question about our eval results

[harlanc]

Hi tommyhq,

We now use your model to evaluate our own video samples,

the only differences between the samples are video bitrate, the bitrate of reference video is 1600Kbit/s,
and the 3 dis ones are 400kbit/s ,800kbit/s and 1200kbits/s,

Why higher bitrates has a lower PRED score??

[tommyhq]

Thanks for pointing out this issue.

First, I noticed the bitrate of reference video is 1600 Kbit/s, and I believe this is not the typical, i.e. lossless, 'reference' for a FR-VQA metric.

Second, the framerate is 16 fps. That is a number rarely encountered.

Could you give more information about the sequence? For example, how the sequence was captured? Any preprocessing on the most original reference to get the 1600Kbit/s 'reference' ? Meanwhile, is it possible to upload those sequences so we could have a look?

[harlanc]

I didn't read README carefully:

We are not sure about the performance when the code is run with the following scenario,

PGC with other distortion types, especially time-related distortions.
PGC with post-processing filters, like de-nosing, super-resolution, artifacts reduction, etc.
UGC videos with pre-processing filter.
UGC videos compressed with common codecs.

We tried to eval a UGC dataset using the pretrained model, this usage was wrong since the model was trained on a PGC dataset.

Next maybe we will train our own model on UGC dataset. Thanks for @tommyhq's help.