libvmaf API does not support `perc*` pool methods

[cosmin]

The libvmaf API used by ffmpeg does not support the perc5, perc10, perc20 pooling methods. Furthermore attempting to pass one of these values will silently fallback to mean as the pooling method due to the implementation in pool_method_map, leading to a confusing user experience (even though the documentation mentions that only min, mean and harmonic_mean are supported).

It would be great to either unify the pooling methods such that perc* works, or to perhaps consider returning an error if an unsupported pooling method is requested.

[HunterAP23]

I'm pretty sure the main reason for this is that FFmpeg uses the compute_vmaf.c code which is set to be deprecated on the VMAF library. That file only has min, mean and harmonic_mean as per this section:

vmaf/libvmaf/src/compute_vmaf.c

Lines 23 to 32 in bf5e496

	static enum VmafPoolingMethod pool_method_map(const char *pool_method)
	{
	if (pool_method) {
	if (!strcmp(pool_method, "min"))
	return VMAF_POOL_METHOD_MIN;
	if (!strcmp(pool_method, "mean"))
	return VMAF_POOL_METHOD_MEAN;
	if (!strcmp(pool_method, "harmonic_mean"))
	return VMAF_POOL_METHOD_HARMONIC_MEAN;
	}

The only other mention of percX pooling methods I could find were only present in the python library which uses numpy to get those percentile points. percX doesn't seem to be implemented in C/C++ for FFmpeg to import.

Not saying it can't be done, but it'd require some work to implement it in C/C++

[cosmin]

Yes, it would require adding percentile support to vmaf_feature_score_pooled in libvmaf.c but it seems like it would be fairly straightforward.

[HunterAP23]

Not trying to plug my code, but I've been working on a python script that takes VMAF report files and turns them into a chart image and a video that graphs the different VMAF values over time. One of the items I provide on the legend is the 1% and 0.1%, and 0.001% lowest percentiles. It's still a work in progress and not yet complete, but if you're willing to use an external program to calculate different percentiles of the different metrics (like PSNR and SSIM) then you can check it out: https://github.com/HunterAP23/VMAF-Over-Time