[gstreamer-vaapi][avc cbr encode] Can not generate correct cbr stream if VAEncMiscParameterRateControl->max_qp=0

[HeJunyan]

If VAEncMiscParameterRateControl->max_qp set to 0 while VAEncMiscParameterRateControl->min_qp is not 0
The gst cmmand:

GST_VAAPI_ALL_DRIVERS=1 LIBVA_DRIVER_NAME=iHD gst-launch-1.0 videotestsrc num-buffers=1000 ! capsfilter caps=video/x-raw,format=NV12,width=720,height=480,framerate=30/1 ! vaapih264enc rate-control=cbr keyframe-period=30 num-slices=1 max-bframes=0 bitrate=500 ! capsfilter caps=video/x-h264,profile=main ! h264parse ! filesink location=test.h264

Generate big size h264 stream. If VAEncMiscParameterRateControl->max_qp set to 0 while VAEncMiscParameterRateControl->min_qp is not 0, the driver seems to use the min_qp generate very good quality stream with much bigger bitrate . VAEncMiscParameterRateControl->max_qp should use 51 when it is set to 0(ignored)
But in now seems use VAEncMiscParameterRateControl->min_qp

[uartie]

Is this the same root-cause for VBR issue too?

[HeJunyan]

Gst's encoder pipeline setting is the same for CBR and VBR mode. The iHD internal handle logic may be different, but I think they are probably the same issue.

[wangyan42164]

@HeJunyan Could you please help test my PR? Thanks.