add prediction direction caps report

[XinfengZhang]

report which frame type is supported by driver

Signed-off-by: XinfengZhang carl.zhang@intel.com

[mypopydev]

I think VAConfigAttribEncMaxRefFrames have implicit support the Frame Type query, why we need another repetitive caps for Frame type query?

[XinfengZhang]

VAConfigAttribEncMaxRefFrames can only represent the forward/backward reference number, but cant distinguish the difference between P and low delay B. although low-delay B have two reference list, but both reference list only has forward reference frame.

[fhvwy]

I think you need to define more precisely what this actually means for each codec.

For H.264 / H.265, the low-delay B type gives you two lists but disallows bidirectional prediction modes in the macroblocks? Or would there be a requirement that the POC values must always be increasing?

For VP9, the P type constrains all reference frames to have the same sign bias, while the two B types allow different sign bias but for the low-delay B type bidirectional prediction is not allowed?

For MPEG-2, the P and B types are the usual frame types, and low-delay B is not allowed?

[XinfengZhang]

@fhvwy yes,agree with you, frame types definition should be different with different codec. but there should be some duplicate. adding some comments to describe low delay is only for h264 and h265?

[fhvwy]

Sure, if the low-delay type doesn't apply to others then they don't need to be defined there. It does still need a clear definition of what the frame types mean for the codecs which do support them, though.

[ceyusa]

https://bugzilla.gnome.org/show_bug.cgi?id=796984 would depend on this API

[XinfengZhang]

@fhvwy hi Mark, how about current solution. combine the frame type and active reference frame number together to support low delay b?

[fhvwy]

For H.26[45] this is really slice types rather than frame types? (Frame types aren't really a thing in those codecs.)

Perhaps a table defining it precisely would make this clearer. From what you've said I think what you want is:

Codec	Always available	Also available with VA_FRAME_TYPE_P	Also available with VA_FRAME_TYPE_B
MPEG-2	picture_coding_type = 1	picture_coding_type = 2	picture_coding_type = 3
H.264	slice_type = 2 or 7	slice_type = 0 or 5	slice_type = 1 or 6
H.265	slice_type = 2	slice_type = 0	slice_type = 1
VP8	key_frame = 1	key_frame = 0	-
VP9	frame_type = 0 or intra_only = 1	frame_type = 1 and intra_only = 0	-
MJPEG	Everything	-	-

[XinfengZhang]

hi @fhvwy , thanks a lot for your comments, agree with you , mostlikely , the P, B is slice type. but it is a little complex. I prefer to change the attrib to prediction direction support.
if only support low delay B: reference list0, reference list 1 represented by VAConfigAttribEncMaxRefFrames should be non zero value. and only forward direction is supported.

[XinfengZhang]

change the PR . report a caps to indicate whether the forward (POC < current frame) or backward (POC> current frame) prediction is supported.

[XinfengZhang]

if no further comments, I will merge it in 1 week

[fhvwy]

I don't understand the intent of this new proposal is. The POC order has no effect on anything the encoder itself supports - the user is providing the reference frames directly, so it doesn't matter what order the frames are output in (though it could make use of it as a hint, and will need it if writing headers). It's also unclear what the meaning would be for VP9, which does not have any global ordering number.

I assume this proposal is still wanting to cover the hardware bug referred to elsewhere in which some sorts of frames are not supported. Would it be possible to explain precisely what the error is there?

[XinfengZhang]

hi ,@fhvwy , the issue is, media driver low power HEVC encode just support I frame and low delay B frame (Generalized P and B picture). which need the L0 & L1 is totally same. so we need a capability to export it. for low delay B frame, the two reference list is same previous frames. no reference frame in future. the old attribute VAConfigAttribEncMaxRefFrames just can identify the reference number. want an interfaces tell application , driver just support low delay B , not normal B.

such like:https://github.com/intel/libva/blob/master/va/va_enc_hevc.h#L462
when application choose hierarchical B, it also can choose low delay B to replace P frame.
about VP9, 3 reference frames from 8 frames. it should not care about this attributes.

ps: I need to check whether media driver can enable normal B frames , will update later

[XinfengZhang]

intel ICL low power encode, no future frame in L1. I afraid that some one set the B frame, and set normal reference frames, but encoder failed because driver only support low-delay B. it's why I want to add this attributes, do you have other ideas to let application know the limitations?

[cwang105]

Did not follow the thread. But adding a capability attribute for this (random access B) is reasonable. Thanks, Ce From: XinfengZhang [mailto:notifications@github.com] Sent: Saturday, May 25, 2019 8:43 AM To: intel/libva <libva@noreply.github.com> Cc: Subscribed <subscribed@noreply.github.com> Subject: Re: [intel/libva] add prediction direction caps report (#220) intel ICL low power encode, no future frame in L1. I afraid that some one set the B frame, and set normal reference frames, but encoder failed because driver only support low-delay B. it's why I want to add this attributes, do you have other ideas to let application know the limitations? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub<#220?email_source=notifications&email_token=AISIW3X446WSQYECUKL2QR3PXFM7JA5CNFSM4FFEYGJ2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODWHUDUA#issuecomment-495927760>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AISIW3UJVZX2MFRMEEU5LJTPXFM7JANCNFSM4FFEYGJQ>.

[xhaihao]

How is the frame prediction deduced? It would be better to detail it in the comment.

You defined VA_PREDICTION_DIRECTION_PREVIOUS (1) and VA_PREDICTION_DIRECTION_FUTURE (2) only, how is the value set to 0?

[XinfengZhang]

if this attrib is not present, or value == 0,
* frame prediction can be deduced by VAConfigAttribEncMaxRefFrames.

[xhaihao]

LGTM with the latest version