WhateverGreen AMD DRM decoder override breaks FaceTime Video decoding

[milmber]

Hi @vit9696,
I have noticed that when setting com.apple.AppleGVA gvaForceAMDKE to true to force hardware decoding (as per https://github.com/acidanthera/WhateverGreen/blob/master/Manual/FAQ.Chart.md.
), this disables FaceTime remote participant video decoding.

I suspect this may be because somehow FaceTime doesn't accept/support hardware video decoding and instead will always default to use the software video decoder ?

Setup

• Gigabyte Z490, i9-10850K
• SMBIOS iMac 20,2
• iGPU enabled in BIOS
• shikigva=80 boot argument
• AAPL,ig-platform-id set to 0300C89B (Desktop iGPU is only used for computing tasks and doesn't drive a display)
• RX 5600 XT dGPU
• OpenCore 0.6.8 & WhateverGreen v1.4.9
• MacOS Big Sur 11.2.3

Steps to reproduce

1. Working Hackintosh with working FaceTime, iMessage, etc
2. Execute defaults write com.apple.AppleGVA gvaForceAMDKE -boolean yes (allows DRM, etc to work)
3. Open a FaceTime call and wait for the call to be accepted
4. Audio works, local video streams to remote participant, but remote video fails with a grey screen and "Poor connection" message.

Executing defaults write com.apple.AppleGVA gvaForceAMDKE -boolean no fixes the issue and remote participant video is streamed and decoded (however DRM no longer works)

Some syslog excerpts below from when the failure occurs which may or may not be relevant.

error	18:39:42.142030+0100	avconferenced	CMIOHardware.cpp:379:CMIOObjectGetPropertyData Error: 2003332927, failed
error	18:39:42.212634+0100	VDCAssistant	[guid:0x1412250005ac8508] StartStream SetPipePolicy returned 0xe00002c2
error	18:39:43.660315+0100	VTDecoderXPCService	Invalid display 0x00000000
error	18:39:43.673227+0100	avconferenced	 [ERROR] -[TundraVideoCapture processFrame]:511 dropping out-of-order camera frame
error	18:39:44.337081+0100	avconferenced	 [ERROR] VideoTransmitter_GetFramerate:4235 Failed to get framerate for stream[0] with error -2146238447
error	18:39:44.337996+0100	avconferenced	 [ERROR] VideoTransmitter_GetFramerate:4235 Failed to get framerate for stream[0] with error -2146238447
error	18:39:44.657691+0100	avconferenced	 [ERROR] VideoDecoder_VTDecompressionCallback:140 VCPDecompressionSession Error: Decoder returned a NULL frame with showFrame 1, RTPTimestamp=134407, decoding order 0, tileID 0

[vit9696]

This is not WhateverGreen decoder, but Apple. The behaviour is normal. Such an override is not recommended for daily use.