Begin refactor of wvdecrypter, sample-aes-cbc support

[glennguy]

This is the result of looking into adding support for CBC encrypted streams. Not very common at the moment but I imagine it would be more popular with 'apple native' services - Fairplay uses this scheme. CBC uses a 'pattern' based encryption, where there is a pattern of clear vs encrypted samples. Typically (and in the case of the test video) there is 1 encrypted sample followed by 9 unencrypted or 'skipped' samples. An advantage of this method can be lower cpu utilization.

main.cpp doesn't want to know the concrete decrypter class, only the generic ap4 cencsinglesampledecrypter. This will provide the flexibility needed in future if there's a Playready or Fairplay decrypter module developed. Therefore for this AES-CBC feature in order to prevent yet more patches to bento4 or to cast into the concrete wv cenc ssd class we have now the 'Adaptive_CencSingleSampleDecrypter' base class which has this extended functionality.

If CBC encryption is found upon stream initialisation we set the decrypter up appropriately.

NOTE: This doesn't include support for the CBCS decoding feature on Android, changes and possibly API bump will be required in Kodi. Follow-up PR to come very soon.

Tested with

#KODIPROP:inputstream=inputstream.adaptive
#KODIPROP:inputstream.adaptive.manifest_type=mpd
#KODIPROP:inputstream.adaptive.license_type=com.widevine.alpha
#KODIPROP:inputstream.adaptive.license_key=https://proxy.uat.widevine.com/proxy?provider=widevine_test||R{SSM}|
https://storage.googleapis.com/wvmedia/cbcs/h264/tears/tears_aes_cbcs.mpd

for CBC, and for regular CTR tested with:

#KODIPROP:inputstreamaddon=inputstream.adaptive
#KODIPROP:inputstream=inputstream.adaptive
#KODIPROP:inputstream.adaptive.manifest_type=mpd
#KODIPROP:inputstream.adaptive.license_type=com.widevine.alpha
#KODIPROP:inputstream.adaptive.license_key=https://cwip-shaka-proxy.appspot.com/no_auth|Content-Type=|R{SSM}|
https://bitmovin-a.akamaihd.net/content/art-of-motion_drm/mpds/11331.mpd

on Windows 10.

edit: some additional background

[CastagnaIT]

nice addition! only some code style requests

[CastagnaIT]

i havent tested the feature, just check code style

[CastagnaIT]

review second round done,
i have tested it on Windows 10 and no problem,
but when i have tested it on CoreElec (last Kodi 20 nightly) every time i try to play multiple times the first test stream Kodi always crashes (usually after 2 or 3 times), this not happens with the second test stream or others test streams

Crash logs:
kodi_crashlog_20220128145622.log.txt
kodi_crashlog_20220128153636.log.txt

[CastagnaIT]

IMO this is not very clear, here we should make it easier understand that we handle flag operations,
at least we should change all these integer variables to handle a enum, something like this:

enum MediaTypeFlags
{
MEDIA_TYPE_UNDEFINED = 0,
MEDIA_TYPE_VIDEO,
MEDIA_TYPE_AUDIO
...

if it is too much work for this PR, keep it for later cleanup

[glennguy]

Thanks for pointing out however let's keep for a later cleanup

[glennguy]

Hmmm very interested in the crash on CoreELEC. Must be some sort of race condition, it seems very similar to other reports we've had from crashes on those platforms. But very good that it seems fairly easy to reproduce.

First crash shows that ISA isn't even running still which makes me think it's to do with the wv timer renewal introduced in #589 and modified again in #729

One hint for us to look at is that there's actually a callback that is invoked by the CDM that we're not currently using which may be the key to doing this right.

I know that the changes here are seeming to cause crashes on your setup but I feel that the changes merely expose an issue that's already there. One thing I didn't mention in the OP is this introduces using separate CDM sessions for audio and video - you'll notice in the log that the license is retrieved twice. Same behaviour as Android is since #734

Or, alternatively something inside ISA is causing a failure or exit but it's not waiting properly for CDM session to finish closing, therefore crash of all Kodi when it tries to call back.... will have to see.

I'll have a dig very soon.

[glennguy]

@CastagnaIT just a thought: would switching to separate audio and video widevine sessions cause issues with the pre-init for NF? Is the platform that you model the plugin from use separate audio/video sessions for playback or just 1?

[CastagnaIT]

i have some questions

if the audio track is not DRM protected, is the licence still required/requested?

you'll notice in the log that the license is retrieved twice. Same behaviour as Android is since

because i have tried this PR on ntfx but i am not able to see in the log the two license requests,
can you explain where is in log?
https://paste.kodi.tv/azoliyifuv.kodi

If i use the licese provided with the licensedManifest request, this give me only one license data, not two licenses,
ATM I can request a license separately then is not a big problem,
but i don't know if request two licenses to the server is the right way
at least in the netfx website service this has never happened over the years

For the sake of completeness, after the implementation of thelicensedManifest in website, there is an additional licese request, but this happens only after the video playback is started, which might be the reason why playback fails in my WIP PR after some minuts

[glennguy]

if the audio track is not DRM protected, is the licence still required/requested?

Sorry I should have clarified, my question should have been this condition first. No licence request for audio if not protected. In this case we are good and it makes sense.

I haven't had much time this weekend to look further but I'll have a go in the evenings this coming week. My gut feeling is this has to do with the CDM timer functions. I logged the timer getting started and finishing, it seems to run every second. Maybe some sort of race condition....

edit:
Just spotted this

inputstream.adaptive/wvdecrypter/cdm/media/cdm/cdm_adapter.cc

Line 524 in dc500f5

if (session_active_)

I don't think this is right. If the cdm wants to set a timer it should be set without condition... it's possible that SetTimer is being called by CDM before wvdecrypter has a chance to set the flag.

[glennguy]

@CastagnaIT I've rebased and cleaned up, and retested on LE11 - I couldn't get it to crash on wv 4.10.2252.5
As discussed previously, I fully believe that 4.10.2391.0 is broken and the cause of crashing previously. I tried to run on Raspberry Pi OS and test in a browser - performance rendered it unwatchable.

If you get a chance could you try again with the 'good' CDM?

[glennguy]

Also - I reverted a change made to this line

inputstream.adaptive/wvdecrypter/wvdecrypter.cpp

Line 483 in 781d59e

while (session_.empty() && ++retrycount < 100)

as the variable was global to the cdm adapter and not the session, when setting up the second session the session_active flag would already be true.

[CastagnaIT]

i will have to check i do not remember anymore the situation on my ARM CoreELEC device
i will let you know

[CastagnaIT]

I have done tests

WV 4.10.2391.0 (last):
as you said kodi crash still present, performance audio/video always good,
and after about 1:30 the video become black, kodi try buffering after resume only audio for 2 secs, after that still black screen and no audio until you stop manually the video.
https://paste.kodi.tv/uguwogajon.kodi

WV 4.10.2252.0 (13904):
i confirm no kodi crashes, performance audio/video always good,
but and after about 1:30 the video become black, kodi try buffering after resume only audio for 2 secs, after that still black screen and no audio until you stop manually the video.
https://paste.kodi.tv/norilasima.kodi

WV 4.10.2252.0 (13816) and WV 4.10.2252.5 (14150):
same behaviour of 4.10.2252.0 (13904) but kodi crashed

after these tests on multiple versions i don't know if we can call it as library bug, crash always occurs at playback startup,
i would not want it to be a side effect of the problem we already have, namely that the WV library is not unloaded from memory, maybe it is possible that some process is like zombie

maybe wait for a new WV version to be released and see if crash still occurs
or merge it but we have to add in Wiki a big warning for unstable feature

[glennguy]

Thanks for the detailed testing!

I haven't been able to find evidence that it's not unloaded from memory. What I can say is there's the long standing issue in CE/LE that seems to do with their filesystem/caching that usually requires a restart after updating the add-on, maybe that has to do with not 'unloading'?

These different versions (13816, 13904, 14150) are coming from different device images right? Are you able do checksums on the two different 4.10.2252.0 versions? Maybe one of these is being optimised in a certain way that causes issues for us, especially when combined with the TLS relocation 'fixes' that are now required in LE/CE?

Regarding the black screen, can you check that the changes from d6b3c52 are in your source? It looks to me like the incorrect init segment is downloaded on stream change (480p instead of 1080p).

[CastagnaIT]

i dont know i have done exactly what you need
i have used md5sum linux command:
WV 4.10.2252.0 (13816): 6f6a10bb4a0cfc492815cab996addc4f
WV 4.10.2252.0 (13904): 6f6a10bb4a0cfc492815cab996addc4f
WV 4.10.2252.5 (14150): 9dfe48c60265444f08548e4769e00d8f

Regarding the black screen, can you check that the changes from d6b3c52 are in your source? It looks to me like the incorrect init segment is downloaded on stream change (480p instead of 1080p).

i have builded from your branch, your branch include that commit

These different versions (13816, 13904, 14150) are coming from different device images right?

yes ISHelper download from different devices images
i think could be also possible that some devices are optimized in different way

[glennguy]

Thanks for md5sums, that was what I was looking for. Both 2252.0 are the same file, interesting that there are crashes on one and not the other. It's making me think about the apparent file caching issue.

Ok, so I think the black screen issue is to do with this 2022-03-14 16:40:51.040 T:4113 ERROR <general>: CDVDVideoCodecAmlogic::Open - InstanceGuard locked in the log, not sure what it means and haven't researched yet. No problems on RPi3 with stream changes though.

i have builded from your branch, your branch include that commit

my bad, I read the log wrong, all is working as it should in this area.

Overall I'd like to just get this in. The feature is unlikely to be used at the moment, and at least it's prepared for the future. If there are issues later on then we can revisit then. When I move my other devices in the house to Kodi 20 I can do some more work.

I found with testing that the SetTimer function was the point at which crashing occurred, usually on the 10th (sometimes 7th) call to TimerExpired. Easiest way to stop timer is to comment out here

inputstream.adaptive/wvdecrypter/cdm/media/cdm/cdm_adapter.cc

Line 507 in 781d59e

std::thread(timerfunc, shared_from_this(), delay_ms, context).detach();

Even 4.10.2391.0 seemed to behave nice (although with stuttering/slowness) with this.
This observation makes me think we're not doing anything wrong on our side.

[CastagnaIT]

InstanceGuard locked error not come from code of xbmc original repository,
seem to be a CoreElec implementation and now i realized that this happens after DEMUX_SPECIALID_STREAMCHANGE
i will try update the latest Coreelec nightly and see if it still happens
in case i will contact CoreElec devs

EDIT: same error on last CE nightly, i ask support to CE dev

[CastagnaIT]

i asked to CE dev and seem there is a problem with the decoder that is forced limited to one instance in CE or something related to this, they will let us know if a possible solution could be done, they suspect/theorize a open/close stream overlapping in kodi not allowed in CE,
for now the only way to playback with CE is disable stream adaptive

[CastagnaIT]

just can you use same spaces of others