When packing apps for the Windows Store, Widevine is not correctly loaded

[delfinof]

When you package an app for the windows store the folder:

%USERPROFILE%\AppData\Roaming\<app name>

Is virtualized in

%USERPROFILE%\AppData\Local\Pacakges\<windows package name>LocalCache\Roaming\<app name>

The WidevineCDM directory is then created under the virtulized folder, and on some computer, this makes widevine to not initialize corretly: the event ready is received but it is impossible to use any DRM protected content (e.g. Spotify SDK).

Manually moving the WidevineCDM directory from the virtualized folder to the "real" folder solves the issue.
This didn't happened with previous versions of Widevine/Electron (we used 4.1.5 without any issue; the problem started moving to electron 7.X).

Is it possible to configure the path to use for downloading the Widevine files, so that the access to that is direct and not somehow virtualized?

[khwaaj]

First off, when you say ready event, are you talking about the widevine-ready event? If not then that is potentially your issue since the CDM initialization is not guaranteed to be completed before widevine-ready has been emitted, which means you could be looking at a race-condition causing erratic behaviour.

[delfinof]

First off, when you say ready event, are you talking about the widevine-ready event? If not then that is potentially your issue since the CDM initialization is not guaranteed to be completed before widevine-ready has been emitted, which means you could be looking at a race-condition causing erratic behaviour.

Yes, I am talking about the widevine-ready event. Apparently the inizialization (in terms of exchanged events) looks went fine.

[khwaaj]

Ok, then we are likely looking at some kind of issue with userData directory we get from Electron, which we use as the base directory for the CDM installation. It kind-of looks like it would be returning different values across different runs of the application. Can you reproduce this reliably? If so, could you log what app.getPath('userData') returns in both the ready and widevine-ready events to see that they appear to match each other, including between different runs?

As for a workaround, there are a couple of ways to override the directory values. The first way is to modify the wvconf.json file, residing in ELECTRON_APP/resources/ on Windows. There you can replace {{baseDir}} for downloadDir, installDir, updateDir and lastDir to use a custom value instead.

The second way is to use the verifyWidevineCdm() API to do the same thing programmatically. The downloadDir, installDir, updateDir and lastDir options are not officially documented, but they can be set the same way as session and disableUpdate in the example code, overriding the values from wvconf.json.

[delfinof]

The problem is that the path that Electron sees is always the not-virtualized one, even if it is writing in the virtualized directory.

The exact behaviour is the following:

If an app running in the desktop bridge tries to access any directory under
%USERPROFILE%\AppData\Roaming
and that directory exists, no virtualized directory is created.

If, otherwise, the directory does not exist yet, the app running in the desktop bridge thinks it is writing in the above directory, while in reality it is accessing files under

%USERPROFILE%\AppData\Local\Pacakges\<windows package name>LocalCache\Roaming\<app name>

I created a POC of this here:
https://github.com/delfinof/castlabs-electron-uwp
With that you can build a version of electron running in the destkop bridge (called bridged-electron ) and reproduce / debug the behaviour above.

If you install and run it on a machine where you have already run electron, the DRM will work since the directory:
%USERPROFILE%\AppData\Roaming\electron
already exists.

But if you delete that directory and run again:
bridged-electron https://bitmovin.com/demos/drm
You will see the DRM is not working, even if all the widevine events are received correctly.
And under:
%USERPROFILE%\AppData\Local\Packages\0e1d9634-0769-40f5-b896-e5361788481b_7wqrhnf42jqfy\LocalCache\Roaming
you will then see an Electron subdirectory with the Widevine files within.

Finally, if you even create the not-virtualized electron dir (using the not-desktop-bridged electron version), the bridged-electron will access its own virutalized version with its cookies/cache/and not working drm.

[khwaaj]

Well, that sounds like a recipe for disaster in combination with the Widevine CDM. It enforces that the CDM binary image was loaded from a "real" path with no filesystem links or virtualizations of any kind, silently failing if that is the case. This has bitten me before during development, e.g. when I've been using drive mappings to avoid issues with long paths on Windows. It is unclear to me why anything would differ between Electron 4 and 7 though since the code that is in question here is essentially the same. Maybe upstream Electron changed something so that the filesystem APIs now honors the virtualizations where it previously did not.

Anyway, thanks for the POC, I'll see if I can use that to further investigate what is happening. I suspect the only way around this is to make the CDM access bypass the filesystem virtualization, but I'm not very familiar with UWP or Desktop Bridge, so I'm not sure if it is possible. Were you able to make it work by overriding the paths used or is all filesystem access virtualized?

[koenoe]

I'm having the same issue as @delfinof.

Were you able to make it work by overriding the paths used or is all filesystem access virtualized?

I tried overriding the paths (with the non-virtualized directories) by passing them to app.verifyWidevineCdm, but still everything gets installed in C:\Users\koenr\AppData\Local\Packages\27241E05630EA_kn85bz84x7te4\LocalCache\Roaming\<APPNAME>

I think this happens because we can only read outside the virtualized folders, but not write.

[osmestad]

To reiterate what Koen is saying, this is hitting us quite hard now, so if there is anything we can do to help resolve this we will!

[khwaaj]

I've isolated the cause to what I was speculating about above. Since AppData is virtualized the Widevine CDM loads without problem but it will fail silently due to the security check I mentioned inside the CDM itself. If I install the CDM outside AppData instead, it works. For example setting something like this when calling verifyWidevineCdm() works fine:

  var wv = app.verifyWidevineCdm({
    downloadDir: 'C:\\Users\\khwaaj\\foobar\\Electron\\Downloads\\WidevineCDM',
    installDir: 'C:\\Users\\khwaaj\\foobar\\Electron\\WidevineCDM',
    updateDir: 'C:\\Users\\khwaaj\\foobar\\Electron\\WidevineCDM.pending',
    lastDir: 'C:\\Users\\khwaaj\\foobar\\Electron\\WidevineCDM.last',
  });

Apparently the virtualization of AppData was a change introduced in Windows 10 1903, so it was likely working before that. I'm trying to figure out what to best do about this and what recommendation to make to people running into this now.

[koenoe]

For example setting something like this when calling verifyWidevineCdm() works fine:

  var wv = app.verifyWidevineCdm({
    downloadDir: 'C:\\Users\\khwaaj\\foobar\\Electron\\Downloads\\WidevineCDM',
    installDir: 'C:\\Users\\khwaaj\\foobar\\Electron\\WidevineCDM',
    updateDir: 'C:\\Users\\khwaaj\\foobar\\Electron\\WidevineCDM.pending',
    lastDir: 'C:\\Users\\khwaaj\\foobar\\Electron\\WidevineCDM.last',
  });

This is what I did with but then with: C:\\Users\\koenr\\AppData\\Roaming\\Electron\\WidevineCDM etc. and didn't work for me. Except when I've already ran Electron, like @delfinof mentioned. With a clean install from the Windows store these folders are not created.

As far as I know you also can't write outside the virtualized directories without permissions (when packed for Windows Store). Or am I missing something?

[khwaaj]

That is just the thing, it won't work inside AppData since the whole thing appears to be virtualized and the CDM can't run from a virtualized location. I'm not sure if there is some other appropriate location to write the data instead, but using a completely separate directory, like I do above, does work when I run locally (using the POC provided by @delfinof).

If anyone has any good suggestions for an alternate location that is not virtualized please go ahead and make them... :-)

[koenoe]

As it probably has to be in the user folder due to permissions, I think it would be best to do something like: C:\\Users\\koenr\\<APPNAME>\\WidevineCDM. Am I correct in thinking that when the user would remove this folder it gets created again on startup?

[khwaaj]

Yeah, it would have to be a writable directory like AppData or ProgramData, but both of those appear to be virtualized, so something like %USERPROFILE%\Widevine\<APP-NAME>\ as a base path may be an acceptable option (having the extra Widevine to avoid polluting the user directory as much if several apps are installed). The directory tree will be created if it does not exist.

Apparently there is another accepted pattern as well where template files are installed in AppData by the AppX installer instead, making sure the required directories are not virtualized. I suppose it can ba a bit fragile though if some user tries to reset the app by removing the AppData contents for it.

[delfinof]

With the path override, I guess it should be possible to distribute widevine with the app and load it from the installation folder.

Is there any legal or technical issue (e.g. using relative paths in loading widevine, too tight release schedule etc) preventing that?

[khwaaj]

While technically possible there are legal issues with providing the CDM with the application bundle, like Chrome does. We have explicitly been warned not to do this by the Widevine Team since it would open us up to potential litigation, specifically over decoder licensing (since there are decoders included in the CDM). As long as the CDM is dynamically installed by download from Google servers it is covered by the license Google has.

There is also the drawback of not being able to update it in case a CDM is deprecated for security reasons, unless you install the CDM in a directory where it can be updated, of course. An alternative may be to run the application outside the Desktop Bridge with a special --install parameter during the installation, waiting for the widevine-ready event and then simply exiting. Not sure if that is possible to do in the AppX installer though.

[khwaaj]

Starting with v5.0.13 it will be possible to override baseDir immediately instead of separately overriding downloadDir, installDir, updateDir and lastDir. It will also be possible to do this using the widevine-base-dir command line switch. This will roll out to v6 and v7 as well in the next release of each release series and should be the preferred way to move the Widevine CDM out of AppData in cases like this.

For example, to use the equivalent of %USERPROFILE%\Widevine\<APP-NAME> as the base path you could do:

app.verifyWidevineCdm({
  baseDir: path.join(app.getPath('home'), 'Widevine', app.getName())
});

Or:

app.commandLine.appendSwitch('widevine-base-dir', path.join(app.getPath('home'), 'Widevine', app.getName()))

[delfinof]

Hi, this new option looks interesting.

In this branch of my POC:
https://github.com/delfinof/castlabs-electron-uwp/tree/5-0-x

I changed the "c# starter" in order to read the LocalFolder of the app without requiring the system virtualization through the UWP API:
ApplicationData.Current.LocalFolder.Path

Which returns:
C:\Users\<username>\AppData\Local\Packages\0e1d9634-0769-40f5-b896-e5361788481b_7wqrhnf42jqfy\LocalState

Then I start electron with the command line switch you explained above.
Everything seems to work and does not violate MS policies now...

[koenoe]

@delfinof That looks awesome! How would I get this ApplicationData.Current.LocalFolder.Path by using Javascript? Does Electron have an API for it?

[khwaaj]

Interesting, I did some similar experiments by hardcoding the paths in Electron v7 but did not get it to work reliably even then. I think that may have still been in the Roaming subdirectory though so perhaps that has some implications. Or maybe the Windows SDK linked into Electron influences this somehow and v5 uses an earlier version?

In any case, exactly how this "virtualization" they are doing since the 1903 update works is still a bit unclear to me. Perhaps there are some loopholes that can be used, but I'm a bit worried about what happens in the next OS update.

I'm just about to release v7.1.6, including the fix I mentioned earlier, so it would be an interesting test to see if this workaround is still viable there.

[delfinof]

@koenoe In case you wish to do everything from within electron, this is a way for accessing the above API:
https://github.com/NodeRT/NodeRT

@khwaaj will test that when available.
On 1903 the virtualization works roughly in this way: if an application module is started from within a virtualized folder, it will see a virtualized world where some dirs like the home folder / documents folder etc have a virtualized version that is in the exact same position of the not virtualized one.
So if you can copy an .exe there, the starting program will see still see a virtualized world while the started one will see the not virtualized world.
Virtualization is mostly used for avoiding messing up end-user systems with application.

(BTW this trick does not work in windows mode S (or how is it called now) and violates Windows Store policies)

[khwaaj]

@delfinof, thanks, I was thinking that you initially mentioned that it worked in v4 but not v7 so that is why I'm interested to see if there is a difference between v5 and v7 too. I've not been able to find any obvious reasons for this discrepancy which is why I mentioned the possibility of differences depending on which Windows SDK version was linked.

Indeed, that is about what I gathered from reading up on the Desktop Bridge in 1903. But the devil is in the details and the CDM obviously detects that it has been placed in a virtualized environment, even for some placements that have not been explicitly documented as virtualized in what I've read. For example, in my testing this was true even if I passed the "real" path to to verifyWidevineCdm(), unlike what your test with v5.0.13 seems to show, which would mean that those folders were detected as virtualized by the CDM too in that particular case (although, like I mentioned, I was not using the exact same path as you were). Since we are not privy to the implementation of either the Desktop Bridge or the CDM, inferring what will work is just an educated guessing game, hopefully we'll get it right... :-)

About the Windows Store, isn't just downloading and executing a binary not part of the initial package a breach off the store policies? I know it is in the Mac App store.

[delfinof]

I think dynamic downloading of WidevineCDM is allowed in the Windows Store because of this:

https://docs.microsoft.com/en-us/windows/uwp/publish/store-policies#102-security

10.2.2
Your product must not attempt to change or extend its described functionality through any form of dynamic inclusion of code that is in violation of Store Policies. Your product should not, for example, download a remote script and subsequently execute that script in a manner that is not consistent with the described functionality.

That is, as long as you don't include code which changes described functionalities, you are ok.

On the other side, IMHO downloading Widevine in the home directory violates the principle at the beginning:

Don’t attempt to cheat customers, the system or the ecosystem.

Since it tries to circumvent the ecosystem concept of virtualization.

But my personal experience suggests that MS Store reviewers are usually open to exceptions if these are required to assure the app being able to work correctly

[khwaaj]

FYI, another workaround for this issue is to run electron without the chromium sandbox, e.g. by passing --no-sandbox at startup. It does have some security implications though, particularly if you are loading untrusted content.

[khwaaj]

I added a FAQ entry relating our stance on this after discussing it with Chromium devs.

I'l go ahead and close the issue, feel free to reopen if there are any outstanding questions or topics pertaining to this.

[toschlog]

@khwaaj I see that the FAQ says the the only solution for packaging ESC for the Windows Store is to use the -no-sandbox option.

But what about redefining widevine-base-dir as suggested above? That sounds like a fine solution. Does it not work?

[khwaaj]

While it used to be technically possible to adjust where the CDM is installed to avoid this issue there are a number of reasons this is probably not a good solution for production grade software, and we are recommending against it. To name a few:

1. The later releases, v16+, uses the Component Updater to install the CDM and no longer supports the widevine-base-dir option
2. Selecting a reliable place outside of AppData to install the CDM is difficult, if not impossible, across different types of Windows installations and user types
3. There are unclarities if this might be in violation of Windows Store policies