Signing cert expired?

[shanselman]

Has the driver signing cert expired? I'm unable to install and per these verbose logs this smells like a cert issue.

DIFXAPP: ERROR: encountered while installing driver package 'C:\Program Files\usbipd-win\Drivers\VBoxUSB\VBoxUSB.inf' DIFXAPP: ERROR: InstallDriverPackages failed with error 0xE0000247 DIFXAPP: RETURN: InstallDriverPackages() 3758096967 (0xE0000247) CustomAction MsiInstallDrivers returned actual error code 1603

[dorssel]

I'll look into that later tonight (will be at home in about 2 hours). I assumed the driver package has a timestamp, which would then only require the certificate to be valid at the time of signing. That's how I sign the msi.

If not, I'll update to the latest VirtualBox driver. The driver code has not changed since, but Oracle does increment the driver version and resigns the package with every release.

[benmcmorran]

For reference, I just tried to repro on a freshly installed Windows 11 retail machine and was able to install usbipd-win successfully.

• usbipd-win: 1.1.0
• Windows: 10.0.22000.194 (released October 4, 2021)

[benmcmorran]

@dorssel I've been chatting with @shanselman offline and if it's easy enough for you to create a new MSI with updated Oracle drivers, that's probably the easiest way for us to narrow down the cause of this issue. Let me know if I can help.

[dorssel]

I doubt that will solve the problem. I just checked: a) the oracle cert is still valid and b) the driver has a timestamp (so it should still work after 2022, when the cert will expire). This is not the problem.

Looking into 0xE0000247: ERROR_DRIVER_STORE_ADD_FAILED.
It is mentioned by VirtualBox here: https://www.virtualbox.org/svn/vbox/trunk/src/VBox/Additions/WINNT/Installer/VBoxDrvInst.cpp

Guess 1 (most likely)

It looks like the root certificate is not trusted. The CA of the oracle cert is: DigiCert Assured ID Root CA. And what do you know: it still is a SHA-1 based root, with a SHA-1 intermediate, and a SHA-1 final cert. I guess they didn't get the memo...
I will find out if they have updated to SHA-256 in the latest VirtualBox release...

Guess 2

It may also be my workaround for silent install (to support winget).
@shanselman Did you install using winget? Or maybe msiexec with one of the silent install options like /qb? Could you try an interactive install with the MSI, or use winget install --interactive? In that case you should get a popup asking whether you trust the oracle signature (unless you ever selected 'always trust').

Guess 3

I also found: https://stackoverflow.com/questions/69620174/driver-install-error-on-windows-11-home-insider-preview-x86-64
@shanselman Did you try to install on a very recent Windows 11 Insider?
In combination with guess 1: it makes sense that SHA-1 is now disallowed completely (in Win 10 SHA-1 is still allowed for driver packages...)

[dorssel]

Update on Guess 1:
The newest VirtualBox release has a driver catalog file that is also signed with a Windows 10 Microsoft attestation signature. The driver we are bunding now only has that on the SYS file, not the catalog.

Update on Guess 2:
I inadvertently register the SHA-1 certificate for silent installs, but the drivers are (correctly) dual-signed with both SHA-1 and SHA-256 signatures.

Update on Guess 3:
I can confirm that the latest Windows 11 Insider no longer allows any install of usbipd-win 1.1.0, silent or interactive.
From: https://www.virtualbox.org/wiki/Changelog

VirtualBox 6.1.28 (released October 19 2021)
Windows Guest: Introduced Windows 11 guest support, including unattended installation

Note that this is talking about guest not host (which is what we do), but it is an indication...

Conclusion: I will update to the latest VirtualBox driver, and change silent install to use the SHA-256 cert. Then I will check if the latest Windows 11 Insider accepts it.

[dorssel]

The update of the driver did not help on Windows 11 Insider. Digging into it, installing the (newest 6.1.28) driver by hand, gave me:

PS> pnputil /add-driver VBoxUSBMon.inf /install
Microsoft PnP Utility

Adding driver package:  VBoxUSBMon.inf
Failed to add driver package: The certificate is not valid for the requested usage.

Total driver packages:  1
Added driver packages:  0

Whereas on my regular Windows 11 it gives:

PS> pnputil /add-driver VBoxUSBMon.inf /install
Microsoft PnP Utility

Adding driver package:  VBoxUSBMon.inf
Driver package added successfully.
Published Name:         oem0.inf

Total driver packages:  1
Added driver packages:  1

Something has changed in Windows 11 Insider such that it no longer accepts it. To my knowledge (and I know quite a bit about driver signing), the oracle drivers are signed correctly. Maybe Microsoft changed the policy such that the signing certificate itself must be EV and not only the Hardware Portal submission certificate? The driver does have a MS attestation signature... Or maybe a different attestation signature (e.g. specifically for Windows 11 Insider) is required?

I don't think I can fix this...

@shanselman Can you confirm that you were installing on Windows 11 Insider?

[benmcmorran]

Thanks so much for looking into this @dorssel. I'll follow up internally to see if I can get more details on driver signing requirements in Windows 11.

[shanselman]

Yes I'm on insiders. I did this to get the logs:

msiexec /i usbipd-win_1.1.0.msi /l*v upbipd-scottha.txt

I will try interactive with winget

[dorssel]

I have updated the driver and certificate, but it does not work for me on Windows 11 Insider.
Artifact at: https://github.com/dorssel/usbipd-win/pull/48/checks?check_run_id=3957665556

@shanselman You could give this a try, but I don't think it will help.

[dorssel]

I have just updated my Windows 11 Insider to the latest Dev channel: 22483.1000
And ... it works again. I could just do winget install -e dorssel.usbipd-win (i.e. the current version 1.1.0).

My guess: a regression bug in Windows 11 Insider that failed on dual-signed catalog files, which is now fixed.

@shanselman @benmcmorran I think we can close this as no longer reproducible.

[shanselman]

Whew! Sorry for the fire drill and thanks for helping everyone! -- Scott Hanselman @shanselman <http://twitter.com/shanselman> - Podcast: https://hanselminutes.com/shows

…

On Wed, Oct 20, 2021 at 4:54 PM Frans van Dorsselaer < ***@***.***> wrote: I have just updated my Windows 11 Insider to the latest Dev channel: 22483.1000 And ... it works again. I could just do winget install -e dorssel.usbipd-win (i.e. the current version 1.1.0). My guess: a regression bug in Windows 11 Insider that failed on dual-signed catalog files, which is now fixed. @shanselman <https://github.com/shanselman> @benmcmorran <https://github.com/benmcmorran> I think we can close this as no longer reproducible. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#47 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAAWTFVQUIH2B6YXUBPDX3UH5JEHANCNFSM5GMF5R4A> .

[dorssel]

And it broke again on the latest Windows 11 Insider Dev Channel...

The root cause has been found.

1. Oracle submits their driver package to MS Hardware Portal
2. Microsoft adds their attestation signature to the .sys driver (which now has 3 signatures, which is OK).
3. Microsoft replaces the catalog file (because the underlying .sys file has changed; the resulting catalog is now signed only by MS. This is fine for Win8+, but won't work for pre-Win8. MS has no problems there, since pre-Win8 is EOL.
4. Oracle tries to use the old "dual-signed" catalog trick to support installation on pre-Win8 OSs, which do not know about the MS attestation signatures. With this trick, a driver can work on both Windows 10 as well as older OSs. This is a common trick I have also used in the past. However ... Oracle messed this up! signtool.exe only supports adding 1 additional signature to catalog files, but they instead added 2: the deprecated SHA-1 (apparently to still support XP?) and their SHA-256 (probably to still support Vista).

By adding two additional signature, the catalog actually becomes corrupted. The first (primary) signature is now the deprecated SHA-1 (the signature itself is correct), the second is the Oracle SHA-256 (this signature is actually corrupt). And the third is the (original) MS attestation signature (which fortunately is still correct).

Now, Win 10 and Win 11 accept this corrupt catalog, because they accept the primary (SHA-1) signature, which is still within the grace period for drivers. It does however require that users accept the 3rd party vendor certificate. This is why usbipd-win required to add Oracle's (SHA-1) certificate to the trusted publishers list for silent installs (winget).

However, Win 11 Insider no longer accepts any SHA-1. In fact it requires the MS attestation signature. But since that comes after the broken Oracle SHA-256 signature, it fails completely.

The solution I got working now is: I reverted the catalog to the original MS one by removing the additional signatures from Oracle. This actually reconstructs the original catalog that Microsoft Hardware Portal generated (but Oracle does not distribute that). This is signed only by MS, which is accepted by default on all Win8+ OSs. No more need for silent install workarounds, and it works on Win 11 Insider too.

I've seen a lot of Authenticode f* ups, but this was really a nasty one to track down...

Solution is in #48.

[klaus-vb]

Can you try again with the re-packaged https://download.virtualbox.org/virtualbox/6.1.34/VirtualBox-6.1.34a-150636-Win.exe ?

This should have the driver signing issues fixed (the more annoying issue was actually an installer bug, resulting in installing the "pre-W10" cat file on Windows 10 and later). It contains the same build (all files are old, having signatures from mid March) as the original release.

It really was a nasty issue (especially as Explorer happily showed the signatures, and IIRC even signtool thought they're good). We fixed it bit by bit since your analysis, but there was always something else which didn't go as planned, rendering the fix as ineffective.

[dorssel]

@klaus-vb
I had already fixed that. see https://github.com/dorssel/usbipd-win/releases/tag/v1.1.1.
Since then, usbipd-win already ships with the original Windows signed .cat file, instead of the re-signed (or rather, faulty doubly-signed) VBox cat file. As a consequence, we are already using what you call 6.1.34**a** . This issue is really closed.

[klaus-vb]

Yes, I know, but the point is that with 6.1.34a and later you shouldn't need to do .cat file surgery any more. I haven't tried usbipd-win myself so far, but I can see that all "driver level" functionality it needs are in the drivers we developed. It's nice to see that reusing our code is possible.

[dorssel]

OK, I've verified that the drivers in VirtualBox-6.1.34a are indeed byte-for-byte identical to what we already have in usbipd-win. This means that the .cat files we both use are now the same, original, Microsoft-signed .cat files. Good to know that I don't have to do the ASN.1 reconstructing again in the future!