We should be able to load pyshellext.dll with Python and test it without having to install it properly.
This will involve adding some entry points, and probably mocking up test data in the "real" locations (or maybe there's a clever point we can inject a ctypes call and provide a different registry key to search?).
In any case, relatively low priority, as it works right now, and the main concern is to do with OS integration that can't be tested in CI.