See if it's possible to use wimlib instead of native WIM API #453

Closed
pbatard opened this Issue Mar 1, 2015 · 7 comments

Comments

Projects
None yet
3 participants
@pbatard
Owner

pbatard commented Mar 1, 2015

It looks like there exists a GPLv3 WIM library (wimlib) that may possibly allow us to apply the install.wim image for ToGo, without having to first mount the ISO. This could possibly solve the Windows To Go compatibility issues and speed up extraction. Will need to investigate.

@pbatard pbatard self-assigned this Mar 1, 2015

@pbatard pbatard added this to the 2.1 milestone Mar 1, 2015

@MagicAndre1981

This comment has been minimized.

Show comment
Hide comment
@MagicAndre1981

MagicAndre1981 Mar 5, 2015

this may also solve my issue. I can't create a 8.1 WinToGo on a 8.0 host. DISM crashes here because the 8.0 DISM can't handle 8.1 WIMs.

this may also solve my issue. I can't create a 8.1 WinToGo on a 8.0 host. DISM crashes here because the 8.0 DISM can't handle 8.1 WIMs.

@pbatard

This comment has been minimized.

Show comment
Hide comment
@pbatard

pbatard Mar 5, 2015

Owner

I'm gonna add a link, pointed by @MagicAndre1981, to the article that details how one can get a 6.3 DISM and extract it:
http://www.msfn.org/board/topic/156869-get-waik-tools-wo-downloading-the-huge-isos/

The important part here is:

curl -N -r 129740800-136032256 http://download.microsoft.com/download/8/E/9/8E9BBC64-E6F8-457C-9B8D-F6C9A16E6D6A/KB3AIK_EN.iso -o"Neutral.cab"

This will download the cab archive containing dism and the library, which can then be extracted.

My only issue, besides the time required to implement something like that, is that Rufus would need to detect the Windows version from the ISO, which isn't as easy as it seems, and maintain a list of revelant online downloads of where to to pick dism and the library from. Also, the day Microsoft decide to revamp their download section, which has happened many times in the past, any links hardcoded in the app will break, so I'd have to add to maintain and query the current URLs from the Rufus server...

Owner

pbatard commented Mar 5, 2015

I'm gonna add a link, pointed by @MagicAndre1981, to the article that details how one can get a 6.3 DISM and extract it:
http://www.msfn.org/board/topic/156869-get-waik-tools-wo-downloading-the-huge-isos/

The important part here is:

curl -N -r 129740800-136032256 http://download.microsoft.com/download/8/E/9/8E9BBC64-E6F8-457C-9B8D-F6C9A16E6D6A/KB3AIK_EN.iso -o"Neutral.cab"

This will download the cab archive containing dism and the library, which can then be extracted.

My only issue, besides the time required to implement something like that, is that Rufus would need to detect the Windows version from the ISO, which isn't as easy as it seems, and maintain a list of revelant online downloads of where to to pick dism and the library from. Also, the day Microsoft decide to revamp their download section, which has happened many times in the past, any links hardcoded in the app will break, so I'd have to add to maintain and query the current URLs from the Rufus server...

@Fuj-

This comment has been minimized.

Show comment
Hide comment
@Fuj-

Fuj- Mar 6, 2015

I had an idea for an NSIS script that would unpack the iso, split install.wim into .swm and repack so it can be used with Rufus. I had everything figured out but decided not to bother since you'd announced UEFI on NTFS support. But I'd been playing with some code and the way I detected OS was by extracting bootsect.exe and reading its FILEVERSION. It corresponds to OS version.

Fuj- commented Mar 6, 2015

I had an idea for an NSIS script that would unpack the iso, split install.wim into .swm and repack so it can be used with Rufus. I had everything figured out but decided not to bother since you'd announced UEFI on NTFS support. But I'd been playing with some code and the way I detected OS was by extracting bootsect.exe and reading its FILEVERSION. It corresponds to OS version.

@MagicAndre1981

This comment has been minimized.

Show comment
Hide comment
@MagicAndre1981

MagicAndre1981 Mar 6, 2015

You can simply download the latest version of DISM with the tool from msfn. The 6.3 should work with 6.2 WIMs. Or have you tried WIMlib? Does it work better compared to DISM?

You can simply download the latest version of DISM with the tool from msfn. The 6.3 should work with 6.2 WIMs. Or have you tried WIMlib? Does it work better compared to DISM?

@pbatard

This comment has been minimized.

Show comment
Hide comment
@pbatard

pbatard Mar 6, 2015

Owner

If I had tried it, you would see an update with my findings here.

One thing you may not realize is that Rufus does not use dism for the Apply-Image part for various reasons (one of which being the that getting a progress report from dism that could be translated to an UI progress would be a major PITA, and dism is mostly a wrapper for the WIM API anyway).

dism is only invoked to apply san_policy.xml, and the failure of this step is actually non fatal, as it just means that internal disks will be accessible when running Windows To Go, instead of hidden, as Microsoft indicates they should be. As a matter of fact, I probably could have used a call to the WIM API instead of invoking dism directly for this part, as I do for Apply-Image, but it was just more convenient to call on dism here since I didn't need progress.

So the part I am really interested in is an updated WIM API library, but what I remember from briefly testing with the Windows 10 version is that you need more that just the WIM library itself, and I got dependency issues preventing Windows 10 dism, and the latest WIM API library, to run on Windows 8.1 (which is probably my mistake, as you most likely want the one Microsoft provides from the AIK instead of trying to pick stuff at random from a Windows 10 installation).

So my concern is "How much extra stuff would Rufus need to download, just to be able to get the WIM library to work with a recent Windows image. On the other hand, if wimlib works just fine, I could avoid all the extra downloading, which, as with any external download, I also have to explain the purpose of to my users, with a popup that needs to be localized in 30 languages"...

If wimlib is updated on regular basis and compatible enough with Windows 10 for Rufus' use, and also if it's faster on Apply-Image than the WIM API (which is very, very slow), I'd rather use that than add extra complexity with trying to herd a new bunch of external download.

Anyway, I will find out more once I start testing wimlib... which may be in a week, a month, or whenever nothing more important takes precedence...

Owner

pbatard commented Mar 6, 2015

If I had tried it, you would see an update with my findings here.

One thing you may not realize is that Rufus does not use dism for the Apply-Image part for various reasons (one of which being the that getting a progress report from dism that could be translated to an UI progress would be a major PITA, and dism is mostly a wrapper for the WIM API anyway).

dism is only invoked to apply san_policy.xml, and the failure of this step is actually non fatal, as it just means that internal disks will be accessible when running Windows To Go, instead of hidden, as Microsoft indicates they should be. As a matter of fact, I probably could have used a call to the WIM API instead of invoking dism directly for this part, as I do for Apply-Image, but it was just more convenient to call on dism here since I didn't need progress.

So the part I am really interested in is an updated WIM API library, but what I remember from briefly testing with the Windows 10 version is that you need more that just the WIM library itself, and I got dependency issues preventing Windows 10 dism, and the latest WIM API library, to run on Windows 8.1 (which is probably my mistake, as you most likely want the one Microsoft provides from the AIK instead of trying to pick stuff at random from a Windows 10 installation).

So my concern is "How much extra stuff would Rufus need to download, just to be able to get the WIM library to work with a recent Windows image. On the other hand, if wimlib works just fine, I could avoid all the extra downloading, which, as with any external download, I also have to explain the purpose of to my users, with a popup that needs to be localized in 30 languages"...

If wimlib is updated on regular basis and compatible enough with Windows 10 for Rufus' use, and also if it's faster on Apply-Image than the WIM API (which is very, very slow), I'd rather use that than add extra complexity with trying to herd a new bunch of external download.

Anyway, I will find out more once I start testing wimlib... which may be in a week, a month, or whenever nothing more important takes precedence...

@pbatard pbatard changed the title from ER: See if it's possible to use wimlib instead of native WIM API to See if it's possible to use wimlib instead of native WIM API May 8, 2015

@pbatard pbatard added the enhancement label May 8, 2015

@pbatard pbatard modified the milestones: 2.3, 2.2 May 8, 2015

@pbatard

This comment has been minimized.

Show comment
Hide comment
@pbatard

pbatard Jul 7, 2015

Owner

Deferring this one. The improvements applied for Windows to Go in 2.3 should be enough to ensure Windows 10 To Go support, and the lack of a bcdboot.exe with UEFI support on Windows 7 makes it problematic to add To Go support to, even if we sort the image mounting issue.

Owner

pbatard commented Jul 7, 2015

Deferring this one. The improvements applied for Windows to Go in 2.3 should be enough to ensure Windows 10 To Go support, and the lack of a bcdboot.exe with UEFI support on Windows 7 makes it problematic to add To Go support to, even if we sort the image mounting issue.

@pbatard pbatard removed this from the 2.3 milestone Jul 7, 2015

@pbatard pbatard added the deferred label Jul 7, 2015

@pbatard

This comment has been minimized.

Show comment
Hide comment
@pbatard

pbatard Aug 28, 2015

Owner

Closing this issue for now.

Owner

pbatard commented Aug 28, 2015

Closing this issue for now.

@pbatard pbatard closed this Aug 28, 2015

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment