Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How to attach other vhdx files to WSL2? #4574

Closed
19317362 opened this issue Oct 5, 2019 · 14 comments
Closed

How to attach other vhdx files to WSL2? #4574

19317362 opened this issue Oct 5, 2019 · 14 comments

Comments

@19317362
Copy link

19317362 commented Oct 5, 2019

WSL2 looks very good for developing Linux realted project (embeded linux & android etc).

But when we do our dev works, we always put code in another different disk from the sys(/).

How to achieve this?

@ajw107
Copy link

ajw107 commented Oct 5, 2019

To mount an extra vhdx file (and read ext4,etc filesystems from within Windows 10:

#Install afflib-tools
sudo apt install afflib-tools

#set some commonly used values:
vhdxFile="/mnt/c/mydisk.vdhx" #Path to VHDX file
mountPoint="/mnt/mydirectory" #Your desired mount point
rawFilePath="${mountPoint}/$(basename "${vhdxFile}".raw)" #do not change this

#Mount the vhdx file
mkdir -p "${mountPoint}"
sudo affuse  "${vhdxFile}" "${mountPoint}"

#Find out the offset where the partition is:
#Example sudo fdisk -l "${rawFilePath}" Output:
#Disk /mnt/mydirectory/mydisk.vmdk.raw: 150 GiB, 161061273600 bytes, 314572800 sectors
#Units: sectors of 1 * 512 = 512 bytes
#Sector size (logical/physical): 512 bytes / 512 bytes
#I/O size (minimum/optimal): 512 bytes / 512 bytes
#Disklabel type: dos
#Disk identifier: 0xe3973aa1

#Device                            Boot Start       End   Sectors  Size Id Type
#/mnt/mydirectory/mydisk.vmdk.raw1 *     2048 164208639 164206592 78.3G 83 Linux

#Multiply the sector size by the partiton Start Value
#In this case 512 from the logical portion the Sector size (3rd line) in the output above
sectorSize=$(sudo fdisk -l "${rawFilePath}" | grep "Sector size" | awk '{print $4}')
#In this case 2048 from the Start number in the final line of the output above
startSector=$(sudo fdisk -l "${rawFilePath}" | tail -n1 | grep -o -E '[0-9]{2,}' | head -n1)
offset=$((${startSector}*${sectorSize}))  #In this case is 1048576

#Now mount the partition you want using the offset (this will replace the original mount which was the full raw disk file)
#Change ro to rw for read/write access instead of read-only access
sudo mount -o ro,loop,offset=${offset} "${rawFilePath}" "${mountPoint}"

#Done
echo "VHDx Disk [${vhdxFile}] mounted at [${mountPoint}] is now accessible in windows from [\\\\wsl$\Ubuntu${mountPoint////\\}], this includes disks with EXTx filesystems"

@zot
Copy link

zot commented Oct 6, 2019

Thanks, this is a potential stop-gap but I believe it mounts over 9p, which I don't want to do -- see #4556 (comment) for more details.

@therealkenc
Copy link
Collaborator

therealkenc commented Oct 6, 2019

There is a line in the 18970 release notes about allowing cross-distro mounts. I haven't played with it yet personally, but that appears to be intended for your use case. Duping because mounting an arbitrary vhd is the same ask, regardless of the use.

@19317362
Copy link
Author

19317362 commented Oct 7, 2019

There is a line in the 18970 release notes about allowing cross-distro mounts. I haven't played with it yet personally, but that appears to be intended for your use case. Duping because mounting an arbitrary vhd is the same ask, regardless of the use.

Thanks for your reply!
What I want is just like mounting a phisical disk to a phisical machine. I don't know how to do a cross-distro mount with a blank VHDX.

@19317362
Copy link
Author

19317362 commented Oct 7, 2019

To mount an extra vhdx file (and read ext4,etc filesystems from within Windows 10:

#Install afflib-tools
sudo apt install afflib-tools

#set some commonly used values:
vhdxFile="/mnt/c/mydisk.vdhx" #Path to VHDX file
mountPoint="/mnt/mydirectory" #Your desired mount point
rawFilePath="${mountPoint}/$(basename "${vhdxFile}".raw)" #do not change this

#Mount the vhdx file
mkdir -p "${mountPoint}"
sudo affuse  "${vhdxFile}" "${mountPoint}"

#Find out the offset where the partition is:
#Example sudo fdisk -l "${rawFilePath}" Output:
#Disk /mnt/mydirectory/mydisk.vmdk.raw: 150 GiB, 161061273600 bytes, 314572800 sectors
#Units: sectors of 1 * 512 = 512 bytes
#Sector size (logical/physical): 512 bytes / 512 bytes
#I/O size (minimum/optimal): 512 bytes / 512 bytes
#Disklabel type: dos
#Disk identifier: 0xe3973aa1

#Device                            Boot Start       End   Sectors  Size Id Type
#/mnt/mydirectory/mydisk.vmdk.raw1 *     2048 164208639 164206592 78.3G 83 Linux

#Multiply the sector size by the partiton Start Value
#In this case 512 from the logical portion the Sector size (3rd line) in the output above
sectorSize=$(sudo fdisk -l "${rawFilePath}" | grep "Sector size" | awk '{print $4}')
#In this case 2048 from the Start number in the final line of the output above
startSector=$(sudo fdisk -l "${rawFilePath}" | tail -n1 | grep -o -E '[0-9]{2,}' | head -n1)
offset=$((${startSector}*${sectorSize}))  #In this case is 1048576

#Now mount the partition you want using the offset (this will replace the original mount which was the full raw disk file)
#Change ro to rw for read/write access instead of read-only access
sudo mount -o ro,loop,offset=${offset} "${rawFilePath}" "${mountPoint}"

#Done
echo "VHDx Disk [${vhdxFile}] mounted at [${mountPoint}] is now accessible in windows from [\\\\wsl$\Ubuntu${mountPoint////\\}], this includes disks with EXTx filesystems"

Thanks for you reply.

This is not what I want. I want to use WSL2 VM like a real VM can mount multple VHDX via /etc/fstab etc.

@therealkenc
Copy link
Collaborator

therealkenc commented Oct 7, 2019

I want to use WSL2 VM like a real VM can mount multple VHDX via /etc/fstab etc.

It is likely until there is some official word over in #4556 you want a real VM, because it is the right tool for the job. I believe one the goals of WSL (both 1 and 2) is not to have to futz with VHDs and configuration thereof. The disk image is an implementation detail, just like the hidden don't touch lxfs was in WSL1. Everyone is welcome to off-road it to greater or lesser success, of course.

A lot here is going to depend on your specific use-case and the motivation behind wanting a VHDX, specifically, what you intend to do with said VHDX on the Windows side. If what you are just looking for a separate filesystem, you can create an ext4 image over loopback (rando google hit guide here no warranties).

@19317362
Copy link
Author

19317362 commented Oct 11, 2019

I want to use WSL2 VM like a real VM can mount multple VHDX via /etc/fstab etc.

It is likely until there is some official word over in #4556 you want a real VM, because it is the right tool for the job. I believe one the goals of WSL (both 1 and 2) is not to have to futz with VHDs and configuration thereof. The disk image is an implementation detail, just like the hidden don't touch lxfs was in WSL1. Everyone is welcome to off-road it to greater or lesser success, of course.

A lot here is going to depend on your specific use-case and the motivation behind wanting a VHDX, specifically, what you intend to do with said VHDX on the Windows side. If what you are just looking for a separate filesystem, you can create an ext4 image over loopback (rando google hit guide here no warranties).

This thought is just for safe: keep thing isolated from system disk. Just like in Windows, I never save work files in C:. Because things like re-install or system crash maybe fatal.

@therealkenc
Copy link
Collaborator

therealkenc commented Oct 11, 2019

Just like in Windows, I never save work files in C:

Okay that's the nuance I missed. It isn't that you want it on a VHD, somuch as you don't want it on your physical system disk (or at least, not the system partition thereof). And you sure don't want to go through a 9p bottleneck to do that. Okay, that's not a crazy ask. Subscribe #4556.

@T-Nalve
Copy link

T-Nalve commented Oct 17, 2019

Hello I have issue, when i use : sudo affuse "${vhdxFile}" "${mountPoint}"
I have : "Can't open image file: Invalid argument"
i have check variable and is good,
i have try with -o or -f option like this :
image

but same issue, please help me !

Best Regards

@zot
Copy link

zot commented Nov 5, 2019

OK, here's a hackaround for this that does seem to provide speedy access, if you don't mind having 1/2G of extra stuff in your VHDX.

You can load in a lightweight WSL 2 distro, like Alpine and then put your files in there-- let's call it light (there are smaller Linux distros out there but Alpine's in the Microsoft Store) . Once light is running, you can run any other WSL 2 distro and access your files in /mnt/wsl/light. If you don't mind the extra files (i.e. the Alpine linux distro, or whatever distro you choose), this seems to be a handy (if hacky) way to attach an external VHDX to your WSL 2 session.

One way to make sure light always runs is to start your WSL session from inside a light session, like this:

wsl -d light exec wsl.exe

or even

wsl -d light exec wsl.exe ps fax

Even though it's done with an exec, the "light context" will stay around until you exit the shell or command you invoked. The exec just removes a bit of overhead from the light session. As long as you have at least one light session running, you don't need any more. Tmux or EMACS might make good command choices for this.

Another way to "keep the light on" is to use the latest Docker Desktop with the "WSL 2 backend" and activate docker in light. This will run the docker proxy in light keep it running (that appears to be the current behavior of Docker Desktop 2.1.5.0 anyway).

@ad-on-is
Copy link

ad-on-is commented Dec 13, 2019

OK, here's a hackaround for this that does seem to provide speedy access, if you don't mind having 1/2G of extra stuff in your VHDX.

You can load in a lightweight WSL 2 distro, like Alpine and then put your files in there-- let's call it light (there are smaller Linux distros out there but Alpine's in the Microsoft Store) . Once light is running, you can run any other WSL 2 distro and access your files in /mnt/wsl/light. If you don't mind the extra files (i.e. the Alpine linux distro, or whatever distro you choose), this seems to be a handy (if hacky) way to attach an external VHDX to your WSL 2 session.

One way to make sure light always runs is to start your WSL session from inside a light session, like this:

wsl -d light exec wsl.exe

or even

wsl -d light exec wsl.exe ps fax

Even though it's done with an exec, the "light context" will stay around until you exit the shell or command you invoked. The exec just removes a bit of overhead from the light session. As long as you have at least one light session running, you don't need any more. Tmux or EMACS might make good command choices for this.

Another way to "keep the light on" is to use the latest Docker Desktop with the "WSL 2 backend" and activate docker in light. This will run the docker proxy in light keep it running (that appears to be the current behavior of Docker Desktop 2.1.5.0 anyway).

This doesn't work for me. I installed Alpine from here: https://github.com/yuk7/AlpineWSL

wsl -d Alpine exec wsl.exe

gets me to my Ubuntu (Default), but there's no mounted Alpine folder in /mnt/wsl/

The only folders I can see there, are the ones that docker created.

drwxr-xr-x  4 root  100 Dec 13 16:10 docker-desktop
drwxr-xr-x  3 root   60 Dec 13 16:10 docker-desktop-bind-mounts
drwxr-xr-x  5 root  100 Dec 13 16:10 docker-desktop-data

I also activated docker to run in the Alpine image, which also doesn't mount anyhting.

@zot
Copy link

zot commented Jan 6, 2020

I think I may have left out a couple key pieces. Apparently WSL 2 no longer mounts into /mnt/wsl by default but it DOES make /mnt/wsl available in all of the WSL 2 containers. I put this in my Alpine dist's /root/.profile (my home directory is in /home/bill.shared):

if [ ! -d /mnt/wsl/Alpine/bin ]; then
        mkdir -p /mnt/wsl/Alpine
        (sleep 5;  mount -o bind / /mnt/wsl/Alpine) &
fi

and I put this into my Ubuntu dist's /root/.profile:

if ! mount|grep /home/bill > /dev/null && [ -d /mnt/wsl/Alpine ]; then
        mount -o bind /mnt/wsl/Alpine/home/bill.shared /home/bill
fi

When I start my machine, I do this dance from a power shell prompt (I should also mention that Ubuntu is my default WSL environment so wsl -u root and wsl.exe -u root will start an Ubuntu session) :

  1. wsl -d alpine -u root
  2. (wait for 5 seconds and hit enter to make sure the mount has completed)
  3. wsl.exe -u root (I just execute this from inside the Alpine container since this is a temporary step)

That sets up my environment. Next, I start a wsltty and do this (maybe a Windows Terminal would work as well, not sure): exec start-tmux.

Once my start-tmux script has run, I can close the power shell prompt because start-tmux keeps an Alpine instance running so my home directory won't disappear out from under me.

I know this is elaborate and should be automated but once we can attach regular VHDX files, I (and maybe others) won't need to do this anymore :).

Here is my start-tmux script:

#!/bin/sh
tmux new-session -d
tmux source-file ~/.wsl.tmux
exec tmux attach

Here is my .wsl.tmux script:

new-window "wsl.exe -d alpine -u root"; split-window -v "run-local -it"; select-window -t 0;

I should probably clean all this up but I'm really hoping we just get the option to mount VHDXs soon so I don't have to think about maintaining this stuff :)

@zot
Copy link

zot commented Apr 27, 2020

I'm still using this fairly broken setup, starting an alpine instance, waiting 5 seconds, then starting the ubuntu instance I actually work in.

Is anyone working on letting us attach vhdx files in WSL2?

@Firsh
Copy link

Firsh commented Apr 14, 2022

I know this doesn't create a new vhdx but makes you feel like you better "own" that file:

For example if you "embrace" developing inisde WSL, your source files will likely live in this vhdx (with Docker): C:\Users\YourName\AppData\Local\Docker\wsl\data\ext4.vhdx

How about you move the data folder wherever you want that is not C partition, and then symlink it back to C? So if your main system SSD crashes or you reinstall windows you just re-create the symlink. This is also the file I'd back up as the ultimate form of backup (yes it'll grow big, but squashable while wsl is shut down) as on windows it's not much different from a VM's disk in VMware. At least it's a single file which is fast to transfer.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

7 participants