Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WSL 2 should automatically release disk space back to the host OS #4699

Open
dsmaher opened this issue Nov 20, 2019 · 126 comments
Open

WSL 2 should automatically release disk space back to the host OS #4699

dsmaher opened this issue Nov 20, 2019 · 126 comments
Labels
feature wsl2

Comments

@dsmaher
Copy link

dsmaher commented Nov 20, 2019

Is your feature request related to a problem? Please describe.
I occasionally have workloads that consume large amounts of disk space for temporary use in /tmp. For example, transcoding large video files or processing large datasets exported from a database. Since WSL2 stores its filesystem on a .vhdx, that file grows when the dataset is processed and never releases that space. Also, it doesn't appear to re-use released disk space in the .vhdx when files are deleted... it seems to prefer to grow the file rather than re-use existing empty space. (I haven't explicitly tested that theory, but my workload deletes temporary files as they are used and I never had enough of them existing simultaneously to reach 250GB, but the size of the .vhdx eventually expanded to that size.

When I'm done with that workload, I have to:

wsl --shutdown
optimize-vhd -Path .\ext4.vhdx -Mode full

This is annoying since it basically means that my system backups include a huge .vhdx file that is mostly empty. Also, until about 2 weeks ago, it was consuming the entire remainder of my system disk. I had only 50GB free, which was enough to handle the workload, but since it continued to grow even after files were removed, that didn't matter. (I figured it was time to upgrade the disk size anyway)

Describe the solution you'd like
Automatic compaction of the .vhdx, or a way to do so while WSL2 is still running so that I can schedule it for frequent cleanups.

Describe alternatives you've considered

  • Going back to WSL1 since it uses the host filesystem and is faster accessing that than WSL2. However, this alternative makes docker less useful. Docker on WSL2 is actually almost perfect; not so much on WSL1.
  • Exporting/Importing WSL 2 to move it to another disk. Not even sure if this is possible, but if it is,
    it would probably be fine for my desktop, but not really an option for my laptop. Not really worth the time either since that entire process would probably take about 30 minutes, but compacting is usually under 3.
@adrianciacob
Copy link

adrianciacob commented Nov 27, 2019

+1 for being able to move the .vhdx to a different partition, either via config file or the "Move" feature in the Control Panel of Windows (currently getting a message that it's disabled)

@dvescovi1
Copy link

dvescovi1 commented Dec 8, 2019

+1 here too

@craigloewen-msft craigloewen-msft added feature wsl2 labels Dec 9, 2019
@wizhippo
Copy link

wizhippo commented Dec 11, 2019

+1 also add the ability to specify the initial size limit.

@dvescovi1
Copy link

dvescovi1 commented Dec 12, 2019

I also would like to see a setting for initial partition size limit. It looks like all distros are installed with a 250GB partition size. I see no (easy) way to extend this. My yocto builds require slightly more space than this and they have to be on an ext3/4 partition so I can not use the /mnt/c or other NTFS partitions.

@davidwin
Copy link

davidwin commented Dec 14, 2019

In Home you don't even have the Optimize-VHD cmdlet, but thankfully (the slightly clumsier) diskpart command which can also be used to compact the .vhdx file. When I did this, it seemed like I first had to run sudo fstrim / in WSL to get a significant saving (14GB reclaimed with 8GB in use). A periodic background task that did this is really needed (and even better if it could be done online, of course).

The default partition size should have some connection to the size of the Windows host partition. What is the use of a limit that exceeds that of the host (unless actual magic is involved)? Also, artificially limiting the maximum size to a fraction of that of the host seems a bit unnecessary.

I think you really want an experience that is as close to that of WSL1 as possible, but with the incredible perf and compat gains, of course. That is, the WSL2 file system shouldn't feel like it was a static volume bound to a file on the host (although it is). Ideally, even the WSL partition/volume size should shrink and grow with available space on the host in a somewhat timely fashion. Now there is a stretch goal for you!

@taylankasap
Copy link

taylankasap commented Jan 15, 2020

FWIW, since these commands looked benign I ran them to reclaim some disk space:

wsl --shutdown
optimize-vhd -Path .\ext4.vhdx -Mode full

This broke my docker. I had to reset to factory defaults & reinstall docker and restart the pc a few times along the way to fix. All docker data is gone (doesn't matter in my case because it's just dev environment). YMMV.

Microsoft Windows 19041.1 with WSL2
Docker Desktop Edge 2.1.7.0 (41536) | Engine: 19.03.5

@StefanScherer
Copy link

StefanScherer commented Jan 17, 2020

I've tried the commands as well. I could find an issue that docker engine (docker.sock) is not available after starting. But after restarting Docker Desktop and changing the settings / resources / wsl integration to off + apply, then to on again + apply did the trick. Looks like the sock

I ran the test a second time, but now my whole Ubuntu 18.04 distro is broken, the terminal closes either immediately or after some seconds without any text in it.

@taylankasap
Copy link

taylankasap commented Jan 17, 2020

Maybe disabling wsl integration first, then quitting docker and then using the optimize-vhd command can make it work. But I'm not going to take that risk now.

@StefanScherer
Copy link

StefanScherer commented Jan 17, 2020

It's back alive, got the Ubuntu distro working again. Seem like I just have to wait a little bit. Terminal opens and I have the bash prompt. The WSL integration was disabled, after enabling again docker also works again in Ubuntu.

@MatyRi
Copy link

MatyRi commented Apr 25, 2020

If you are looking for a way how to reclaim your disk space from Docker using WSL2, then there is a button for this if you are using Docker for Desktop (Edge 2.3.0.0). In the Dashboard / Troubleshoot - there is a Clean / Purge Data option which will let you select what do you want to remove. It safely shrank the Docker vhdx disk file to a few kilobytes. Just a warning that this obviously removes everything and not just unused space.

image

It does not solve the original issue but it's a quick way to a fresh start.

@merkuriy
Copy link

merkuriy commented May 12, 2020

For Windows 10 Home (alternative Optimize-VHD cmdlet):

wsl --shutdown
diskpart
# open window Diskpart
select vdisk file="C:\WSL-Distros\…\ext4.vhdx"
attach vdisk readonly
compact vdisk
detach vdisk
exit

Thanks to @davidwin for the tip #4699 (comment).

@sorenwacker
Copy link

sorenwacker commented May 14, 2020

I just got no disc space, I have not even installed much. I added a symbolic link that was no problem on my old system.

@nickjj
Copy link

nickjj commented May 28, 2020

I'm on Windows 10 Pro build 2004 here (stable 2020 spring edition) and I just noticed I lost 60GB on my primary SSD from this because I started moving my source code into WSL 2's file system and forgot one of my podcast sites has a bunch of wave files in a private directory.

When I run optimize-vhd -Path .\ext4.vhdx -Mode full as admin from PowerShell it throws an error with:

optimize-vhd : '.\ext4.vhdx' is not an existing virtual hard disk file. 
At line:1 char:1                                                                                                        
+ optimize-vhd -Path .\ext4.vhdx -Mode full
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo          : InvalidArgument: (:) [Optimize-VHD], VirtualizationException
+ FullyQualifiedErrorId : InvalidParameter,Microsoft.Vhd.PowerShell.Cmdlets.OptimizeVhd                                                                    

How can I get around this error? This put my SSD at a dangerously low level of space and I'd rather not have to reinstall the entire WSL 2 instance from scratch and reconfigure Docker every time I accidentally copy an unused file into WSL 2's file system.

@onomatopellan
Copy link

onomatopellan commented May 28, 2020

@nickjj You need to be in the directory where the distro's vhdx resides. In my case with Ubuntu20.04 I had to

wsl.exe --shutdown
cd C:\Users\onoma\AppData\Local\Packages\CanonicalGroupLimited.Ubuntu20.04onWindows_79rhkp1fndgsc\LocalState\
optimize-vhd -Path .\ext4.vhdx -Mode full

@nickjj
Copy link

nickjj commented May 28, 2020

Thanks, I was able to find it there too. That and what @StefanScherer mentioned about restarting Docker, along with disabling / enabling the distro in Docker Desktop again got things back to normal without any system level restart.

@rainabba
Copy link

rainabba commented May 28, 2020

My contribution, bringing together knowledge from above. It still requires a little manual work to get the DistroFolder variable set for the distro you want to target (you could have more than one, find them with wsl --list)

Cheers!

### Optimize (shrink) WSL 2 .vhdx
## Must be run in PowerShell as Administrator user
# DistroFolder found at: $env:LOCALAPPDATA\Packages\
# Examples:
#   CanonicalGroupLimited.UbuntuonWindows_79rhkp1fndgsc
#   CanonicalGroupLimited.Ubuntu20.04onWindows_79rhkp1fndgsc

cd $env:LOCALAPPDATA\Packages\REPLACE_ME_WITH_TARGET_DISTRO_FOLDERNAME\LocalState\
wsl --shutdown
optimize-vhd -Path .\ext4.vhdx -Mode full
#Run `wsl` or your favorite terminal to resume use

BELOW UNTESTED!!
More info at https://docs.microsoft.com/en-us/powershell/module/hyper-v/resize-vhd?view=win10-ps

### Resize WSL 2 .vhdx
## WARNING!! UNTESTED!! Read and make note of the MaxSizeGB variable and functionality before running
## Must be run in PowerShell as Administrator user
# DistroFolder found at: $env:LOCALAPPDATA\Packages\
# Examples:
#   CanonicalGroupLimited.UbuntuonWindows_79rhkp1fndgsc
#   CanonicalGroupLimited.Ubuntu20.04onWindows_79rhkp1fndgsc

Set-Variable -Name "MaxSizeGB" -Value "32GB"
cd $env:LOCALAPPDATA\Packages\REPLACE_ME_WITH_TARGET_DISTRO_FOLDERNAME\LocalState\
wsl --shutdown
resize-vhd -Path .\ext4.vhdx -SizeBytes $MaxSize
#Run `wsl` or your favorite terminal to resume use

@eromoe
Copy link

eromoe commented Jun 4, 2020

Both cmd and powershell can't find optimize-vhd command even in admin preveliege

optimize-vhd : 无法将“optimize-vhd”项识别为 cmdlet、函数、脚本文件或可运行程序的名称。请检查名称的拼写,如果包括路径
,请确保路径正确,然后再试一次。
所在位置 行:1 字符: 1
+ optimize-vhd -Path .\ext4.vhdx -Mode full
+ ~~~~~~~~~~~~
    + CategoryInfo          : ObjectNotFound: (optimize-vhd:String) [], CommandNotFoundException
    + FullyQualifiedErrorId : CommandNotFoundException

@onomatopellan
Copy link

onomatopellan commented Jun 4, 2020

@eromoe optimize-vhd is only available in Windows 10 Pro with Hyper-v feature installed. Otherwise you will need to use the compact option in Diskpart.

@mieslep
Copy link

mieslep commented Jun 9, 2020

The WSL2 disk management is definitely a pig. Developing some (legacy app) Docker containers, after a week my "working" WSL2 .vhdx is 110G (with only 14 GB of actual data) and my docker-desktop-data .vhdx is 40G (with only 3 GB of actual data).

@nickjj
Copy link

nickjj commented Jul 6, 2020

Has anyone ever seen this when trying to run Optimize-VHD to compact things?

Optimize-VHD : Failed to compact the virtual disk.
The system failed to compact 'C:\Users\Nick\AppData\Local\Packages\CanonicalGroupLimited.Ubuntu20.04onWindows_79rhkp1fndgsc\LocalState\ext4.vhdx'.   Failed to compact the virtual disk.
The system failed to compact
'C:\Users\Nick\AppData\Local\Packages\CanonicalGroupLimited.Ubuntu20.04onWindows_79rhkp1fndgsc\LocalState\ext4.vhdx':   The process cannot access the file because it is being used by another process. (0x80070020).
At line:1 char:1
+ Optimize-VHD -Path .\ext4.vhdx -Mode full
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo          : ResourceBusy: (:) [Optimize-VHD], VirtualizationException
+ FullyQualifiedErrorId : ObjectInUse,Microsoft.Vhd.PowerShell.Cmdlets.OptimizeVhd  

At this point my drive has almost no space and I'm not finding anything on Google on how to compact it, short of destroying the entire WSL 2 instance and remaking it from scratch.

I've stopped Docker Desktop and everything. There's no hint on what's using it.

@onomatopellan
Copy link

onomatopellan commented Jul 7, 2020

@nickjj With wsl.exe --shutdown should be enough to free all the mounted vhdx files.

@datocrats-org
Copy link

datocrats-org commented Jul 9, 2020

The second alternative mentioned is proving the fastest and most reliable for me, Exporting/Importing WSL 2 to move it to another disk. The other thing I have tried is converting between Fixed and Dynamic volumes in combination with this. My use is for docker's WSL2 integration, and the need to export and restore large data volumes (>250GB). This is similar to what most of the respondents are referring to, yet they do not always say whether they are using WSL1 vs. WSL2. WSL1 sets a fixed disk size as specified in your Docker settings. WSL2 uses windows default setting of 256GB max volume size. I could not bash into or ssh into the docker-data distro shown or access its ext4.vhdx volume through various methods. What would be really nice is if WSL would adopt a docker overlay2 driver or some other more friendly HyperV linux filesystem, faster than vhdx on ntfs, with an API to control allocation. Even better would be some type of cloud filesystem integration that could rival local dev speeds.

Here are my notes in gist

@jcoutch
Copy link

jcoutch commented Jul 14, 2020

I just symlinked ~\AppData\Local\Docker\wsl\data to a folder on a different partition/drive, and moved the VHDX. While the problem with the VHDX growing is still there, at least it won't eat all the space on my primary drive. I wrote up a blog post about it, and here's the script I used (use at your own risk):

$ErrorActionPreference = "Stop"

$newLocation = "E:\VMs\WSL2\"

cd "~\AppData\Local\Docker\wsl\data"
wsl --shutdown
Optimize-VHD .\ext4.vhdx -Mode Full
mkdir $newLocation -Force
mv ext4.vhdx $newLocation
cd ..
rm "data"
New-Item -ItemType SymbolicLink -Path "data" -Target $newLocation

@onomatopellan
Copy link

onomatopellan commented Jul 15, 2020

@mikemaccana That's where Docker Desktop with WSL2 backend saves the docker images. Use the Troubleshoot option in the Dashboard to clean them.

@mikemaccana
Copy link

mikemaccana commented Jul 15, 2020

OK, ran:

image

To get rid of the docker images that had been accumulating. I have my hard drive back, and Docker is now pulling down images (good) but it's failing with:

ERROR: Service 'postgres' failed to build: Get https://registry-1.docker.io/v2/: dial tcp: lookup registry-1.docker.io on 192.168.65.1:53: read udp 192.168.65.3:39176->192.168.65.1:53: i/o timeout

Which I'm now troubleshooting.

Edit: since the IP addresses are local, poking around in Docker networking config and restarting docker fixed it.

@onomatopellan
Copy link

onomatopellan commented Jul 15, 2020

For further Docker Desktop issues is better to ask them in https://github.com/docker/for-win/issues

@AmineI
Copy link

AmineI commented Jul 15, 2020

@mikemaccana 150GB taken in Docker wsl/data vdhx

This issue is also logged here indeed docker/for-win#244 , go there for Docker-specific elements; as mentioned by onomatopellan above.

@anlandu
Copy link

anlandu commented Nov 11, 2021

My corrupted ext4.vhdx seems to be unmountable even as a regular disk outside of WSL; it's not mountable by things like guestmount or 7zip's vhdx extractor. Maybe Diskpart is touching the vhdx header, or a partition table, causing the compacted vhdx to be unreadable.

The files and directories themselves still exist though. You are able to recover the files with DMDE: image

(@woss @shuoyang129)

Thank you so much @aetheryx, had a corrupted disk after compact vdisk as well and this saved days of work!!!

@qwertynik
Copy link

qwertynik commented Nov 22, 2021

For Windows 10 Home (alternative Optimize-VHD cmdlet):

wsl --shutdown
diskpart
# open window Diskpart
select vdisk file="C:\WSL-Distros\…\ext4.vhdx"
attach vdisk readonly
compact vdisk
detach vdisk
exit

Thanks to @davidwin for the tip #4699 (comment).

@merkuriy
Any idea if this deletes any useful files from WSL2? Is running this similar in effect to cleaning up the Recycle Bin on Windows?

Looking forward to your response and reclaiming some 'space' back 😄

Thanks!

@diegoclair
Copy link

diegoclair commented Nov 23, 2021

For Windows 10 Home (alternative Optimize-VHD cmdlet):

wsl --shutdown
diskpart
# open window Diskpart
select vdisk file="C:\WSL-Distros\…\ext4.vhdx"
attach vdisk readonly
compact vdisk
detach vdisk
exit

Thanks to @davidwin for the tip #4699 (comment).

@merkuriy Any idea if this deletes any useful files from WSL2? Is running this similar in effect to cleaning up the Recycle Bin on Windows?

Looking forward to your response and reclaiming some 'space' back 😄

Thanks!

@qwertynik I tried it now and it looks that do not delete any useful files. And it free some space!

@qwertynik
Copy link

qwertynik commented Nov 24, 2021

For Windows 10 Home (alternative Optimize-VHD cmdlet):

wsl --shutdown
diskpart
# open window Diskpart
select vdisk file="C:\WSL-Distros\…\ext4.vhdx"
attach vdisk readonly
compact vdisk
detach vdisk
exit

Thanks to @davidwin for the tip #4699 (comment).

@merkuriy Any idea if this deletes any useful files from WSL2? Is running this similar in effect to cleaning up the Recycle Bin on Windows?
Looking forward to your response and reclaiming some 'space' back 😄
Thanks!

@qwertynik I tried it now and it looks that do not delete any useful files. And it free some space!

Thanks for confirming @diegoclair. That instills some confidence in the commands. Hope there are no bitter surprises awaiting :)

@germangp088
Copy link

germangp088 commented Dec 3, 2021

OK, ran:

image

To get rid of the docker images that had been accumulating. I have my hard drive back, and Docker is now pulling down images (good) but it's failing with:

ERROR: Service 'postgres' failed to build: Get https://registry-1.docker.io/v2/: dial tcp: lookup registry-1.docker.io on 192.168.65.1:53: read udp 192.168.65.3:39176->192.168.65.1:53: i/o timeout

Which I'm now troubleshooting.

Edit: since the IP addresses are local, poking around in Docker networking config and restarting docker fixed it.

Thanks! this worked for me after tried everything else for hours!

@berlincount
Copy link

berlincount commented Feb 4, 2022

What I absolutely like best is the way you can do it on VirtualBox: the disk image is a sparse file (i.e. with holes in it) like the vhdx file, but when you have your linux configured correctly (i.e. LVM, block devices, etc) the simulated SSD forwards the deletes and trims from ext4 to the underlying blockdevice through to VirtualBox, which re-punches the hole into the underlying disk image, releasing the space on the host operating system. I consider this the optimum solution, as it releases the space at runtime already.

@mahdibx
Copy link

mahdibx commented Feb 12, 2022

some programs write files constantly then clear them, yet wsl expands the disk continuously anyway. It should first take available space in the virtual disk before trying to expand it.

@philippe-granet
Copy link

philippe-granet commented Feb 25, 2022

Is there a solution to compact disk without admin rights?
diskpart or optimize-vhd commands needs admin rights and admin account is blocked on my Corporate computer.

@svet-am
Copy link

svet-am commented Mar 6, 2022

I got tired of trying to do this manually so I wrote a PowerShell script to do this more-or-less automatically. It's not perfect and I could use some help cleaning it up but it's here in case anyone else wants it: https://github.com/svet-am/wsl-vhd-compactor/blob/main/compact_vhds.ps1

It uses diskpart directly to get around the issue where optimize-vhd might not be available on a particular host.

@qwertynik
Copy link

qwertynik commented Mar 28, 2022

I got tired of trying to do this manually so I wrote a PowerShell script to do this more-or-less automatically. It's not perfect and I could use some help cleaning it up but it's here in case anyone else wants it: https://github.com/svet-am/wsl-vhd-compactor/blob/main/compact_vhds.ps1

It uses diskpart directly to get around the issue where optimize-vhd might not be available on a particular host.

Thanks for sharing the script @svet-am. Any ideas if this deletes files on Docker? As per this comment, #4699 (comment) working on the vhdx file led to important files being deleted.

@mikemaccana
Copy link

mikemaccana commented Mar 28, 2022

I got tired of trying to do this manually so I wrote a PowerShell script to do this more-or-less automatically. I

@svet-am a few of us have been working on https://github.com/mikemaccana/compact-wsl2-disk, you're welcome to contribute there if you like, or not, that's fine too.

@svet-am
Copy link

svet-am commented Mar 29, 2022

Thanks @mikemaccana - wasn't aware of your project when I wrote mine. I'll contribute over on yours as the need arises. Yours is a bit less brute force than mine anyway.

@theAkito
Copy link

theAkito commented Apr 22, 2022

The optimize-vhd -Path .\ext4.vhdx -Mode full workaround does not work anymore, on my computer. I am stuck with a 69GB ext4.vhdx on my NVMe SSD and I don't know what to do.

@Chris2011
Copy link

Chris2011 commented Apr 22, 2022

The optimize-vhd -Path .\ext4.vhdx -Mode full workaround does not work anymore, on my computer. I am stuck with a 69GB ext4.vhdx on my NVMe SSD and I don't know what to do.

I moved the vdisk to an external disk. I followed these steps: http://nuts4.net/post/moving-wsl2-vhdx-file-to-a-different-location

@svet-am
Copy link

svet-am commented Apr 22, 2022

The optimize-vhd -Path .\ext4.vhdx -Mode full workaround does not work anymore, on my computer. I am stuck with a 69GB ext4.vhdx on my NVMe SSD and I don't know what to do.

I moved the vdisk to an external disk. I followed these steps: http://nuts4.net/post/moving-wsl2-vhdx-file-to-a-different-location

Did you try ring diskpart method rather than optimize-vhd? I just tried and the diskpart method still works for me even with very large (100GB+) VHD.

@Chris2011
Copy link

Chris2011 commented Apr 22, 2022

I also had problems with both neither diskpart nor optimze-vhd worked for me. So I moved the file to my external SSD and I can live with that.

@theAkito
Copy link

theAkito commented Apr 23, 2022

Did you try ring diskpart method rather than optimize-vhd?

Yes, I tried the workaround for the workaround, but it showed the same symptoms. Compacting goes up to 10% instantly, then slowly progresses to around 20% and then instantly finishes with 100%. Then, I check the disk space and nothing changed, at all.

I moved the vdisk to an external disk. I followed these steps: http://nuts4.net/post/moving-wsl2-vhdx-file-to-a-different-location

The problem with that is, that the disk would keep growing infinitely. So, it does not solve any problem, except maybe my SSD dying quickly. I also have some spare space on hard drives, but I won't waste hundreds of GB, because Windows is too stupid to compact a native virtual disk.


When not even the workaround for the workaround works for some users, how does this not have a huge priority? How does the majority of the user base use Docker Desktop on Windows without pulling their hair out after a couple of months of accumulating space for no reason? Even if the workarounds work, this situation is awful.

The outcry about this issue is not adequately representing the urgency of this situation. I probably would need to purge Docker Desktop or just plainly force remove the vhdx, soon. Which would also be the case for anyone else, who cannot even get the workaround for the workaround to work...

@sarim
Copy link

sarim commented Apr 23, 2022

I personally removed docker desktop long ago, after having to diskpart the disk a few times, also other reasons. Now I just run podman (rootless container). diskpart is still working for me. But you can try recreating the distro, wsl --export to export it, then wsl --import to import it again. This export / import makes the whole distro/container a tar archive, so it'll not take any extra disk space, I hope.

As for the main issue, I guess hyperv's commands / technology were not really made for ext4? In that case error/bug report should be reported to hyperv team I guess.

@qwertynik
Copy link

qwertynik commented Apr 28, 2022

@craigloewen-msft
Although there are many solutions suggested here, not certain how it would impact the distro - could there be any data loss, etc. Like me, there would be many who are not fully aware of the internals and would be hesitant to try out the solutions here.

Can the WSL team look into this and suggest an official way to resolve this issue?

@theAkito
Copy link

theAkito commented Apr 28, 2022

Although there are many solutions suggested here, not certain how it would impact the distro - could there be any data loss, etc.

I think someone mentioned possible data loss, somewhere in this long thread. Using one of the provided workarounds is definitely not super safe. Apply at your own risk.

Can the WSL team look into this and suggest an official way to resolve this issue?

That's what we are waiting for, since 2019. 😄

@richyeiv
Copy link

richyeiv commented May 14, 2022

In my case the problems were related to disk space. When you export the .vhdx is removed and replaced by a .tar. On import and recreate the vhdx you need at least twice the space (vhdx + tar). At first it gave the 'unespecified error' message then when I tried to open the .tar it said the file was corrupted.
In the end I moved the file to another drive with more space and imported in the same directory where the .tar was located:
wsl --import Ubuntu .\ .\Ubuntu.tar
It works for me.

@mklueh
Copy link

mklueh commented May 15, 2022

Having permission issues with "optimize-vhd -Path .\DockerDesktop.vhdx -Mode full" despite having launched PowerShell as admin

@Zaffer
Copy link

Zaffer commented May 15, 2022

For WSL2 Ubuntu it is recommended to use Docker Desktop. If you read here they specifically mention VHD space concern:

"`If you have concerns about the size of the docker-desktop-data VHDX, or need to change it, take a look at the WSL tooling built into Windows."

Instructions here: https://docs.microsoft.com/en-us/windows/wsl/tutorials/wsl-containers

@iuriguilherme
Copy link

iuriguilherme commented May 24, 2022

To all of you copying and pasting commands from github issues (yes, you!): STOP.

If you have a large, inefficient virtual hard drive that you need to optimize, will you please read the manual of the tool, it's available in the official website: https://docs.microsoft.com/en-us/powershell/module/hyper-v/optimize-vhd

What the OP is complaining about is the fact that with WSL the hard drives keep growing and no automatic effort is made to keep them trimmed. In order to perform any operations on those virtual hard drive files, you have to go through the checklist:

  • Drive is mounted in read only mode. This is not the case if you are running processes in the virtual machine which is using the virtual hard drive. Hence you must stop the running distribution with --terminate and the WSL program with --shutdown, in power shell: wsl.exe --shutdown. This will turn off all the virtual machines, so only issue the command when you sure you're ready for this. Actually, you don't really have to shutdown the WSL program if you're sure the virtual machine is not running. But if you feel either too confident or in much doubt, you probably want to be in the safe side. In most circumstances, if you just close the open terminals for that distro, everything will stop in a couple of seconds. But if you're running any daemons or detached scripts, then you'll have to terminate it.
  • Find where the virtual hard drive for the target distribution is. Each machine will have a unique path. If you can't find this, you don't know which file you have to perform the command into, and the command will either do nothing or wreck something else. An example of a common place, your mileage will vary: C:\Users\user\AppData\Local\Packages\TheDebianProject.DebianGNULinux_49vf1zgz67zfv\LocalState\ext4.vhdx.

Be aware that this operation, if done incorrectly can lock you out of your virtual machine and if you're not an expert in fixing volumes, you may have to start over from scratch. Use the options -WhatIf to check what will happen and when you're sure it's the right command, run it with -Confirm so it asks you for confirmation before each operation.

You may want to consider if you really need the Full mode. In most circumstances, the Retrim is all you need. If that doesn't claim back space in the underlying NTFS partition, then the Quick mode will reclaim all unused space without seeking for blocks filled with zeroes. You would only use the Full option if you really need to compact the virtual hard drive as much as possible and you have already used UNIX tools inside the virtual machine to prepare and fill empty spaces with zeroes beforehand. After you do the initial trim of the disk using the windows tool, weekly management can be done with something easier such as the Pretrimmed option, which doesn't require you to do a wsl.exe --shutdown first.

So all things considered, the actual command in this example would be:

  1. Check if everything will work as expected
Optimize-VHD -Path "C:\Users\user\AppData\Local\Packages\TheDebianProject.DebianGNULinux_49vf1zgz67zfv\LocalState\ext4.vhdx" -Mode Retrim -Confirm -Whatif
  1. Actually running it
wsl --shutdown
Optimize-VHD -Path "C:\Users\user\AppData\Local\Packages\TheDebianProject.DebianGNULinux_49vf1zgz67zfv\LocalState\ext4.vhdx" -Mode Retrim -Confirm
  1. Seeing it doesn't do what you want and come back to github to complain about it

@tomaspritrsky
Copy link

tomaspritrsky commented Jun 28, 2022

bump. I see no reason why docker should consume disk space to infinity.

@dbjungle
Copy link

dbjungle commented Jul 16, 2022

Is there a process to manually migrate the WSL2 VHDX disk to a smaller VHDX? I'm not comfortable with the default max size of 250GB and I'd like to change it to something smaller even if there is some effort involved.

@andymcblane
Copy link

andymcblane commented Aug 10, 2022

Also ran into this issue, after seeing 0 bytes free on C:\

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature wsl2
Projects
None yet
Development

No branches or pull requests