WSL2 + Docker causes severe memory leaks in vmmem process, consuming all my machine's physical memory

[davidr-PA]

Version

Microsoft Windows [Version 10.0.19044.1889]

WSL Version

• WSL 2
• WSL 1

Kernel Version

5.10.102.1

Distro Version

Ubuntu-20.04

Other Software

Docker Desktop: 4.11.1

Repro Steps

You simply need to build docker images several times. It results in this:

Note: I don't even have any active Docker containers running. Everything's terminated.

It seems to be the same as this issue, reported 3 years ago and still unresolved: #4166

I never encountered this on MacOS, but it's slowing my development speed down greatly on Windows. It definitely gets worse the more times I build Docker images. It's resulting in build processes slowing down to somewhere between 1/10th and 1/50th the speed they run at when physical memory is available. It makes me need to restart my computer sometimes multiple times per day.

Expected Behavior

Memory gets released after processes terminate.

Actual Behavior

Memory does not get released, resulting in my computer using virtual memory and slowing to a crawl.
Resource usage (with no active containers running - all terminated):

Diagnostic Logs

No response

[labanv]

I have the same issue only by running a few fixed docker images. Within 2 days the memory is 100% used only stopping Docker Destop releases the memory. So I have switched back to Docker Desktop 4.11.0

[kkruit]

This is also happening to our team. Seems to have started for us last week. It appears to be maxing out our WSL memory allotted also, as the total calculated processes memory in task manager was not equal the total memory being used.

[Drol]

Same here on Windows 11 and Docker Desktop 4.11.1. Running asp.net containers from Visual Studio.

[julestop]

This is also happening to our team. Seems to have started for us last week. It appears to be maxing out our WSL memory allotted also, as the total calculated processes memory in task manager was not equal the total memory being used.

Same for me. Memory in task manager doesn't add up to total. Limiting the wslconfig file does limit the memory af the task, however, it's like the memory doesn't get returned and after 2-3 days or runtime my computer hits 100% memory usage and I have to reboot.

[LiamKarlMitchell]

Please fix.

[chrisb2244]

Also saw this, unsure if was related to unstarted containers - resource monitor showed ~6 instances of vmmem with 1GB committed and 0KB working set (and one or two with both committed and working, looking more functional).
Task manager showed 99% RAM used, but no really large consumers (just things like Chrome, JDK, Teams, one functioning(ish, given the RAM usage) vmmem, all in 100s MB).

Also running 4.11.1.

@labanv, did rolling back help you?

[Ador-able]

Same here on Windows server 2022 and Docker Desktop 4.11.1. After a few days the server memory takes up 100% the total calculated processes memory in task manager was not equal the total memory being used.

[CurtisVermeeren]

Same issue on Windows 10 Docker Desktop v4.11.1. Left my laptop for a bit and came back to 100% memory usage with 16GB and only one image running.

[davidr-PA]

This is also happening to our team. Seems to have started for us last week. It appears to be maxing out our WSL memory allotted also, as the total calculated processes memory in task manager was not equal the total memory being used.

Same for me. Memory in task manager doesn't add up to total. Limiting the wslconfig file does limit the memory af the task, however, it's like the memory doesn't get returned and after 2-3 days or runtime my computer hits 100% memory usage and I have to reboot.

Nice, I also limited memory allocation in wslconfig. It helped, but doesn't eliminate the issue (it just allows me to get away with building more docker containers, before my machine starts grinding to a halt as it dips into virtual memory).

[JamienAU]

@labanv, did rolling back help you?

Rolled mine back to 4.10.0 and haven't noticed the issue. So far so good for the last ~24 hours.

[labanv]

@labanv, did rolling back help you?

Rolled mine back to 4.10.0 and haven't noticed the issue. So far so good for the last ~24 hours.

for me it works, memory usage is still increasing (some leaks) but sirtainly not as fast as it would hit 100% in 4.11.1 it must be a combination bug of windows vmmem and docker desktop

[julestop]

@labanv, did rolling back help you?

Rolled mine back to 4.10.0 and haven't noticed the issue. So far so good for the last ~24 hours.

Are you back on 4.11.0 or 4.10.0? Looks like you and @labanv are on different versions. I went back to 4.11.0, and seems like my memory still creeps up, maybe more slowly though.

[labanv]

@labanv, did rolling back help you?

Rolled mine back to 4.10.0 and haven't noticed the issue. So far so good for the last ~24 hours.

Are you back on 4.11.0 or 4.10.0? Looks like you and @labanv are on different versions. I went back to 4.11.0, and seems like my memory still creeps up, maybe more slowly though.

I'm back at 4.11.0 but as I say it also increases memory usage but slowly.

[julestop]

I just rolled back from 4.11.0 to 4.10.0 and rebuilt everything, will report back.

[JamienAU]

Have been monitoring memory usage for 48 h now and it's looking good, RAM usage is at the expected level and hasn't creeped up.

[ben186]

Probably related and it will be fixed soon: docker/for-win#12877

[mrgreywater]

Also duplicates #8703

[jonny-circles]

Hi mrgreywater,

I have a fresh install of both WSL and Docker 4.12 and this issue still persists.

[davidr-PA]

@mrgreywater I re-installed with Docker Desktop 4.12, and I still seem to be encountering memory leaks when I build Docker images. Will continue to test as I will definitely be building more Docker images soon - and I will confirm if I see otherwise - but as of now this appears to still be active.

EDIT:
Docker version is:

docker --version
Docker version 20.10.17, build 100c701

[ivsokol]

for me process is based on:

• stopping docker desktop
• running wsl --shutdown
• manually restarting LxssManager service - this frees up all memory
• starting docker desktop
• do it again after couple of hours, when memory is again almost full

[thangnq1001]

Try these steps:

1. in C:\Users\<yourusername>, create .wslconfig file (skip if already there)
2. Edit the file:

[wsl2]
memory=1GB # Limits VM memory in WSL 2. If you want less than 1GB, use something like 500MB, not 0.5GB
processors=2 # Makes the WSL 2 VM use two virtual processors

• Check if the file is in UTF-8
• Some guys had to change the end-of-line type to LF also (Mine was fine with CR-LF).

3. Quit docker and in PowerShell (admin), run Restart-Service LxssManager
4. Run things again. Inside wsl, run free -mh and check total value, it should be 1GB as configured above

---

Updated: Some folks below (#8725 (comment)) had couple of additional configs in the file that worked for them (kernelCommandLine and autoMemoryReclaim).

[wsl2]
memory=1GB # Limits VM memory in WSL 2. If you want less than 1GB, use something like 500MB, not 0.5GB
processors=2 # Makes the WSL 2 VM use two virtual processors
kernelCommandLine = systemd.unified_cgroup_hierarchy=1 cgroup_no_v1=all

[experimental]
autoMemoryReclaim=gradual

[albcunha]

Try these steps:

1. in C:\Users\<yourusername>, create .wslconfig file (skip if already there)
2. Edit the file:

[wsl2]
memory=1GB # Limits VM memory in WSL 2. If you want less than 1GB, use something like 500MB, not 0.5GB
processors=2 # Makes the WSL 2 VM use two virtual processors

Check if the file is in UTF-8. Some guys had to change the end-of-line type to LF also (Mine was fine with CR-LF).

3. Quit docker and in PowerShell (admin), run Restart-Service LxssManager
4. Run things again. Inside wsl, run free -mh and check total value, it should be 1GB as configured above

Finally!!! This worked great!! Thanks a lot! It was driving me crazy!!!

[riccardogabellone]

Try these steps:

1. in `C:\Users\<yourusername>`, create `.wslconfig` file (skip if already there)

2. Edit the file:

[wsl2]
memory=1GB # Limits VM memory in WSL 2. If you want less than 1GB, use something like 500MB, not 0.5GB
processors=2 # Makes the WSL 2 VM use two virtual processors

Check if the file is in UTF-8. Some guys had to change the end-of-line type to LF also (Mine was fine with CR-LF).

3. Quit docker and in PowerShell (admin), run `Restart-Service LxssManager`

4. Run things again. Inside wsl, run `free -mh` and check `total` value, it should be 1GB as configured above

Great man! Thank you!! Works like a charm for me in Windows 11, WSL2 + Docker

One more check I did:

• With no .wslconfig file enter the wsl e see cpus and ram in htop command, actual settings
• After wsl --shutdown, adding that configs and entering the wsl again, htop outputs wanted settings

Anyway, here is official doc

[chabad360]

This should not be the official recommended fix, a memory leak is a memory leak and it needs to be fixed.
I'm not interested in restarting whenever I want to do something more memory hungry.

[LiamKarlMitchell]

Well there are worse issues, when I copy files with Explorer sometimes it locks up all processes/ide etc and I can't even end task them.

[wonson]

Similar issue here.

I have a script with curl dl in a loop, multiple bg process.
I investigated the mem within linux with 'top', and the ram is perfectly fine.

But the ram for Vmmem in Win10 side keeps increasing

[jorgegarcia-ukg]

Same issue here. Running version 4.13.1 (90346).

[ifigueroap]

Same issue here

[themegabyte]

I guess 32Gb ram only cuts about 30 minutes of docker building. I should get 256GB before this is fixed since I tend to develop and test builds for more then 7 hours. Any deals around ya'all?

Edit: I figured my comment was not very helpful.

To add to some context, when I am running VSCODE and normal containers, this is not that much of a problem.

Its very noticeble when I am building containers for my staging builds. It eats way quickly.

Edit 2: I have moved to VMWARE for now. It has much better memory managmenet then WSL2.

[BastLast]

same issue there, I have to restart wsl 2 times a day...

Windows 10 + docker desktop 4.13.1

[somsomers]

Docker Desktop 4.27.0
Same issue.

[ealcantara22]

Docker Desktop 4.27.2
Same issue

[TheGuardianWolf]

Took me a month to get to this point after diagnosing random server crashes after I leave it alone for a while.

Yep I am seeing this on my media server 😭

[digitalpacman]

Same thing happening to me.

[somsomers]

I'm not sure 100%, but it seems they've fixed it in 4.28.0. Two days after update and no issues

[matej-kaska]

I'm still having that problem. Tried switching to Hyper-V and that didnt work, tried reinstall that didnt work and tried WSL config and still getting 100 % memory usage in two days of running docker containers.

Docker: 4.28.0
WSL: 2.1.5.0

.wslconfig:

[wsl2]
kernelCommandLine = systemd.unified_cgroup_hierarchy=1 cgroup_no_v1=all
memory=8GB
[experimental]
autoMemoryReclaim=gradual

I don't see that wsl/docker is using the memory in taskmgr.exe or RAMMap.

[Ali-thepro]

I deleted docker recently, but vmmem is still using a lot of memory, couple days ago it was fine now after fresh boot its using 3 gb of memory. I just have 2 wsl terminals open. If I close them it will go down to 1.5.

This is what I get from htop

I have 32 gb ram on my laptop but its getting annoying, its taking too much storage. I tried to limit it in .wslconfig but it causes perfomance issues if I set it too low.

[razeenf]

Still have the same issue on Docker Desktop version 4.28.0

[Ali-thepro]

just ran a small django and front end app on docker 👍, nice stuff wsl team.

[RohitM-IN]

Try these steps:

1. in C:\Users\<yourusername>, create .wslconfig file (skip if already there)
2. Edit the file:

[wsl2]
memory=1GB # Limits VM memory in WSL 2. If you want less than 1GB, use something like 500MB, not 0.5GB
processors=2 # Makes the WSL 2 VM use two virtual processors

Check if the file is in UTF-8. Some guys had to change the end-of-line type to LF also (Mine was fine with CR-LF).

3. Quit docker and in PowerShell (admin), run Restart-Service LxssManager
4. Run things again. Inside wsl, run free -mh and check total value, it should be 1GB as configured above

it worked for me thanks i guess i will need to gist my wslconfig XD

[wsl2]
memory=6GB
kernelCommandLine = systemd.unified_cgroup_hierarchy=1 cgroup_no_v1=all

[experimental]
autoMemoryReclaim=gradual

good tested few times using lando for me and stopped started rebuild several times didn't cause the issue

after closing the wsl --shutdown

much much better
Thanks

[Yiidiir]

Try these steps:

1. in C:\Users\<yourusername>, create .wslconfig file (skip if already there)
2. Edit the file:

[wsl2]
memory=1GB # Limits VM memory in WSL 2. If you want less than 1GB, use something like 500MB, not 0.5GB
processors=2 # Makes the WSL 2 VM use two virtual processors

Check if the file is in UTF-8. Some guys had to change the end-of-line type to LF also (Mine was fine with CR-LF).

3. Quit docker and in PowerShell (admin), run Restart-Service LxssManager
4. Run things again. Inside wsl, run free -mh and check total value, it should be 1GB as configured above

it worked for me thanks i guess i will need to gist my wslconfig XD

[wsl2]
memory=6GB
kernelCommandLine = systemd.unified_cgroup_hierarchy=1 cgroup_no_v1=all

[experimental]
autoMemoryReclaim=gradual

good tested few times using lando for me and stopped started rebuild several times didn't cause the issue

after closing the wsl --shutdown much much better Thanks

This seems to work!

[ndalchau]

I uninstalled and stopped using docker desktop ages ago, because of related problems, but was still experiencing this issue with Vmmem hogging memory, CPU, and high power usage. To reclaim my machines resources, I was closing and reopening VS code, which I think did most of the trick.

The .wslconfig additions (autoMemoryReclaim, kernelCommandLine) above have resolved my issue. Thank you!

[craigloewen-msft]

Hi folks, on the latest pre-release WSL versions we've made the 'dropCache' option on by default which should help address this use case. Please upgrade using wsl --update --pre-release and you'll get the autoMemoryReclaim=dropCache behaviour on by default and this should go a long way to fixing it! Thanks.

[prototact]

Hi folks, on the latest pre-release WSL versions we've made the 'dropCache' option on by default which should help address this use case. Please upgrade using wsl --update --pre-release and you'll get the autoMemoryReclaim=dropCache behaviour on by default and this should go a long way to fixing it! Thanks.

I did as was instructed and memory was not reclaimed. I use fedoraremix as my WSL distro and Docker Desktop as a container backend. To test it out, I started a container and installed some large python packages within. After the installation was finished, memory use fell at ~10GB from the max 12GB allowed. The container was otherwise idle, so memory should have been freed.

[NovemLinguae]

I reached an all time high the other night. 13GB memory used by WSL while running Ubuntu, Docker (mediawiki, mariadb), VS Code. Memory usage seems to climb uncontrollably until I do wsl --shutdown. Not sure if the cause is WSL, Docker, etc. but sure would be nice to get to the bottom of it.

I am also having problems with Windows File Explorer revving to like 10% CPU and staying there until I force close Windows File Explorer.

Both regressions are pretty recent for me. Last 2 months I'd say.

Windows 10

[onomatopellan]

@NovemLinguae What's your WSL version (wsl.exe --version)? Notice if you install latest version of WSL with wsl.exe --update now it should appear in task manager as VmmemWSL. If it still appears as Vmmem is not WSL and it could be a different Hyper-V Virtual machine.

[Takahiro3D]

I will share in my understood for this issue.

I tested memory leak with following 3 environments.

Environments	Memory leak
WSL2	Occurred
Docker Desktop with WSL2	Occurred
Docker Desktop with Hyper-V	Not occurred

So, memory leak is caused by WSL2 side and Docker does not affect the issue.

I checked the memory leak by RamMap.
The active area of Page Table and Unused is increased gradually.
Following screenshot is memory usage 2 days after launching WSL2.

WSL version is here.

>wsl.exe --version
WSL バージョン: 2.1.5.0
カーネル バージョン: 5.15.146.1-2
WSLg バージョン: 1.0.60
MSRDC バージョン: 1.2.5105
Direct3D バージョン: 1.611.1-81528511
DXCore バージョン: 10.0.25131.1002-220531-1700.rs-onecore-base2-hyp
Windows バージョン: 10.0.22631.3593

.wslconfig setting is here.

[wsl2]
memory=8GB
swap=0
localhostForwarding=true

[experimental]
autoMemoryReclaim=dropcache

[mjamilrahman]

WSL3 Release writen in Javascript ^_^

[Lounarok]

Couple days ago, if I ran sudo sh -c "sync;echo 3 > /proc/sys/vm/drop_caches", Ubuntu20 will reply ~15G memory available and task manager would find vmmem starts to drop gradually.
Today I ran the same command and only Ubuntu20 reports ~15G memory available but task manager reports vmmem used 17G.
The only difference is that I enabled Hyper-V feature... hope that doesn't affect it.

My .wslconfig in user

[wsl2]
memory=20GB

Using Ubuntu20.04

[repalash]

The active area of Page Table and Unused is increased gradually.

I noticed exactly the same thing. around 12GB showing Unused Active in RAMMap.

I have since been able to fix this by disabling integrated GPU from the bios (AMD CPU in my case). Also removed all the display drivers and utilities by AMD. This solved all the memory-related issues. I am assuming that the iGPU was being used in the docker container instead of the discrete Nvidia GPU, and that memory was reserved as VRAM. If someone has a better explanation for this, let me know.
After this, docker containers didn't have any hardware acceleration for rendering, which was then fixed by using nvidia runtime(container toolkit) properly in wslg.

Also disabled Hyper-V from the Windows features, but not sure if that had any effect on the wsl backend.

[Takahiro3D]

I have since been able to fix this by disabling integrated GPU from the bios (AMD CPU in my case). Also removed all the display drivers and utilities by AMD. This solved all the memory-related issues. I am assuming that the iGPU was being used in the docker container instead of the discrete Nvidia GPU, and that memory was reserved as VRAM. If someone has a better explanation for this, let me know. After this, docker containers didn't have any hardware acceleration for rendering, which was then fixed by using nvidia runtime(container toolkit) properly in wslg.

I checked memory usage with iGPU disabled (dGPU is used Radeon).
It was maintained around 50% (16GB/32GB) for over 5days.
My issue was fixed. Thanks!

[huyz]

Like some others in this thread, my particular memory issue manifests principally in the Active column of the Page Table, as shown by RamMap—not Vmmem's Memory usage in in Task Manager.

After a while, I have to reboot.

Anyone know if this is the same issue or just a separate issue as the others?

[xfl12345]

I found a cause. That is an extension of VS code and the ID is ms-vscode-remote.remote-containers . After I disabled the extension and rebooted, there was finally no more memory leak.

[huyz]

Like some others in this thread, my particular memory issue manifests principally in the Active column of the Page Table, as shown by RamMap—not Vmmem's Memory usage in in Task Manager.

After a while, I have to reboot.

Anyone know if this is the same issue or just a separate issue as the others?

To answer my own question, the issue with a ballooning Page Table seems to be separate and is discussed in https://superuser.com/a/1841517/79987 (with potential culprit being AMD) and docker/for-win#13929

[ntauth]

Had to switch back to Hyper-V because because Docker + WSL2 is rendering my Razer Blade 15 unusable. Using VSCode devcontainers makes it worse.

[shanecranor]

There exist a simple approach to prove this bug: When I quit WSL2 using command wsl --shutdown then the memory is STILL full; so it is a bug on a whole system level, and not just WSL

I am having this exact issue. Also running AMD CPU and iGPU/dGPU, but disabling iGPU is not an option for me since I'm working on a laptop.

[huyz]

Also running AMD CPU and iGPU/dGPU, but disabling iGPU is not an option for me since I'm working on a laptop.

Did you try downgrading AMD Adrenaline to v23? Worked for me

[Eboubaker]

WSL: Windows Subsystem For Linux ❌
WSL: RAM eating simulator ✔