Docker process doesn't free up memory - macOS, Apple Silicon, Virtualization.framework

[z1r0-]

With Virtualization.framework enabled the Docker process (not Virtual Machine Service) grows in memory-usage and doesn't free it up properly. After some testing I noticed, that it seems to be related to transfers between host and docker-container.
Easiest example: copy a 1GB file using "docker cp" and the memory-usage will grow by 1GB. But happens with other stuff too (like importing a db with local client over a host-bind port).

Example:

dd if=/dev/zero of=output.dat  bs=1m  count=1024
docker run -d --rm --name mem_test alpine tail -f /dev/null
docker cp output.dat mem_test:/tmp/
# Watch Memory grow in Activity Monitor

I've seen the discussion about "Memory" and "Real Mem" in Activity Monitor and first thought "okay, Real Mem returns to normal after some time so it will be fine" but then I noticed, that MacOS starts swapping when "Memory" value gets to big even "Real Mem" is just some MB.

After restarting Docker (from docker context menu) everything goes back to normal again.. So I'm currently restarting Docker every time memory usage gets to huge.. I've usually 20+GB of free memory and no swap usage if I don't forget to restart Docker, so it's deff it.

I'm using the most recent version: Docker Desktop 4.3.2 (72729) and MacOS 12.2 Beta (but I had same behaviour with 12.0, 12.1).

[djs55]

@z1r0- thanks for the report. Could you upload a set of diagnostics and a matching screenshot? I'd like to confirm exactly which process is leaking (and view the command-line etc)

[z1r0-]

@djs55 Sure. Diagnostics ID: 72F4E822-C849-4117-981D-06B38A63C1BA/20220106145448

Used docker cp to let it grow (like in the example above) and stopped/removed the container.

[bjonnh-work]

I can confirm that, let me know if you need any additional info/testing/logs.

[joaopbnogueira]

Pitching in as I have to restart docker multiple times a day to work around these issues

I'm currently on the experimental build 4.6.0 (75045) running 12.2.1 on an M1 Pro with VirtioFS enabled, but the behavior is the same on the GA 4.5.0 version.

Diagnostics ID AF628F15-7B04-4806-BEE7-33B5827B55A8/20220309231313

[bjonnh-work]

75324 has been well behaved for me for the last few days (M1 and VirtIO FS enabled)

[z1r0-]

uh. completely missed, that there's a 75324 build. just downloaded it and made the docker cp test and the issue sadly is still there.

75324 has been well behaved for me for the last few days (M1 and VirtIO FS enabled)

It completely depends on what you are doing. On days where I only need to code it works without any issues and I can let it run for days. But when I need to import db-dumps > 15GB it's a huuuge issue. Started to copy the gziped dumps to the db container and then import them instead of using local psql client to reduce the issue but it still remains a problem.

Something else: I'm also using mutagen for better io-performance and somehow it seems not to be affected by this issue. It's only "docker cp" and "shared ports" that I identified so far.

[tysliu]

M1 and VirtIO FS enabled. Docker version 4.6.0 (75818). A high disk IO app runs for 5 minutes docker process ends up using 6 GB of ram. This is the high disk IO container I was running lscr.io/linuxserver/transmission:arm64v8-latest.

[joaopbnogueira]

Checking in as this issue is still present on 4.6.1.

Docker process using 103GB (and swapping a lot of course).

Diagnostics ID: AF628F15-7B04-4806-BEE7-33B5827B55A8/20220331104836

[marcoscale98]

Alsa without virtualization.framework!

[yunussandikci]

I am having the same issue with Docker Desktop 4.7.0 (77141)
Diagnostic ID: 79C10953-31E7-4308-9955-4FFEF6A3129F/20220418100515

[b4rlw]

What actually is Virtual Machine Service?