Memory Leakage com.docker.hyperkit

[Sebastianbrg]

NOTES TO FUTURE READERS OF THIS THREAD.
PLEASE READ THE WHOLE THREAD BEFORE REPLYING. IN PARTICULAR, PLEASE UNDERSTAND:

1. There is a bug in MacOS that it reports twice the amount of memory actually allocated.
2. It is normal that all the memory has to be reserved up front before any containers are running, because of the way that virtualisation works in MacOS Hypervisor.framework.

See https://docs.google.com/document/d/17ZiQC1Tp9iH320K-uqVLyiJmk4DHJ3c4zgQetJiKYQM/edit?usp=sharing for a detailed investigation on this.

-- Stephen Turner

---

• [ x ] I have tried with the latest version of my channel (Stable or Edge)
• [ x ] I have uploaded Diagnostics
• Diagnostics ID:
  C19AE3AD-3E52-4F8C-A16A-D06D797B36A4/20180925100823

Expected behavior

No memory leakage.

Actual behavior

Memory leakage.

Information

• Is it reproducible? Yes. Just start Docker on Mac OS Mojave
• Is the problem new? Yes, when I upgraded to Mac OS Mojave
• Did the problem appear with an update?
• A reproducible case if this is a bug, Dockerfiles FTW.
• macOS Version: 10.14 (18A391)

Diagnostic logs

Docker for Mac: Version 18.06.1-ce-mac73 (26764)

[lou-lan]

I also get a memory leak

[sickness29]

Me too

[EduardRakov]

and me too :( also com.docker.hyperkit loads CPU very much

[Zabudska-Yuliia]

Me too

[iMerica]

See #178.

[vfontes]

Also happening for me. Running on macOS High Sierra 10.13.6.

Docker version 18.06.1-ce, build e68fc7a

[gvbkr]

Yes this is happening on macOS High Sierra 10.13.6.

Docker version Version 18.06.1-ce-mac73 (26764)

This started happening after I updated the Docker version.

[ylor]

Seeing the same thing.

2.83GB being used without anything running.

Docker version 18.06.1-ce, build e68fc7a

[AnielloFalcone]

4.77Gb with no images and no containers on:
macOS: 10.14.1 Beta (18B45d),
Docker: 2.0.0.0-beta1-mac75 (27117),
Engine: 18.09.0-ce-beta1

[mkohlmyr]

Mac OS High Sierra 10.13.4
Docker 18.03.1-ce

Regularly consumes ~1.5GB while completely idle after startup.

[ghost]

[ x ] I have tried with the latest version of my channel (Stable or Edge)
[ x ] I have uploaded Diagnostics
Diagnostics ID: 01B9140E-D975-46AE-AEA6-442DC281FAE0/20181001181235
Expected behavior: Low memory use
Actual behavior: Doing nothing and consuming 5Gb of memory, it is also reported as the highest energy consumer on my system.

Information
Is it reproducible? Yes. Just start Docker on Mac OS Mojave
Is the problem new? Don't know, never used it on a different version
Did the problem appear with an update? Don't know, never used it on a different version.

MacOS 14 (18A391)
Docker Community Edition 18.06.1-ce-mac73 (26764)

Hope this helps.

[gauravthadani]

Likewise, noticing this after the os upgrade to mojave.
does the update fix the issue? I couldn't figure out from the commits

[asburymr]

I have run into the same issue. Below are the specifics of my environment. This is with no containers running and a fresh restart.

[asburymr]

An interesting data point may be that increasing the SWAP seems to dramatically affect the Memory consumed by the hyperkit.

512MB SWAP -> 2GB of memory

1GB SWAP -> 3 GB Memory

2 GB SWAP -> 5 GB Memory

[alvarobem]

I have the same issue

[wskinner-eml]

Same here. With a container running, com.docker.hyperkit used over 9GB of memory. After a restart, with no containers running, it immediately jumped up to 5.23 GB.
Docker version: 18.06.1-ce-mac73 (26764)
Docker memory limit: 12GB
Docker Swap limit: 4.0GB

I was also able to reproduce the effect @asburymr noticed. When I reduced the swap limit to 512MB, hyperkit went down to under 2GB RAM.

[andypan-twn]

Same issue.

[imuller]

Same issue here with version 18.06.1-ce-mac73, using almost 3 GB memory after startup of my mac with Mojave (10.14)

[TedSpinks]

I have this issue with High Sierra 10.13.6 and Docker version 18.06.1-ce-mac73 (26764). com.docker.hyperkit consumes 1.6GB with no containers and 1GB swap. 800MB with no containers and 512MB swap.

Running a container causes com.docker.hyperkit to consume 50-60MB while it is running, but it releases the 50-60MB as soon as the container stops.

[rurkss]

same here, on mac

[bradrich]

Upgrading to the Edge release seems to alleviate the problem. The below screenshot is with a set of containers running.

[TedSpinks]

@bradrich what does your memory look like after you stop those containers?

[bradrich]

@TedSpinks If there are no containers running, it is a little better, but not perfect.

[ylor]

I had to give up and create a Linux VM inside of Parallels to be my Docker host. It uses less energy and memory than Docker. As a bonus it allows for --net=host.

It's an unfortunate state of affairs, but until Docker for Mac improves it's honestly the best way to go.

[marekpizner]

Me too
OS: macOS Mojave (10.14)
Docker 18.06.1-ce-mac73 (26764)
EEAD09C8-BC82-4E6E-993E-F4BBB220B93B/20181104225806

[florentin]

Same issue here, Docker Version 18.06.1-ce-mac73 (26764) on Mohave 10.14 (18A391)
com.docker.hyperkit takes 3.69GiB without any container running
Docker settings: swap 512MiB, CPUS: 4, Memory: 6GiB
Increasing the swap value automatically increases the hyperkit's memory
For a swap setting of 1.0 GiB the hyperkit takes 4.69 GiB of system's memory.

For the Edge version (Version 2.0.0.0-beta1-mac75 (27117)
hyperkit: 2.91 GiB for the Docker settings: 4 cpus, 6GiB ram and 1GiB swap

[markomitranic]

Headless, still some weird quirks here and there, but very usable and no issues with memory etc.

Yeah, same here. To automate protected ports forwarding i use:

sudo ifconfig lo0 127.0.0.1 alias
echo "rdr pass on lo0 inet proto tcp from any to 127.0.0.1 port 443 -> 127.0.0.1 port ${HOST_HTTPS_PORT}" | sudo pfctl -ef -
# cleanup later
sudo pfctl -f /etc/pf.conf

And decoupling as much volumes as possible, using virtual unbound ones for cache and tmp, and making others (for example nginx-php relation) read only. This minimal conf really helped calm down the CPU even on d4m. (again, for our d4m friends: this is not a solution just a good practice which helps!)

[markpradhan]

Hmm, regarding I/O speed issue.
I think moving as many files possible into the VM will help alot.
With syncing tools or just work from within the VM.

Or just install Linux, haha.

[stephen-turner]

@markpradhan Thanks, I've added the link to the Google doc to my comment at the top.

[stephen-turner]

FYI We have noticed in #4229 that there is a change of behaviour in Desktop 2.2, caused by a newer Linux kernel. It seems that the memory management in the kernel has changed such that the amount of memory reserved now rises more quickly at startup time. This makes the symptom that people are worried about in this thread occur more often. It will still stay within the amount configured in the Preferences though, and will still be released if the system comes under memory pressure.

[roysG]

Docker You Fucked Up MY Mac!!!

[iurynogueira]

Don't have conditions.. My memory going to space.

[andersonmalheiro]

Help me to help you my friends

[rokinsky]

I'm having the same issue

[behrangsa]

Yeap. It is consuming 3.83GB here, even after stopping all the containers.

[chrisharrisonkiwi]

100% this is getting worse.

[seagle0128]

No one really cares about this issue although it's getting worse...

[Eraldo]

Any known workarounds other than using non-hyperkit virtualbox?

[ismith78]

Downgrading to 2.1.0.5 seems to have resulted in a more stable docker environment for me. Link to the DMG can be found at https://docs.docker.com/docker-for-mac/release-notes/

[ilijaplusx]

Is this a new record?

After restart it dropped to "mere" 8.3GB. Nothing is running...

[shahrukh-alizai]

OS: macOS Catalina 10.15.5
Docker: 2.3.0.3 (45519)
Channel: Stable
Engine: 19.03.8

No containers running. Swap size 512 MB.

Memory usage: 8.33 GB

Any solution?

[fronbasal]

I have the same issue with com.docker.hyperkit consuming well beyond 5 gigabytes of memory.

[DeadPackets]

Why is this issue considered closed if it's still affecting so many users?

Brand new MacBook, macOS Catalina (10.15.5) with the latest Docker version (2.3.0.3) and on boot with no containers running com.docker.hyperkit allocated 5GB of RAM.

[roysG]

Welcome to club

…

On Thu, 23 Jul 2020, 13:47 DeadPackets ***@***.***> wrote: Why is this issue considered closed if it's still affecting so many users? Brand new MacBook, macOS Catalina (10.15.5) with the latest Docker version (2.3.0.3) and on boot with no containers running com.docker.hyperkit allocated 5GB of RAM. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#3232 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABXZDOX5GTPZFOSO5AIGQBLR5AILJANCNFSM4FXBF2GA> .

[samuelfaj]

Same here.

[olujedai]

This isn't a bug in docker per se. Based on the comment here which references the investigation here, the bug is in the Mac OS:

• com.docker.hyperkit isn't actually allocated the RAM the Memory column of the Activity Monitor.
• The actual memory consumed by com.docker.hyperkit can be found by adding the Real Memory column to the Activity Monitor view.

[markpradhan]

@olujedai although there is no memory leakage other apps start reacting weirdly.
So it still messes up your work environment.

I don't actually know this for a fact, but other apps might react to the reported memory usage, resulting in overall poor performance of your device.

I think the devs at docker should keep investigating this and/or push Apple to fix their bug.

[rliffredo]

You might not want to call it "memory leak", and instead you can call it "design flaw".
Names are not the point. The point is that when using docker, you might very often get to a point where your Mac is unusable, even when no container is running.
The only option is to shut down docker. Then, yes, memory goes back to the system. But docker is not running.

Everybody has read that analysis. It's also well written. Unfortunately, it does not stop reality to creep in, and making life miserable for people trying to use docker on mac.
And no, the bug they reported (which is just about data visualization) cannot explain the behavior that many people are experiencing -- so please stop blaming Apple.

[roysG]

Serious bug, you have to fix it!

…

On Thu, 30 Jul 2020, 18:22 Roberto Liffredo ***@***.***> wrote: You might not want to call it "memory leak", and instead you can call it "design flaw". Names are not the point. The point is that when using docker, you might very often get to a point where your Mac is unusable, even when no container is running. The only option is to shut down docker. Then, yes, memory goes back to the system. But docker is not running. Everybody has read that analysis. It's also well written. Unfortunately, it does not stop reality to creep in, and making life miserable for people trying to use docker on mac. And no, the bug they reported (which is just about data visualization) cannot explain the behavior that many people are experiencing -- so please stop blaming Apple. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#3232 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABXZDOQEPXYVH5SQIQUD7CDR6GF47ANCNFSM4FXBF2GA> .

[roysG]

I can't use my Mac with docker, I have MacBook pro 2019, the mac works very slow

…

On Thu, 30 Jul 2020, 18:58 Liroy Sabah ***@***.***> wrote: Serious bug, you have to fix it! On Thu, 30 Jul 2020, 18:22 Roberto Liffredo ***@***.***> wrote: > You might not want to call it "memory leak", and instead you can call it > "design flaw". > Names are not the point. The point is that when using docker, you might > very often get to a point where your Mac is unusable, even when no > container is running. > The only option is to shut down docker. Then, yes, memory goes back to > the system. But docker is not running. > > Everybody has read that analysis. It's also well written. Unfortunately, > it does not stop reality to creep in, and making life miserable for people > trying to use docker on mac. > And no, the bug they reported (which is just about data visualization) > cannot explain the behavior that many people are experiencing -- so please > stop blaming Apple. > > — > You are receiving this because you commented. > Reply to this email directly, view it on GitHub > <#3232 (comment)>, > or unsubscribe > <https://github.com/notifications/unsubscribe-auth/ABXZDOQEPXYVH5SQIQUD7CDR6GF47ANCNFSM4FXBF2GA> > . >

[notfalsedev]

Got a 2020 Macbook PRO with a i9 and 32GB of ram. My mac is getting hot hot hot! 200%+ CPU and lots of memory used, and no containers running. Please fix this!

[docker-robott]

Closed issues are locked after 30 days of inactivity.
This helps our team focus on active issues.

If you have found a problem that seems similar to this, please open a new issue.

Send feedback to Docker Community Slack channels #docker-for-mac or #docker-for-windows.
/lifecycle locked