vm.max_map_count in docker-desktop distro for WSL2

[thespatt]

When running an elasticsearch container in the WSL2 Tech Preview, the container continually restarts itself complaining "max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]"

Would it be bad to just modify sysctl.conf in the docker-desktop distro to set vm.max_map_count = 262144? I have confirmed that modifying that is persistent across restarts.

Expected behavior

elasticsearch container should start successfully

Actual behavior

elasticsearch container continually restarts itself

Information

• Windows Version:
  Windows 10 Preview Build 19025
• Docker Desktop Version:
  2.1.6.1 edge

Steps to reproduce the behavior

DOCKERFILE:
FROM docker.elastic.co/elasticsearch/elasticsearch:6.8.4
RUN /usr/share/elasticsearch/bin/elasticsearch-plugin install analysis-phonetic

Compose:
version: '2.4'
services:
elasticsearch:
image: elastic-nwps:6
platform: linux
container_name: elasticsearch
environment:
- cluster.name=docker-cluster
- bootstrap.memory_lock=false
- "ES_JAVA_OPTS=-Xms1500m -Xmx1500m"
- http.cors.enabled=true
- http.cors.allow-origin="*"
- http.cors.allow-headers="Access-Control-Allow-Origin, X-Requested-With, Content-Type, Content-Length, Authorization"
mem_limit: 2g
restart: always
volumes:
- esdata1:/usr/share/elasticsearch/data
ports:
- 9200:9200
- 9300:9300
networks:
- esnet

[mikeparker]

@simonferquel

[gertjvr]

@thespatt is there a workaround to getting it work for now?

[simonferquel]

Hi, as modifying this sysctl setting would apply to all wsl distros (they all run in the same VM), we won't do that automatically.
However, you should be able to set this flag from your own wsl distro (using sudo sysctl -w), and it will apply to Docker as well.

[gertjvr]

for future me next time I find this issue https://www.elastic.co/guide/en/elasticsearch/reference/current/vm-max-map-count.html

[thespatt]

what I have to do on every system restart or docker desktop update:
open powershell
wsl -d docker-desktop
sysctl -w vm.max_map_count=262144

It does seem odd that there's not a way to make this sticky across restarts. @simonferquel is there a way for me to set this permanently at the VM level?

[gertjvr]

same here, setting sysctl.conf has no affect, doesn't get loaded, I am close to reverting the bashrc hacks

[gertjvr]

@thespatt think I found a possible solution could you verify it for me.

• open powershell
• wsl -d docker-desktop
• echo "vm.max_map_count = 262144" > /etc/sysctl.d/99-docker-desktop.conf

Restart docker-desktop

[simonferquel]

I am not sure about persistent sysctl configs. Maybe @benhillis has the info ?

[tomofu74]

I wrote "vm.max_map_count=262144" on docker-desktop's /etc/sysctl.d/00-alpine.conf last line.
Then I restart , that has no effect. But that line is recorded on that file.
@gertjvr Does your setting run properly?

[gertjvr]

Been working fine for me, haven't tried using existing files, created my own.

[johnsonlu]

I wrote "vm.max_map_count=262144" on docker-desktop's /etc/sysctl.d/00-alpine.conf last line.
Then I restart , that has no effect. But that line is recorded on that file.
@gertjvr Does your setting run properly?

same for me. I append the to the end of /etc/sysctl.d/99-sysctl.conf .

[CyberAP]

This fixed the issue for me:

• wsl -d docker-desktop
• echo -e "\nvm.max_map_count = 262144\n" >> /etc/sysctl.d/00-alpine.conf

[DenisValcke]

what I have to do on every system restart or docker desktop update:
open powershell
wsl -d docker-desktop
sysctl -w vm.max_map_count=262144

It does seem odd that there's not a way to make this sticky across restarts. @simonferquel is there a way for me to set this permanently at the VM level?

This seems to be the only thing that's working for me so far. I tried adding setting the oo-alpine.conf in docker-desktop tried create a new file as well. Also tried setting these things in my Ubuntu wsl. Nothing seems to work except for the above.

[simonferquel]

Note that most kernel settings (except those which can be set per-namespace) will apply both to docker desktop and your own distro. So 3 things:

• Please make sure it does not conflict with things you need in your own distros
• You can actually apply these settings with whatever automation tool in your own distro, and it will apply to Docker Desktop
• We won't add sysctl config in docker desktop, because it has side effects in every wsl distro.

[joaociocca]

even echoing the setting into 00-alpine.conf and 99-docker-desktop.conf, both, and restarting Docker didn't work for me, I still get that error...

@thespatt's solution, however, worked... but I'll have do to it manually every time I restart?

[thespatt]

@joaociocca @DenisValcke @johnsonlu finally solved it! Have to set the environment variable "discovery.type=single-node"

I had to solve it because all of the devs on my team will be switching to wsl2 soon and there's no way I can deal with people forgetting to set that.

https://twitter.com/thespatt/status/1260221113768062978
https://www.elastic.co/guide/en/elasticsearch/reference/current/docker.html#docker-cli-run-dev-mode

[joaociocca]

I'm aware of that setting, but my goal is to make a "realistic lab", so single-node is a no-no. Thanks, though! Hope it helps others where it helps <3

[simon-hofmann]

This worked for me.

Added this to my compose file:

privileged: true
    user: root
    command:
      - /bin/bash
      - -c
      - |
        sysctl -w vm.max_map_count=262144 &&
        su elasticsearch -c bin/es-docker

[asampal]

Setting this to 262144 as @CyberAP had it works for me, but the problem is that my Docker stack then ends up consuming huge amounts of memory. Before using Docker with WSL2, I had docker-desktop set to use 8GB. With WSL2 and this setting I'm seeing close to 19GB consumed. That's way more than I can spare on my 32GB system due to other required apps. My system is sluggish and apps are running out of memory.

Can this be set only for one container (in my case)? Elasticsearch is where I saw a problem.

It seems the high mem use is due to the way Linux behaves wrt to allocated memory and caching and adding a .wslconfig file allows setting some parameters to constrain that.

[thespatt]

@asampal if you also add settings you can limit elasticsearch memory per node:

  elasticsearch:
    image: docker.elastic.co/elasticsearch/elasticsearch:6.8.9
    platform: linux
    container_name: elasticsearch
    environment:
      - cluster.name=docker-cluster
      - bootstrap.memory_lock=false
      - "ES_JAVA_OPTS=-Xms1500m -Xmx1500m"
      - discovery.type=single-node
    mem_limit: 2g

[asampal]

I think I'll do that @thespatt - thanks.

[beccari]

There is a "one-shot" command line to solve it from wsl.exe

❯ wsl --shutdown # because we don't really need to restart the computer to see the config is lost ...
⚡ beccari@RAYGUN  ~                                                                                         [00:28]
❯ wsl -d docker-desktop cat /proc/sys/vm/max_map_count # current value
65530
⚡ beccari@RAYGUN  ~                                                                                         [00:28]
❯ wsl -d docker-desktop sysctl -w vm.max_map_count=262144
vm.max_map_count = 262144
⚡ beccari@RAYGUN  ~                                                                                         [00:28]
❯ wsl -d docker-desktop cat /proc/sys/vm/max_map_count 
262144
⚡ beccari@RAYGUN  ~                                                                                         [00:28]
❯

So, in to make it stick across restarts, I've created a nasty fix-sysctl.bat as follows

@echo off
wsl -d docker-desktop sysctl -w vm.max_map_count=262144 

and put it at shell:startup folder.

Far from ideal (awful), but it looks like it does the trick.

[denov]

@echo off
wsl -d docker-desktop cat /proc/sys/vm/max_map_count

cat?

and put it at shell:startup folder.

where and what is this?

[zzFluke]

@echo off
wsl -d docker-desktop cat /proc/sys/vm/max_map_count

cat?

and put it at shell:startup folder.

where and what is this?

In Windows go to Start -> Run (or Win+R) -> and type 'shell:startup' this short path to Win Startup directory -> %APPDATA%\Roaming\Microsoft\Windows\Start Menu\Programs\Startup

there create suggested by @beccari .bat file

[archon810]

The question is why is @gertjvr's solution not working?

PS C:\Users\Artem Russakovskii> wsl -d docker-desktop
Archon-SkylakeD:/tmp/docker-desktop-root/mnt/host/c/Users/Artem Russakovskii# cat /etc/sysctl.d/99-docker-desktop.conf
cat: can't open '/etc/sysctl.d/99-docker-desktop.conf': No such file or directory
Archon-SkylakeD:/tmp/docker-desktop-root/mnt/host/c/Users/Artem Russakovskii# echo "vm.max_map_count = 262144" > /etc/sysctl.d/99-docker-desktop.conf
Archon-SkylakeD:/tmp/docker-desktop-root/mnt/host/c/Users/Artem Russakovskii# cat /etc/sysctl.d/99-docker-desktop.conf
vm.max_map_count = 262144
Archon-SkylakeD:/tmp/docker-desktop-root/mnt/host/c/Users/Artem Russakovskii#
PS C:\Users\Artem Russakovskii> wsl -d docker-desktop cat /proc/sys/vm/max_map_count
65530

[shortcodes]

This worked for me even after restart. Hope it helps others

wsl -d docker-desktop
echo 262144 >> /proc/sys/vm/max_map_count

It worked for me. Hope it will work for you too.

[rabauss]

Are there any news for this topic?

[tomofu74]

No. I understand it is difficult for WSL2.
Thank you.

[jamshid]

To all the people saying their fix persists through a "restart", I think you're only testing a Docker for Desktop restart. Everybody else is confused because they're looking for a solution which survives a Windows reboot.

These solutions are equivalent, they allow an elasticsearch container to run, but the setting is not persisted after Windows reboots.

wsl -d docker-desktop sysctl -w vm.max_map_count=262144

Or

docker run --rm -ti --privileged centos sysctl vm.max_map_count=262144

Or

wsl -d docker-desktop
# echo "vm.max_map_count = 262144" > /etc/sysctl.d/99-docker-desktop.conf
#    OR
# echo 262144 >> /proc/sys/vm/max_map_count

Besides running an elasticsearch container you can test by checking this:

docker run --privileged centos sysctl -a | grep max_map_count
vm.max_map_count = 262144

Unfortunately the Elastic docs https://www.elastic.co/guide/en/elasticsearch/reference/current/docker.html#_set_vm_max_map_count_to_at_least_262144 don't mention this reboot problem.

As @simonferquel said earlier #5202 (comment):

Hi, as modifying this sysctl setting would apply to all wsl distros (they all run in the same VM), we won't do that automatically.
However, you should be able to set this flag from your own wsl distro (using sudo sysctl -w), and it will apply to Docker as well.

Simon do you know where the central wsl VM is? Is there a way to make the sysctl permanent, surviving Windows reboots? I wouldn't mind that it affects all wsl distros I run on my PC.

[Aschay]

I have the same issue but with sonarqube using elasticearch , i got [1]: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144], the only thing that worked for me was:
wsl -d docker-desktop && sysctl -w vm.max_map_count=262144

[gricn]

This worked for me.

Added this to my compose file:

privileged: true
    user: root
    command:
      - /bin/bash
      - -c
      - |
        sysctl -w vm.max_map_count=262144 &&
        su elasticsearch -c bin/es-docker

Your solution works perfect after changing su elasticsearch -c bin/es-docker to su elasticsearch -c bin/elasticsearch

Here is one part of code in my docker-compose.yml

version: '2.2'
services:
  elasticsearch:
    image: elasticsearch:6.8.14
    privileged: true
    user: root
    command:
      - /bin/bash
      - -c
      - |
        sysctl -w vm.max_map_count=262144 &&
        su elasticsearch -c bin/elasticsearch

[viceice]

privileged: true is a nogo, so not a solution 😕

so using wsl -d docker-desktop sysctl -w vm.max_map_count=262144 as a workaround

[gricn]

privileged: true is a nogo, so not a solution 😕

so using wsl -d docker-desktop sysctl -w vm.max_map_count=262144 as a workaround

So how to config docker-compose.yml, I need to run this script in cmd after every time I restart my computer?

[PS6433]

• bootstrap.memory_lock=false
  - "ES_JAVA_OPTS=-Xms1500m -Xmx1500m"
  - discovery.type=single-node
  mem_limit: 2g

Thanks a lot! this worked perfectly!

[docker-robott]

Issues go stale after 90 days of inactivity.
Mark the issue as fresh with /remove-lifecycle stale comment.
Stale issues will be closed after an additional 30 days of inactivity.

Prevent issues from auto-closing with an /lifecycle frozen comment.

If this issue is safe to close now please do so.

Send feedback to Docker Community Slack channels #docker-for-mac or #docker-for-windows.
/lifecycle stale

[jamshid]

/lifecycle frozen

[koll051]

I have the same issue but with sonarqube using elasticearch , i got [1]: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144], the only thing that worked for me was:
wsl -d docker-desktop && sysctl -w vm.max_map_count=262144

Same problem here. I found a nice solution automating the sysctl -w vm.max_map_count=262144 (and thus persistent across Docker restarts) with docker-compose here: https://community.sonarsource.com/t/docker-install-elastic-search-vm-max-map-count/37270/2 . Synopsis: let a priviledged bash init container do the sysctl -w vm.max_map_count=262144 before every start of SonarQube / ElasticSearch.

[rayjaymor85]

/remove-lifecycle stale

[RaulSokolova]

I tried all the approaches and non worked for me... However I was able to set it permanently like this.

1. Create a bat file for example (set_vm_max.bat)
2. Write the command inside the file "wsl -d docker-desktop sysctl -w vm.max_map_count=262144"
3. Now will have to copy the file to the startup location, press "Windows + R"
4. Type "shell:startup" and press Enter.
5. Copy the file you just created to start location.

To verify it works:

1. Reboot
2. wsl -d docker-desktop
3. cat /proc/sys/vm/max_map_count

Hope it works for you.

[SamLebarbare]

Hi, we finnaly found the trick via the .wslconfig file :
https://stackoverflow.com/questions/69214301/using-docker-desktop-for-windows-how-can-sysctl-parameters-be-configured-to-per/69294687#69294687

[thinkalikenl]

Follow the steps in https://techbrij.com/wsl-2-ubuntu-services-windows-10-startup. Change the contents of /etc/init-wsl into:

#!/bin/sh
sudo sysctl -w vm.max_map_count=262144

That worked fine as a solution for me after a windows reboot and starting wsl.

[kiview]

Is there any chance that WSL2 and Docker Desktop somehow changed the default values? I can't observe this issue anymore on my machine.

[lsmith-dotnetfurther]

Hi, we finnaly found the trick via the .wslconfig file : https://stackoverflow.com/questions/69214301/using-docker-desktop-for-windows-how-can-sysctl-parameters-be-configured-to-per/69294687#69294687

I recently ran into all the above problems, especially with restart, and the best solution was the one referenced above.

[rmukaila]

thanks man!

[gabrieledarrigo]

Hi, we finnaly found the trick via the .wslconfig file : https://stackoverflow.com/questions/69214301/using-docker-desktop-for-windows-how-can-sysctl-parameters-be-configured-to-per/69294687#69294687

I recently ran into all the above problems, especially with restart, and the best solution was the one referenced above.

Thank you, man 👍