mmap fails at size 8GB

[jianchun]

Tried on my box and the maximum successful size was 4GB. mmap returns -1 at 8GB.

#include <stdio.h>
#include <sys/mman.h>

int main(int argc, char* argv[]) {
  size_t MemSize = 0x200000000; // maximum successful size is 0x100000000
  int mmapFlags = 0;
  int mmapFile = -1;
  off_t mmapOffset = 0;

  mmapFlags |= MAP_ANON | MAP_PRIVATE;

  void* p = mmap(NULL, MemSize, PROT_NONE,
                 mmapFlags, mmapFile, mmapOffset);

  printf("mmap: %p\n", p);
  return 0;
}

[benhillis]

Thanks for reporting this. By default native Linux will not allowing you to map an address range larger than your swap file size. In WSL we report the size of the swap file as the size of the NT page file (check the output of top or htop to see how big your swap file is). You should be able to get around this by increasing your NT page file size.

On native Linux this behavior is configurable by setting /proc/sys/vm/overcommit_memory to 1, but that ProcFs file is not currently exposed in WSL.

[jianchun]

@benhillis Thanks for your explanation! Out of curiosity I did some more experiments. Does following result look normal? Native Ubuntu (running in hyper-V VM) succeeds up to 64TB. Doesn't look like 50 x anything?

On WSL, max mmap size for above test -- 0x100000000ul * 1.

top - 15:43:14 up 1 min,  0 users,  load average: 0.52, 0.58, 0.59
Tasks:   3 total,   1 running,   2 sleeping,   0 stopped,   0 zombie
%Cpu(s):  2.8 us, 48.9 sy,  0.0 ni, 48.1 id,  0.0 wa,  0.2 hi,  0.0 si,  0.0 st
KiB Mem:  16524180 total,  9800980 used,  6723200 free,    34032 buffers
KiB Swap:  2490368 total,    30812 used,  2459556 free.   188576 cached Mem

On Native Ubuntu, max mmap size for above test -- 0x100000000ul * 1024 * 16.

top - 15:43:11 up 2 days, 20:21,  1 user,  load average: 3.15, 4.02, 4.32
Tasks: 235 total,   1 running, 233 sleeping,   0 stopped,   1 zombie
%Cpu(s): 24.7 us,  4.3 sy,  1.2 ni, 69.7 id,  0.0 wa,  0.0 hi,  0.1 si,  0.0 st
KiB Mem :  4038040 total,  1120808 free,   753412 used,  2163820 buff/cache
KiB Swap:  2095100 total,  1477388 free,   617712 used.  2968824 avail Mem 

$ cat /proc/sys/vm/overcommit_memory 
0
$ cat /proc/sys/vm/overcommit_ratio 
50

[benhillis]

Native Ubuntu might do something strange with PROT_NONE. Could you change that to PROT_READ | PROT_WRITE and try the same experiment?

[jianchun]

@benhillis Yes, you are right -- With "PROT_READ | PROT_WRITE" Native Ubuntu can only do 0x100000000ul * 1 too.

[benhillis]

Support for this has vastly improved in Creators Update.

[obastemur]

Latest insider (updated today) this issue appears again.

[benhillis]

@obastemur - What is your output of ver.exe? There was an additional fix that did not make creators update but reached Windows Insiders in build 16215.