Skip to content

too much mem cost when create a lot of thread? #1107

Open
@MagicGeeker

Description

@MagicGeeker

in windows 10,create 10000 threads and each alloc 1000B,use too much memory.

#include "mimalloc.h"
#include <iostream>
#include <memory>
#include <thread>

using namespace std;

#if 0
#define my_malloc malloc
#define my_free free
#define my_realloc realloc
#else
#define my_malloc mi_malloc
#define my_free mi_free
#define my_realloc mi_realloc
#endif

const int N = 10 * 1000;
void *ptrs[N];

int main() {
  for (int i = 0; i < N; i++) {
    std::thread([i]() {
      ptrs[i] = my_malloc(1000);
      memset(ptrs[i], 0x11, 1000);
      std::this_thread::sleep_for(chrono::seconds(30));
      my_free(ptrs[i]);
    }).detach();
  }
  getchar();
}

malloc test:

Image

mimalloc test:

Image

if set N bigger,the program will crash:

PS D:\Codes\CMake\mimalloc_test\build> .\mimalloc_test.exe
mimalloc: warning: thread 0xa484d66000: commit error: start: 000002ED0C000000, csize: 0x200000, err: 1455
mimalloc: warning: thread 0xa535fbe000: mimalloc: warning: thread 0xa535fbc000: commit error: start: 000002ED10000000, csize: 0x200000, err: 1455
mimalloc: warning: thread 0xa535fc2000: commit error: start: 000002ED1C000000, csize: 0x200000, err: 1455
commit error: start: 000002ED14000000, csize: 0x200000, err: 1455
mimalloc: assertion failed: at "D:\Codes\CMake\mimalloc_test\mimalloc\src\os.c":952, mi_os_commitx
  assertion: "err == 0"
mimalloc: warning: thread 0xa484d6e000: commit error: start: 000002ED20000000, csize: 0x200000, err: 1455
mimalloc: assertion failed: at "D:\Codes\CMake\mimalloc_test\mimalloc\src\os.c":952, mi_os_commitx
  assertion: "err == 0"
mimalloc: assertion failed: at "D:\Codes\CMake\mimalloc_test\mimalloc\src\os.c":952, mi_os_commitx
  assertion: "err == 0"
mimalloc: warning: thread 0xa535fc8000: commit error: start: 000002ED28000000, csize: 0x200000, err: 1455
mimalloc: warning: thread 0xa535fd4000: commit error: start: 000002ED40000000, csize: 0x200000, err: 1455
mimalloc: warning: thread 0xa535fcc000: mimalloc: warning: thread 0xa606f08000: mimalloc: assertion failed: at "D:\Codes\CMake\mimalloc_test\mimalloc\src\os.c":952, mi_os_commitx
  assertion: "err == 0"
mimalloc: warning: thread 0xa535fd0000: commit error: start: 000002ED38000000, csize: 0x200000, err: 1455
mimalloc: warning: thread 0xa535fd2000: commit error: start: 000002ED3C000000, csize: 0x200000, err: 1455
mimalloc: warning: thread 0xa535fc0000: commit error: start: 000002ED18000000, csize: 0x200000, err: 1455
mimalloc: assertion failed: at "D:\Codes\CMake\mimalloc_test\mimalloc\src\os.c":952, mi_os_commitx
  assertion: "err == 0"
mimalloc: warning: thread 0xa535fd6000: mimalloc: assertion failed: at "D:\Codes\CMake\mimalloc_test\mimalloc\src\os.c":952, mi_os_commitx
  assertion: "err == 0"
mimalloc: assertion failed: at "D:\Codes\CMake\mimalloc_test\mimalloc\src\os.c":952, mi_os_commitx
  assertion: "err == 0"
mimalloc: warning: thread 0xa535fda000: commit error: start: 000002ED4C000000, csize: 0x200000, err: 1455
mimalloc: assertion failed: at "D:\Codes\CMake\mimalloc_test\mimalloc\src\os.c":952, mi_os_commitx
  assertion: "err == 0"
commit error: start: 000002ED34000000, csize: 0x200000, err: 1455
mimalloc: assertion failed: at "D:\Codes\CMake\mimalloc_test\mimalloc\src\os.c":952, mi_os_commitx
  assertion: "err == 0"
mimalloc: assertion failed: at "D:\Codes\CMake\mimalloc_test\mimalloc\src\os.c":952, mi_os_commitx
  assertion: "err == 0"
mimalloc: warning: thread 0xa606efc000: mimalloc: assertion failed: at "D:\Codes\CMake\mimalloc_test\mimalloc\src\os.c":952, mi_os_commitx
  assertion: "err == 0"
mimalloc: assertion failed: at "D:\Codes\CMake\mimalloc_test\mimalloc\src\os.c":952, mi_os_commitx
  assertion: "err == 0"
mimalloc: assertion failed: at "D:\Codes\CMake\mimalloc_test\mimalloc\src\os.c":952, mi_os_commitx
  assertion: "err == 0"
mimalloc: assertion failed: at "D:\Codes\CMake\mimalloc_test\mimalloc\src\os.c":952, mi_os_commitx
  assertion: "err == 0"
mimalloc: assertion failed: at "D:\Codes\CMake\mimalloc_test\mimalloc\src\os.c":952, mi_os_commitx
  assertion: "err == 0"
mimalloc: assertion failed: at "D:\Codes\CMake\mimalloc_test\mimalloc\src\os.c":952, mi_os_commitx
  assertion: "err == 0"
mimalloc: assertion failed: at "D:\Codes\CMake\mimalloc_test\mimalloc\src\os.c":952, mi_os_commitx
  assertion: "err == 0"
mimalloc: warning: thread 0xa535fca000: commit error: start: 000002ED2C000000, csize: 0x200000, err: 1455
commit error: start: 000002ED44000000, csize: 0x200000, err: 1455
mimalloc: warning: thread 0xa535fd8000: commit error: start: 000002ED48000000, csize: 0x200000, err: 1455
mimalloc: assertion failed: at "D:\Codes\CMake\mimalloc_test\mimalloc\src\os.c":952, mi_os_commitx
  assertion: "err == 0"
commit error: start: 000002ED30000000, csize: 0x200000, err: 1455
mimalloc: assertion failed: at "D:\Codes\CMake\mimalloc_test\mimalloc\src\os.c":952, mi_os_commitx
  assertion: "err == 0"
mimalloc: assertion failed: at "D:\Codes\CMake\mimalloc_test\mimalloc\src\os.c":952, mi_os_commitx
  assertion: "err == 0"
mimalloc: assertion failed: at "D:\Codes\CMake\mimalloc_test\mimalloc\src\os.c":952, mi_os_commitx
  assertion: "err == 0"
mimalloc: assertion failed: at "D:\Codes\CMake\mimalloc_test\mimalloc\src\os.c":952, mi_os_commitx
  assertion: "err == 0"
mimalloc: assertion failed: at "D:\Codes\CMake\mimalloc_test\mimalloc\src\os.c":952, mi_os_commitx
  assertion: "err == 0"
mimalloc: assertion failed: at "D:\Codes\CMake\mimalloc_test\mimalloc\src\os.c":952, mi_os_commitx
  assertion: "err == 0"
commit error: start: 000002ED24000000, csize: 0x200000, err: 1455
mimalloc: assertion failed: at "D:\Codes\CMake\mimalloc_test\mimalloc\src\os.c":952, mi_os_commitx
  assertion: "err == 0"
mimalloc: assertion failed: at "D:\Codes\CMake\mimalloc_test\mimalloc\src\os.c":952, mi_os_commitx
  assertion: "err == 0"
mimalloc: assertion failed: at "D:\Codes\CMake\mimalloc_test\mimalloc\src\os.c":952, mi_os_commitx
  assertion: "err == 0"
mimalloc: assertion failed: at "D:\Codes\CMake\mimalloc_test\mimalloc\src\os.c":952, mi_os_commitx
  assertion: "err == 0"
mimalloc: assertion failed: at "D:\Codes\CMake\mimalloc_test\mimalloc\src\os.c":952, mi_os_commitx
  assertion: "err == 0"
mimalloc: assertion failed: at "D:\Codes\CMake\mimalloc_test\mimalloc\src\os.c":952, mi_os_commitx
  assertion: "err == 0"
mimalloc: assertion failed: at "D:\Codes\CMake\mimalloc_test\mimalloc\src\os.c":952, mi_os_commitx
  assertion: "err == 0"
mimalloc: assertion failed: at "D:\Codes\CMake\mimalloc_test\mimalloc\src\os.c":952, mi_os_commitx
  assertion: "err == 0"
mimalloc: assertion failed: at "D:\Codes\CMake\mimalloc_test\mimalloc\src\os.c":952, mi_os_commitx
  assertion: "err == 0"
mimalloc: assertion failed: at "D:\Codes\CMake\mimalloc_test\mimalloc\src\os.c":952, mi_os_commitx
  assertion: "err == 0"
mimalloc: assertion failed: at "D:\Codes\CMake\mimalloc_test\mimalloc\src\os.c":952, mi_os_commitx
  assertion: "err == 0"
mimalloc: assertion failed: at "D:\Codes\CMake\mimalloc_test\mimalloc\src\os.c":952, mi_os_commitx
  assertion: "err == 0"
mimalloc: assertion failed: at "D:\Codes\CMake\mimalloc_test\mimalloc\src\os.c":952, mi_os_commitx
  assertion: "err == 0"
mimalloc: assertion failed: at "D:\Codes\CMake\mimalloc_test\mimalloc\src\os.c":952, mi_os_commitx
  assertion: "err == 0"
mimalloc: assertion failed: at "D:\Codes\CMake\mimalloc_test\mimalloc\src\os.c":952, mi_os_commitx
  assertion: "err == 0"
mimalloc: assertion failed: at "D:\Codes\CMake\mimalloc_test\mimalloc\src\os.c":952, mi_os_commitx
  assertion: "err == 0"
mimalloc: assertion failed: at "D:\Codes\CMake\mimalloc_test\mimalloc\src\os.c":952, mi_os_commitx
  assertion: "err == 0"
mimalloc: assertion failed: at "D:\Codes\CMake\mimalloc_test\mimalloc\src\os.c":952, mi_os_commitx
  assertion: "err == 0"
mimalloc: assertion failed: at "D:\Codes\CMake\mimalloc_test\mimalloc\src\os.c":952, mi_os_commitx
  assertion: "err == 0"
mimalloc: assertion failed: at "D:\Codes\CMake\mimalloc_test\mimalloc\src\os.c":952, mi_os_commitx
  assertion: "err == 0"
mimalloc: assertion failed: at "D:\Codes\CMake\mimalloc_test\mimalloc\src\os.c":952, mi_os_commitx
  assertion: "err == 0"
mimalloc: assertion failed: at "D:\Codes\CMake\mimalloc_test\mimalloc\src\os.c":952, mi_os_commitx
  assertion: "err == 0"
mimalloc: error: thread 0xa535e1a000: unable to allocate thread local heap metadata (6968 bytes)
mimalloc: assertion failed: at "D:\Codes\CMake\mimalloc_test\mimalloc\src\os.c":952, mi_os_commitx
  assertion: "err == 0"
libc++abi: terminating due to uncaught exception of type std::__1::system_error: thread constructor failed: Input/output error
PS D:\Codes\CMake\mimalloc_test\build>

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions