Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add mimalloc memory allocator #90815

Closed
tiran opened this issue Feb 6, 2022 · 35 comments
Closed

Add mimalloc memory allocator #90815

tiran opened this issue Feb 6, 2022 · 35 comments
Labels
3.12 bugs and security fixes interpreter-core (Objects, Python, Grammar, and Parser dirs) type-feature A feature request or enhancement

Comments

@tiran
Copy link
Member

tiran commented Feb 6, 2022

BPO 46657
Nosy @nascheme, @tiran, @corona10, @erlend-aasland, @h-vetinari
PRs
  • gh-90815: Add mimalloc memory allocator #31164
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = None
    created_at = <Date 2022-02-06.14:49:12.884>
    labels = ['interpreter-core', 'type-feature', '3.11']
    title = 'Add mimalloc memory allocator'
    updated_at = <Date 2022-03-24.02:10:23.628>
    user = 'https://github.com/tiran'

    bugs.python.org fields:

    activity = <Date 2022-03-24.02:10:23.628>
    actor = 'h-vetinari'
    assignee = 'none'
    closed = False
    closed_date = None
    closer = None
    components = ['Interpreter Core']
    creation = <Date 2022-02-06.14:49:12.884>
    creator = 'christian.heimes'
    dependencies = []
    files = []
    hgrepos = []
    issue_num = 46657
    keywords = ['patch']
    message_count = 12.0
    messages = ['412639', '412641', '412645', '412654', '412669', '412679', '412741', '412749', '412796', '412835', '412867', '412986']
    nosy_count = 5.0
    nosy_names = ['nascheme', 'christian.heimes', 'corona10', 'erlendaasland', 'h-vetinari']
    pr_nums = ['31164']
    priority = 'normal'
    resolution = None
    stage = 'patch review'
    status = 'open'
    superseder = None
    type = 'enhancement'
    url = 'https://bugs.python.org/issue46657'
    versions = ['Python 3.11']

    Linked PRs

    @tiran
    Copy link
    Member Author

    tiran commented Feb 6, 2022

    From https://github.com/microsoft/mimalloc

    mimalloc (pronounced "me-malloc") is a general purpose allocator with excellent performance characteristics. Initially developed by Daan Leijen for the run-time systems of the Koka and Lean languages.

    mimalloc has several interesting properties that make it useful for CPython. Amongst other it is fast, thread-safe, and NUMA-aware. It has built-in free lists with multi-sharding and allocation heaps. While Python's obmalloc requires the GIL to protect its data structures, mimalloc uses mostly thread-local and atomic instructions (compare-and-swap) for efficiency. Sam Gross' nogil relies on mimalloc's thread safety and uses first-class heaps for heap walking GC.

    mimalloc works on majority of platforms and CPU architectures. However it requires a compiler with C11 atomics support. CentOS 7's default GCC is slightly too old, more recent GCC from Developer Toolset is required.

    For 3.11 I plan to integrate mimalloc as an optional drop-in replacement for obmalloc. Users will be able to compile CPython without mimalloc or disable mimalloc with PYTHONMALLOC env var. Since mimalloc will be optional in 3.11, Python won't depend or expose on any of the advanced features yet. The approach enables the community to test and give feedback with minimal risk of breakage.

    mimalloc sources will vendored without any option to use system libraries. Python's mimalloc requires several non-standard compile-time flags. In the future Python may extend or modify mimalloc for heap walking and nogil, too.

    (This is a tracking bug until I find time to finish a PEP.)

    @tiran tiran added 3.11 only security fixes interpreter-core (Objects, Python, Grammar, and Parser dirs) type-feature A feature request or enhancement labels Feb 6, 2022
    @corona10
    Copy link
    Member

    corona10 commented Feb 6, 2022

    I add Neil to the nosy list since he is one of the kick-off members with this amazing works :)

    @tiran
    Copy link
    Member Author

    tiran commented Feb 6, 2022

    New features:

    • vendored mimalloc 2.0.3 + two patches from mimalloc dev branch. Mimalloc is embedded in obmalloc.o. Symbols are either hidden or names are mangled to have a _Py_ prefix.
    • ./configure --with[out]-mimalloc (default: yes), fails if atomics are not available.
    • PYTHONMALLOC=mimalloc, PYTHONMALLOC=mimalloc-debug env var settings
    • PYMEM_ALLOCATOR_MIMALLOC, PYMEM_ALLOCATOR_MIMALLOC_DEBUG
    • sys.debugmallocstats() and _PyObject_DebugMallocStats() prints mimalloc stats
    • sys._malloc_info struct, contains information about available and current allocator

    @tiran
    Copy link
    Member Author

    tiran commented Feb 6, 2022

    Buildbots "PPC64 Fedora PR" and all RHEL 7 build bots provided by David Edelsohn are failing because compiler is missing support for stdatomic.h.

    @nascheme
    Copy link
    Member

    nascheme commented Feb 6, 2022

    Thanks, I'm indeed interested. Most credit goes to Christian for advancing this.

    For the missing stdatomic.h, would it be appropriate to have an autoconfig check for it? Can just disable mimalloc if it doesn't exist.

    @tiran
    Copy link
    Member Author

    tiran commented Feb 6, 2022

    We have an autoconf check for stdatomic.h. The test even verifies that a program with atomic_load_explicit() compiles and links.

    How do we want to use mimalloc in the future? Is it going to stay optional in 3.12? Then the default setting for --with-mimalloc should depend on presence of stdatomic.h. Do we want to make it mandatory for GC heap walking and nogil? Then --with-mimalloc should default to "yes" and configure should abort when stdatomic.h is missing.

    I'm leaning towards --with-mimalloc=yes. It will make users aware that they need a compiler with atomics:

    configure: error: --with-mimalloc requires stdatomic.h. Update your compiler or rebuild with --without-mimalloc. Python 3.12 will require stdatomic.

    @tiran
    Copy link
    Member Author

    tiran commented Feb 7, 2022

    ICC might be a problem. Apparently some version have an incomplete stdatomic.h, see bpo-37415.

    @tiran
    Copy link
    Member Author

    tiran commented Feb 7, 2022

    @nascheme
    Copy link
    Member

    nascheme commented Feb 7, 2022

    My preference would be for --with-mimalloc=yes in an upcoming release. For platforms without the required stdatomic.h stuff, they can manually specify --with-mimalloc=no. That will make them aware that a future release of Python might no longer build (if mimalloc is no longer optional).

    A soft-landing for merging nogil is not a good enough reason to merge mimalloc, IMHO. nogil may never be merged. There should be some concrete and immediate advantage to switch to mimalloc. The idea of using the "heap walking" to improve is cyclic GC is not concrete enough. It's just an idea at this point.

    I think the (small) performance win could be enough of a reason to merge. This seems to be the most recent benchmark:

    https://gist.github.com/pablogsal/8027937b71cd30f17aaaa5ef7c885d3e

    There is also the long-term maintenance issue. So far, mimalloc upstream has been responsive. The mimalloc code is not so huge or complicated that we couldn't maintain it (if for some reason it gets abandoned upstream). However, I think we would prefer to maintain obmalloc rather than mimalloc, all else being equal. Abandonment by the upstream seems fairly unlikely. So, I'm not too concerned about maintenance.

    @tiran
    Copy link
    Member Author

    tiran commented Feb 8, 2022

    New benchmark:

    Benchmark 2022-02-08_11-54-master-69e10976b2e7 2022-02-08_11-57-master-d6f5f010b586
    mako 8.85 ms 7.83 ms: 1.13x faster
    hexiom 6.04 ms 5.54 ms: 1.09x faster
    spectral_norm 81.4 ms 75.2 ms: 1.08x faster
    pyflate 380 ms 352 ms: 1.08x faster
    scimark_sparse_mat_mult 4.05 ms 3.76 ms: 1.08x faster
    pickle_pure_python 312 us 290 us: 1.07x faster
    unpickle_pure_python 238 us 222 us: 1.07x faster
    float 63.1 ms 59.5 ms: 1.06x faster
    tornado_http 90.3 ms 86.0 ms: 1.05x faster
    html5lib 62.8 ms 60.2 ms: 1.04x faster
    regex_compile 121 ms 116 ms: 1.04x faster
    scimark_lu 106 ms 102 ms: 1.04x faster
    nqueens 70.9 ms 68.4 ms: 1.04x faster
    crypto_pyaes 70.1 ms 67.8 ms: 1.03x faster
    logging_silent 97.5 ns 94.4 ns: 1.03x faster
    sympy_integrate 17.2 ms 16.7 ms: 1.03x faster
    sympy_str 260 ms 252 ms: 1.03x faster
    sympy_expand 441 ms 427 ms: 1.03x faster
    pathlib 14.1 ms 13.7 ms: 1.03x faster
    regex_dna 164 ms 159 ms: 1.03x faster
    regex_v8 21.1 ms 20.6 ms: 1.02x faster
    sympy_sum 138 ms 136 ms: 1.02x faster
    scimark_fft 286 ms 281 ms: 1.02x faster
    pickle 9.34 us 9.19 us: 1.02x faster
    xml_etree_parse 126 ms 124 ms: 1.01x faster
    richards 43.0 ms 42.4 ms: 1.01x faster
    xml_etree_generate 71.2 ms 70.5 ms: 1.01x faster
    scimark_monte_carlo 58.8 ms 58.3 ms: 1.01x faster
    deltablue 3.60 ms 3.58 ms: 1.01x faster
    chaos 64.6 ms 64.3 ms: 1.01x faster
    2to3 216 ms 215 ms: 1.00x faster
    pidigits 155 ms 154 ms: 1.00x faster
    nbody 76.4 ms 77.0 ms: 1.01x slower
    python_startup_no_site 3.96 ms 3.99 ms: 1.01x slower
    xml_etree_iterparse 82.5 ms 83.1 ms: 1.01x slower
    scimark_sor 103 ms 104 ms: 1.01x slower
    unpickle 11.3 us 11.4 us: 1.01x slower
    telco 5.53 ms 5.58 ms: 1.01x slower
    python_startup 5.56 ms 5.62 ms: 1.01x slower
    json_loads 20.6 us 20.8 us: 1.01x slower
    json_dumps 9.61 ms 9.77 ms: 1.02x slower
    dulwich_log 60.9 ms 62.1 ms: 1.02x slower
    logging_format 5.47 us 5.62 us: 1.03x slower
    pickle_list 3.06 us 3.15 us: 1.03x slower
    django_template 30.2 ms 31.2 ms: 1.03x slower
    meteor_contest 80.7 ms 84.1 ms: 1.04x slower
    pickle_dict 21.9 us 23.4 us: 1.07x slower
    logging_simple 4.84 us 5.20 us: 1.07x slower
    Geometric mean (ref) 1.01x faster

    Benchmark hidden because not significant (9): unpack_sequence, go, raytrace, chameleon, xml_etree_process, fannkuch, sqlite_synth, regex_effbot, unpickle_list

    @tiran
    Copy link
    Member Author

    tiran commented Feb 8, 2022

    I re-ran the benchmark of d6f5f010b586:

    Benchmark 2022-02-08_11-54-master-69e10976b2e7 2022-02-08_11-57-master-d6f5f010b586
    pickle_pure_python 312 us 281 us: 1.11x faster
    unpickle_pure_python 238 us 216 us: 1.10x faster
    pyflate 380 ms 349 ms: 1.09x faster
    hexiom 6.04 ms 5.55 ms: 1.09x faster
    logging_silent 97.5 ns 89.8 ns: 1.09x faster
    float 63.1 ms 59.3 ms: 1.07x faster
    html5lib 62.8 ms 59.1 ms: 1.06x faster
    crypto_pyaes 70.1 ms 66.1 ms: 1.06x faster
    json_loads 20.6 us 19.4 us: 1.06x faster
    tornado_http 90.3 ms 86.1 ms: 1.05x faster
    mako 8.85 ms 8.45 ms: 1.05x faster
    richards 43.0 ms 41.1 ms: 1.05x faster
    xml_etree_parse 126 ms 120 ms: 1.05x faster
    logging_format 5.47 us 5.25 us: 1.04x faster
    sympy_integrate 17.2 ms 16.5 ms: 1.04x faster
    sympy_str 260 ms 251 ms: 1.04x faster
    fannkuch 325 ms 314 ms: 1.04x faster
    regex_v8 21.1 ms 20.4 ms: 1.04x faster
    sympy_expand 441 ms 425 ms: 1.04x faster
    regex_compile 121 ms 117 ms: 1.03x faster
    sympy_sum 138 ms 134 ms: 1.03x faster
    scimark_lu 106 ms 103 ms: 1.03x faster
    go 128 ms 125 ms: 1.03x faster
    pathlib 14.1 ms 13.7 ms: 1.02x faster
    scimark_monte_carlo 58.8 ms 57.9 ms: 1.02x faster
    nqueens 70.9 ms 69.9 ms: 1.02x faster
    pidigits 155 ms 153 ms: 1.01x faster
    pickle 9.34 us 9.22 us: 1.01x faster
    raytrace 278 ms 275 ms: 1.01x faster
    2to3 216 ms 213 ms: 1.01x faster
    deltablue 3.60 ms 3.56 ms: 1.01x faster
    logging_simple 4.84 us 4.78 us: 1.01x faster
    xml_etree_iterparse 82.5 ms 81.7 ms: 1.01x faster
    regex_dna 164 ms 162 ms: 1.01x faster
    unpack_sequence 32.7 ns 32.4 ns: 1.01x faster
    telco 5.53 ms 5.48 ms: 1.01x faster
    python_startup 5.56 ms 5.58 ms: 1.00x slower
    xml_etree_generate 71.2 ms 71.6 ms: 1.01x slower
    unpickle_list 4.08 us 4.12 us: 1.01x slower
    chameleon 6.07 ms 6.14 ms: 1.01x slower
    chaos 64.6 ms 65.3 ms: 1.01x slower
    json_dumps 9.61 ms 9.75 ms: 1.01x slower
    xml_etree_process 49.9 ms 50.7 ms: 1.01x slower
    meteor_contest 80.7 ms 82.0 ms: 1.02x slower
    scimark_sparse_mat_mult 4.05 ms 4.12 ms: 1.02x slower
    unpickle 11.3 us 11.5 us: 1.02x slower
    django_template 30.2 ms 31.0 ms: 1.02x slower
    scimark_sor 103 ms 106 ms: 1.02x slower
    spectral_norm 81.4 ms 84.9 ms: 1.04x slower
    pickle_dict 21.9 us 23.5 us: 1.08x slower
    Geometric mean (ref) 1.02x faster

    Benchmark hidden because not significant (7): scimark_fft, dulwich_log, python_startup_no_site, regex_effbot, sqlite_synth, nbody, pickle_list

    @tiran
    Copy link
    Member Author

    tiran commented Feb 10, 2022

    ICC 2021 has full support for stdatomic.h and compiles mimalloc just fine:

    $ CC="icc" ./configure -C --with-pydebug
    $ make
    $ ./python
    Python 3.11.0a5+ (main, Feb  9 2022, 15:57:40) [GCC Intel(R) C++ gcc 7.5 mode] on linux
    Type "help", "copyright", "credits" or "license" for more information.
    >>> import sys
    >>> sys._malloc_info
    sys._malloc_info(allocator='mimalloc_debug', with_pymalloc=True, with_mimalloc=True, mimalloc_secure=4, mimalloc_debug=2)

    AIX xlc is still a problem. It does not support C11 stdatomic.h. But it comes with older GCC atomic memory access __sync function family, https://www.ibm.com/docs/en/xl-c-and-cpp-aix/13.1.3?topic=cbif-gcc-atomic-memory-access-built-in-functions-extension . It might be possible to re-implement mimalloc's atomics with __sync functions (e.g. https://gist.github.com/nhatminhle/5181506). The implementation would be less efficient, though. The __sync functions don't have memory order, atomic_load_explicit(v) becomes __sync_fetch_and_add(v, 0), and atomic_store_explicit() requires two full memory barriers.

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    @tiran tiran added 3.12 bugs and security fixes and removed 3.11 only security fixes labels May 18, 2022
    @ericsnowcurrently
    Copy link
    Member

    FYI, PR #31164 indicates the following 3 blockers:

    After a conversation between @tiran, @daanx (the mimalloc creator), and me, we have a good idea on how to move forward. I'm looking into the PGO issue (with some help).

    miss-islington pushed a commit to miss-islington/cpython that referenced this issue Jul 13, 2022
    …ythonGH-94790)
    
    Fixes the failure of PGO building with `mimalloc` on Windows, ensuring that `test_bpo20891` does not break profiling data (`python31*.pgc`).
    (cherry picked from commit 4a6bb30)
    
    Co-authored-by: neonene <53406459+neonene@users.noreply.github.com>
    miss-islington pushed a commit that referenced this issue Jul 13, 2022
    Fixes the failure of PGO building with `mimalloc` on Windows, ensuring that `test_bpo20891` does not break profiling data (`python31*.pgc`).
    miss-islington pushed a commit to miss-islington/cpython that referenced this issue Jul 13, 2022
    …ythonGH-94790)
    
    Fixes the failure of PGO building with `mimalloc` on Windows, ensuring that `test_bpo20891` does not break profiling data (`python31*.pgc`).
    (cherry picked from commit 4a6bb30)
    
    Co-authored-by: neonene <53406459+neonene@users.noreply.github.com>
    miss-islington added a commit that referenced this issue Jul 13, 2022
    Fixes the failure of PGO building with `mimalloc` on Windows, ensuring that `test_bpo20891` does not break profiling data (`python31*.pgc`).
    (cherry picked from commit 4a6bb30)
    
    Co-authored-by: neonene <53406459+neonene@users.noreply.github.com>
    vstinner added a commit to vstinner/cpython that referenced this issue Oct 30, 2023
    vstinner added a commit that referenced this issue Oct 30, 2023
    Include <unistd.h> to get sbrk() function.
    vstinner added a commit that referenced this issue Oct 30, 2023
    mi_atomic_load_explicit() casts 'p' argument to drop the 'const'
    qualifier on Windows arm64 platform. Fix the compiler warning:
    
        'function': different 'const' qualifiers
        (compiling source file ..\Objects\mimalloc\options.c)
    vstinner added a commit to vstinner/cpython that referenced this issue Oct 30, 2023
    @thesamesam
    Copy link
    Contributor

    thesamesam commented Oct 31, 2023

    [...]
    There is also the long-term maintenance issue. So far, mimalloc upstream has been responsive. The mimalloc code is not so huge or complicated that we couldn't maintain it (if for some reason it gets abandoned upstream). However, I think we would prefer to maintain obmalloc rather than mimalloc, all else being equal. Abandonment by the upstream seems fairly unlikely. So, I'm not too concerned about maintenance.

    I made a similar point wrt the bundling at #109914 (comment).

    DinoV added a commit that referenced this issue Oct 31, 2023
    * Don't include mimalloc .c's in Windows build
    * Fix warnings on Windows related to mimalloc
    FullteaR pushed a commit to FullteaR/cpython that referenced this issue Nov 3, 2023
    * Add mimalloc v2.12
    
    Modified src/alloc.c to remove include of alloc-override.c and not
    compile new handler.
    
    Did not include the following files:
    
     - include/mimalloc-new-delete.h
     - include/mimalloc-override.h
     - src/alloc-override-osx.c
     - src/alloc-override.c
     - src/static.c
     - src/region.c
    
    mimalloc is thread safe and shares a single heap across all runtimes,
    therefore finalization and getting global allocated blocks across all
    runtimes is different.
    
    * mimalloc: minimal changes for use in Python:
    
     - remove debug spam for freeing large allocations
     - use same bytes (0xDD) for freed allocations in CPython and mimalloc
       This is important for the test_capi debug memory tests
    
    * Don't export mimalloc symbol in libpython.
    * Enable mimalloc as Python allocator option.
    * Add mimalloc MIT license.
    * Log mimalloc in Lib/test/pythoninfo.py.
    * Document new mimalloc support.
    * Use macro defs for exports as done in:
      python#31164
    
    Co-authored-by: Sam Gross <colesbury@gmail.com>
    Co-authored-by: Christian Heimes <christian@python.org>
    Co-authored-by: Victor Stinner <vstinner@python.org>
    FullteaR pushed a commit to FullteaR/cpython that referenced this issue Nov 3, 2023
    …#111522)
    
    Don't declare _PyMem_MimallocEnabled() if WITH_PYMALLOC macro is not
    defined (./configure --without-pymalloc).
    
    Fix also a typo in _PyInterpreterState_FinalizeAllocatedBlocks().
    FullteaR pushed a commit to FullteaR/cpython that referenced this issue Nov 3, 2023
    Include <unistd.h> to get sbrk() function.
    FullteaR pushed a commit to FullteaR/cpython that referenced this issue Nov 3, 2023
    mi_atomic_load_explicit() casts 'p' argument to drop the 'const'
    qualifier on Windows arm64 platform. Fix the compiler warning:
    
        'function': different 'const' qualifiers
        (compiling source file ..\Objects\mimalloc\options.c)
    FullteaR pushed a commit to FullteaR/cpython that referenced this issue Nov 3, 2023
    …111532)
    
    * Don't include mimalloc .c's in Windows build
    * Fix warnings on Windows related to mimalloc
    @nascheme
    Copy link
    Member

    I think this issue could be closed now that mimalloc went into HEAD already.

    @erlend-aasland
    Copy link
    Contributor

    [...]
    There is also the long-term maintenance issue. So far, mimalloc upstream has been responsive. The mimalloc code is not so huge or complicated that we couldn't maintain it (if for some reason it gets abandoned upstream). However, I think we would prefer to maintain obmalloc rather than mimalloc, all else being equal. Abandonment by the upstream seems fairly unlikely. So, I'm not too concerned about maintenance.

    I made a similar point wrt the bundling at #109914 (comment).

    See follow-up issue:

    aisk pushed a commit to aisk/cpython that referenced this issue Feb 11, 2024
    * Add mimalloc v2.12
    
    Modified src/alloc.c to remove include of alloc-override.c and not
    compile new handler.
    
    Did not include the following files:
    
     - include/mimalloc-new-delete.h
     - include/mimalloc-override.h
     - src/alloc-override-osx.c
     - src/alloc-override.c
     - src/static.c
     - src/region.c
    
    mimalloc is thread safe and shares a single heap across all runtimes,
    therefore finalization and getting global allocated blocks across all
    runtimes is different.
    
    * mimalloc: minimal changes for use in Python:
    
     - remove debug spam for freeing large allocations
     - use same bytes (0xDD) for freed allocations in CPython and mimalloc
       This is important for the test_capi debug memory tests
    
    * Don't export mimalloc symbol in libpython.
    * Enable mimalloc as Python allocator option.
    * Add mimalloc MIT license.
    * Log mimalloc in Lib/test/pythoninfo.py.
    * Document new mimalloc support.
    * Use macro defs for exports as done in:
      python#31164
    
    Co-authored-by: Sam Gross <colesbury@gmail.com>
    Co-authored-by: Christian Heimes <christian@python.org>
    Co-authored-by: Victor Stinner <vstinner@python.org>
    aisk pushed a commit to aisk/cpython that referenced this issue Feb 11, 2024
    …#111522)
    
    Don't declare _PyMem_MimallocEnabled() if WITH_PYMALLOC macro is not
    defined (./configure --without-pymalloc).
    
    Fix also a typo in _PyInterpreterState_FinalizeAllocatedBlocks().
    aisk pushed a commit to aisk/cpython that referenced this issue Feb 11, 2024
    Include <unistd.h> to get sbrk() function.
    aisk pushed a commit to aisk/cpython that referenced this issue Feb 11, 2024
    mi_atomic_load_explicit() casts 'p' argument to drop the 'const'
    qualifier on Windows arm64 platform. Fix the compiler warning:
    
        'function': different 'const' qualifiers
        (compiling source file ..\Objects\mimalloc\options.c)
    aisk pushed a commit to aisk/cpython that referenced this issue Feb 11, 2024
    …111532)
    
    * Don't include mimalloc .c's in Windows build
    * Fix warnings on Windows related to mimalloc
    Glyphack pushed a commit to Glyphack/cpython that referenced this issue Sep 2, 2024
    * Add mimalloc v2.12
    
    Modified src/alloc.c to remove include of alloc-override.c and not
    compile new handler.
    
    Did not include the following files:
    
     - include/mimalloc-new-delete.h
     - include/mimalloc-override.h
     - src/alloc-override-osx.c
     - src/alloc-override.c
     - src/static.c
     - src/region.c
    
    mimalloc is thread safe and shares a single heap across all runtimes,
    therefore finalization and getting global allocated blocks across all
    runtimes is different.
    
    * mimalloc: minimal changes for use in Python:
    
     - remove debug spam for freeing large allocations
     - use same bytes (0xDD) for freed allocations in CPython and mimalloc
       This is important for the test_capi debug memory tests
    
    * Don't export mimalloc symbol in libpython.
    * Enable mimalloc as Python allocator option.
    * Add mimalloc MIT license.
    * Log mimalloc in Lib/test/pythoninfo.py.
    * Document new mimalloc support.
    * Use macro defs for exports as done in:
      python#31164
    
    Co-authored-by: Sam Gross <colesbury@gmail.com>
    Co-authored-by: Christian Heimes <christian@python.org>
    Co-authored-by: Victor Stinner <vstinner@python.org>
    Glyphack pushed a commit to Glyphack/cpython that referenced this issue Sep 2, 2024
    …#111522)
    
    Don't declare _PyMem_MimallocEnabled() if WITH_PYMALLOC macro is not
    defined (./configure --without-pymalloc).
    
    Fix also a typo in _PyInterpreterState_FinalizeAllocatedBlocks().
    Glyphack pushed a commit to Glyphack/cpython that referenced this issue Sep 2, 2024
    Include <unistd.h> to get sbrk() function.
    Glyphack pushed a commit to Glyphack/cpython that referenced this issue Sep 2, 2024
    mi_atomic_load_explicit() casts 'p' argument to drop the 'const'
    qualifier on Windows arm64 platform. Fix the compiler warning:
    
        'function': different 'const' qualifiers
        (compiling source file ..\Objects\mimalloc\options.c)
    Glyphack pushed a commit to Glyphack/cpython that referenced this issue Sep 2, 2024
    …111532)
    
    * Don't include mimalloc .c's in Windows build
    * Fix warnings on Windows related to mimalloc
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.12 bugs and security fixes interpreter-core (Objects, Python, Grammar, and Parser dirs) type-feature A feature request or enhancement
    Projects
    None yet
    Development

    No branches or pull requests

    10 participants