Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[WIP, RFC] Problem: lot of use of dynamic memory allocation, leading to performance degradation and non-determism #3911

Draft
wants to merge 58 commits into
base: master
Choose a base branch
from
Draft

[WIP, RFC] Problem: lot of use of dynamic memory allocation, leading to performance degradation and non-determism #3911

wants to merge 58 commits into from

Commits on Aug 13, 2019

  1. Implement a very simple zero-lock message pool to test performance gains 

    by completely removing malloc/free from the hot path of benchmark util
    @f18m
    f18m committed Aug 13, 2019
    Configuration menu
    Copy the full SHA
    4f84758 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'master' of https://github.com/f18m/libzmq.git

    @f18m
    f18m committed Aug 13, 2019
    Configuration menu
    Copy the full SHA
    ea0dc06 View commit details
    Browse the repository at this point in the history

  3. Allow to choose message sizes as well

    @f18m
    f18m committed Aug 13, 2019
    Configuration menu
    Copy the full SHA
    a24f2af View commit details
    Browse the repository at this point in the history

  4. Allow using env variables to do some basic overriding

    @f18m
    f18m committed Aug 13, 2019
    Configuration menu
    Copy the full SHA
    1bd2ae1 View commit details
    Browse the repository at this point in the history

  5. fix typo

    @f18m
    f18m committed Aug 13, 2019
    Configuration menu
    Copy the full SHA
    252e8d4 View commit details
    Browse the repository at this point in the history

  6. add TCP kernel socket buffer setting

    @f18m
    f18m committed Aug 13, 2019
    Configuration menu
    Copy the full SHA
    4a30795 View commit details
    Browse the repository at this point in the history

Commits on Aug 25, 2019

  1. Merge remote-tracking branch 'upstream/master'

    @f18m
    f18m committed Aug 25, 2019
    Configuration menu
    Copy the full SHA
    577232e View commit details
    Browse the repository at this point in the history

Commits on Aug 28, 2019

  1. Merge remote-tracking branch 'upstream/master'

    @f18m
    f18m committed Aug 28, 2019
    Configuration menu
    Copy the full SHA
    ff8d79f View commit details
    Browse the repository at this point in the history

  2. First implementation of global memory pool for ZMQ

    @f18m
    f18m committed Aug 28, 2019
    Configuration menu
    Copy the full SHA
    00e514e View commit details
    Browse the repository at this point in the history

  3. Remove changes related to graph generation

    @f18m
    f18m committed Aug 28, 2019
    Configuration menu
    Copy the full SHA
    18c52c4 View commit details
    Browse the repository at this point in the history

  4. allow testing up to 8k msg sizes

    @f18m
    f18m committed Aug 28, 2019
    Configuration menu
    Copy the full SHA
    a720a31 View commit details
    Browse the repository at this point in the history

Commits on Aug 30, 2019

  1. correctly deallocate memory pool blocks

    @f18m
    f18m committed Aug 30, 2019
    Configuration menu
    Copy the full SHA
    b9e1f01 View commit details
    Browse the repository at this point in the history

  2. fix build with no draft API

    @f18m
    f18m committed Aug 30, 2019
    Configuration menu
    Copy the full SHA
    1649701 View commit details
    Browse the repository at this point in the history

Commits on Aug 31, 2019

  1. never use allocator for VSM

    @f18m
    f18m committed Aug 31, 2019
    Configuration menu
    Copy the full SHA
    0baafa4 View commit details
    Browse the repository at this point in the history

Commits on May 2, 2020

  1. Merge branch 'master' of https://github.com/zeromq/libzmq into memory… 

    …pool

# Conflicts:
#	src/msg.cpp
#	src/msg.hpp
#	src/zmq.cpp
    Mark Jan van Kampen committed May 2, 2020
    Configuration menu
    Copy the full SHA
    59cbfac View commit details
    Browse the repository at this point in the history

  2. Fixes cmake build

    Mark Jan van Kampen committed May 2, 2020
    Configuration menu
    Copy the full SHA
    f0a7a7f View commit details
    Browse the repository at this point in the history

  3. Changes to base class with virtuals

    Mark Jan van Kampen committed May 2, 2020
    Configuration menu
    Copy the full SHA
    f682600 View commit details
    Browse the repository at this point in the history

  4. Makes max message size dynamic

    Mark Jan van Kampen committed May 2, 2020
    Configuration menu
    Copy the full SHA
    3a3d877 View commit details
    Browse the repository at this point in the history

  5. Dynamically grows mempool

    Mark Jan van Kampen committed May 2, 2020
    Configuration menu
    Copy the full SHA
    b416348 View commit details
    Browse the repository at this point in the history

Commits on May 3, 2020

  1. Updates dynamic global pool

    Mark Jan van Kampen committed May 3, 2020
    Configuration menu
    Copy the full SHA
    cfd4c85 View commit details
    Browse the repository at this point in the history

  2. Removes unnecessary class

    Mark Jan van Kampen committed May 3, 2020
    Configuration menu
    Copy the full SHA
    1dd2304 View commit details
    Browse the repository at this point in the history

Commits on May 4, 2020

  1. Adds new files to makefile

    Mark Jan van Kampen committed May 4, 2020
    Configuration menu
    Copy the full SHA
    d06f868 View commit details
    Browse the repository at this point in the history

  2. Adds concurrentqueue to sources

    Mark Jan van Kampen committed May 4, 2020
    Configuration menu
    Copy the full SHA
    cfa228b View commit details
    Browse the repository at this point in the history

  3. Fixes some warnings

    Mark Jan van Kampen committed May 4, 2020
    Configuration menu
    Copy the full SHA
    d96d616 View commit details
    Browse the repository at this point in the history

  4. Adds includes

    Mark Jan van Kampen committed May 4, 2020
    Configuration menu
    Copy the full SHA
    caf7798 View commit details
    Browse the repository at this point in the history

Commits on May 5, 2020

  1. Makes initial number of messages a bit more dynamic 

    In practice I observed that 256 byte messages require +- 1024 messages
pre-allocated to reach max. performance on my pc.
This scales depending on message size I believe.
    Mark Jan van Kampen committed May 5, 2020
    Configuration menu
    Copy the full SHA
    5fbc4cc View commit details
    Browse the repository at this point in the history

  2. Hides global allocator implementation and option when C++11 not avail… 

    …able
    Mark Jan van Kampen committed May 5, 2020
    Configuration menu
    Copy the full SHA
    d2c53c5 View commit details
    Browse the repository at this point in the history

  3. Fixes msvc __cplusplus reporting

    Mark Jan van Kampen committed May 5, 2020
    Configuration menu
    Copy the full SHA
    348865f View commit details
    Browse the repository at this point in the history

  4. Improves <C++11 support

    Mark Jan van Kampen committed May 5, 2020
    Configuration menu
    Copy the full SHA
    59c6a6c View commit details
    Browse the repository at this point in the history

  5. Fixes missing declaration

    Mark Jan van Kampen committed May 5, 2020
    Configuration menu
    Copy the full SHA
    1f5abc1 View commit details
    Browse the repository at this point in the history

  6. Adds more c++11 guards

    Mark Jan van Kampen committed May 5, 2020
    Configuration menu
    Copy the full SHA
    2c29abc View commit details
    Browse the repository at this point in the history

Commits on May 12, 2020

  1. Adds test and moves queue to external

    Mark Jan van Kampen committed May 12, 2020
    Configuration menu
    Copy the full SHA
    d7f9452 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'master' of https://github.com/zeromq/libzmq into memory… 

    …pool

# Conflicts:
#	CMakeLists.txt
    Mark Jan van Kampen committed May 12, 2020
    Configuration menu
    Copy the full SHA
    1450beb View commit details
    Browse the repository at this point in the history

  3. Fixes bad path

    Mark Jan van Kampen committed May 12, 2020
    Configuration menu
    Copy the full SHA
    f4973dc View commit details
    Browse the repository at this point in the history

  4. Fixes bad path

    Mark Jan van Kampen committed May 12, 2020
    Configuration menu
    Copy the full SHA
    3b9ec2f View commit details
    Browse the repository at this point in the history

  5. Fixes formatting

    Mark Jan van Kampen committed May 12, 2020
    Configuration menu
    Copy the full SHA
    a260668 View commit details
    Browse the repository at this point in the history

Commits on May 13, 2020

  1. Adds destroy to test

    Mark Jan van Kampen committed May 13, 2020
    Configuration menu
    Copy the full SHA
    7dafdf7 View commit details
    Browse the repository at this point in the history

  2. Fixes wrong increment 

    Consider switch to range based for (only since C++11 though)
    Mark Jan van Kampen committed May 13, 2020
    Configuration menu
    Copy the full SHA
    a3bfc67 View commit details
    Browse the repository at this point in the history

  3. Fixes bad c+11

    Mark Jan van Kampen committed May 13, 2020
    Configuration menu
    Copy the full SHA
    cd90418 View commit details
    Browse the repository at this point in the history

  4. Adds a basic concurrent queue

    Mark Jan van Kampen committed May 13, 2020
    Configuration menu
    Copy the full SHA
    a575335 View commit details
    Browse the repository at this point in the history

  5. Removes some debug code

    Mark Jan van Kampen committed May 13, 2020
    Configuration menu
    Copy the full SHA
    74dd371 View commit details
    Browse the repository at this point in the history

  6. Adds newline and nullptr

    Mark Jan van Kampen committed May 13, 2020
    Configuration menu
    Copy the full SHA
    e9c3a01 View commit details
    Browse the repository at this point in the history

  7. Adds a start on function pointer interface

    Mark Jan van Kampen committed May 13, 2020
    Configuration menu
    Copy the full SHA
    aaa10dd View commit details
    Browse the repository at this point in the history

  8. Adds missing free

    Mark Jan van Kampen committed May 13, 2020
    Configuration menu
    Copy the full SHA
    77293ab View commit details
    Browse the repository at this point in the history

Commits on May 16, 2020

  1. Cleans up some includes

    Mark Jan van Kampen committed May 16, 2020
    Configuration menu
    Copy the full SHA
    1309316 View commit details
    Browse the repository at this point in the history

  2. Fixes bad options 

    Changed some options that should not have changed
    Mark Jan van Kampen committed May 16, 2020
    Configuration menu
    Copy the full SHA
    d666af8 View commit details
    Browse the repository at this point in the history

  3. Moves to draft

    Mark Jan van Kampen committed May 16, 2020
    Configuration menu
    Copy the full SHA
    73807f8 View commit details
    Browse the repository at this point in the history

  4. Fixes formatting

    Mark Jan van Kampen committed May 16, 2020
    Configuration menu
    Copy the full SHA
    ffcede1 View commit details
    Browse the repository at this point in the history

  5. Updates copyright years

    Mark Jan van Kampen committed May 16, 2020
    Configuration menu
    Copy the full SHA
    312f8c3 View commit details
    Browse the repository at this point in the history

  6. Switches to new/delete 

    Cpp equivalent of malloc/free
    Mark Jan van Kampen committed May 16, 2020
    Configuration menu
    Copy the full SHA
    bf495f8 View commit details
    Browse the repository at this point in the history

  7. Switches to alternative log2

    Mark Jan van Kampen committed May 16, 2020
    Configuration menu
    Copy the full SHA
    61870a4 View commit details
    Browse the repository at this point in the history

  8. More copyright years

    Mark Jan van Kampen committed May 16, 2020
    Configuration menu
    Copy the full SHA
    c13f837 View commit details
    Browse the repository at this point in the history

  9. Fixes more formatting

    Mark Jan van Kampen committed May 16, 2020
    Configuration menu
    Copy the full SHA
    ea9c5dc View commit details
    Browse the repository at this point in the history

  10. Fixes more bad years

    Mark Jan van Kampen committed May 16, 2020
    Configuration menu
    Copy the full SHA
    20f49ec View commit details
    Browse the repository at this point in the history

Commits on May 17, 2020

  1. Fixes some concurrency issues and bugs

    Mark Jan van Kampen committed May 17, 2020
    Configuration menu
    Copy the full SHA
    ba05e8f View commit details
    Browse the repository at this point in the history

Commits on Sep 19, 2020

  1. Merge branch 'master' into memorypool

    Mark Jan van Kampen committed Sep 19, 2020
    Configuration menu
    Copy the full SHA
    afb858d View commit details
    Browse the repository at this point in the history

  2. Adds destroy fn

    Mark Jan van Kampen committed Sep 19, 2020
    Configuration menu
    Copy the full SHA
    32d827d View commit details
    Browse the repository at this point in the history

Commits on Jan 16, 2021

  1. Adds destroy

    Mark Jan van Kampen committed Jan 16, 2021
    Configuration menu
    Copy the full SHA
    84b4f8f View commit details
    Browse the repository at this point in the history