Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Reduce Memory Usage #125

Merged
merged 8 commits into from
Apr 3, 2023
Merged

Reduce Memory Usage #125

merged 8 commits into from
Apr 3, 2023

Commits on Apr 3, 2023

  1. in-situ subdirectory processing 

    if a directory has too many subdirectories, subdirectories past
a user provided count will be processed in the same thread as
the parent directory instead of being enqueued
    Use 0 to always enqueue

use -C to set the cutoff point between "small" and "large" directories
    @calccrypto
    calccrypto committed Apr 3, 2023
    Configuration menu
    Copy the full SHA
    496e226 View commit details
    Browse the repository at this point in the history

  2. deferred processing 

    when memory limit is reached, push new work into deferred work queue
    memory limit is constantly exceeded since there is no way to predict number of subdirectories a directory has
only process deferred work queue when active work queue is empty
    processing order is different than before, but work is still constantly processed
use -M to set size
    value is divided across threads
    @calccrypto
    calccrypto committed Apr 3, 2023
    Configuration menu
    Copy the full SHA
    3cfc30b View commit details
    Browse the repository at this point in the history

  3. remove bfmi

    @calccrypto
    calccrypto committed Apr 3, 2023
    Configuration menu
    Copy the full SHA
    fabe6e3 View commit details
    Browse the repository at this point in the history

  4. split struct work in half to reduce size of items enqueued in thread … 

    …pool

changed type to a single char
    @calccrypto
    calccrypto committed Apr 3, 2023
    Configuration menu
    Copy the full SHA
    9cbed2b View commit details
    Browse the repository at this point in the history

  5. use zlib and realloc to reduce size of queued items

    @calccrypto
    calccrypto committed Apr 3, 2023
    Configuration menu
    Copy the full SHA
    bbeafb3 View commit details
    Browse the repository at this point in the history

  6. QPTPool_threads_started and QPTPool_threads_completed now return uint… 

    …64_t
    @calccrypto
    calccrypto committed Apr 3, 2023
    Configuration menu
    Copy the full SHA
    1ef908f View commit details
    Browse the repository at this point in the history

  7. merge BENCHMARK with CUMULATIVE_TIMES

    @calccrypto
    calccrypto committed Apr 3, 2023
    Configuration menu
    Copy the full SHA
    673b170 View commit details
    Browse the repository at this point in the history

  8. update latex docs 

    in-situ processing
deferred queues,
reduced struct work size
compression with zlib
affected executables
    @calccrypto
    calccrypto committed Apr 3, 2023
    Configuration menu
    Copy the full SHA
    b998ef5 View commit details
    Browse the repository at this point in the history