limit number of threads

[milahu]

currently, i cannot limit the number of threads

using all threads is nice for me, but its bad on a shared machine

$ nproc
128

$ rar2john asdf.rar > asdf.rar.hashes

$ john asdf.rar.hashes
Will run 128 OpenMP threads

$ john --node=1-1/4 asdf.rar.hashes
Will run 128 OpenMP threads

$ john --fork=1 asdf.rar.hashes
--fork number must be between 2 and 1024

$ john --fork=2 asdf.rar.hashes
Will run 64 OpenMP threads per process (128 total across 2 processes)

john version 9f913a7

john --list=build-info

$ john --list=build-info
Version: 1.9.0-jumbo-1+bleeding-9f913a7340 2024-05-21 23:59:57 +0200
Build: linux-gnu 64-bit x86_64 SSE2 AC OMP
SIMD: SSE2, interleaving: MD4:3 MD5:3 SHA1:1 SHA256:1 SHA512:1
System-wide exec: /nix/store/mwknqmz7mcs13106w0wm6x7lwmfr4mnv-john-1.9.0-jumbo-1/bin
System-wide home: /nix/store/mwknqmz7mcs13106w0wm6x7lwmfr4mnv-john-1.9.0-jumbo-1/share/john
Private home: ~/.john
$JOHN is /nix/store/mwknqmz7mcs13106w0wm6x7lwmfr4mnv-john-1.9.0-jumbo-1/share/john/
Format interface version: 14
Max. number of reported tunable costs: 4
Rec file version: REC4
Charset file version: CHR3
CHARSET_MIN: 1 (0x01)
CHARSET_MAX: 255 (0xff)
CHARSET_LENGTH: 24
SALT_HASH_SIZE: 1048576
SINGLE_IDX_MAX: 32768
SINGLE_BUF_MAX: 4294967295
Effective limit: Max. KPC 32768
Max. Markov mode level: 400
Max. Markov mode password length: 30
gcc version: 13.2.0
GNU libc version: 2.39 (loaded: 2.39)
Crypto library: OpenSSL
OpenSSL library version: 0300000d0
OpenSSL 3.0.13 30 Jan 2024
GMP library version: 6.3.0
File locking: fcntl()
fseek(): fseek
ftell(): ftell
fopen(): fopen
memmem(): System's
times(2) sysconf(_SC_CLK_TCK) is 100
Using times(2) for timers, resolution 10 ms
HR timer: clock_gettime(), latency 20 ns
Total physical host memory: 188 GiB
Available physical host memory: 145 GiB
Terminal locale string: en_US.UTF-8
Parsed terminal locale: UTF-8

workaround

$ cpulimit --help
Usage: cpulimit [OPTIONS...] TARGET
   OPTIONS
      -l, --limit=N          percentage of cpu allowed from 0 to 12800 (required)
      -v, --verbose          show control statistics
      -z, --lazy             exit if there is no target process, or if it dies
      -i, --include-children limit also the children processes
      -h, --help             display this help and exit
   TARGET must be exactly one of these:
      -p, --pid=N            pid of the process (implies -z)
      -e, --exe=FILE         name of the executable program file or path name
      COMMAND [ARGS]         run this command and limit it (implies -z)

$ cpulimit --limit=3200 --include-children john asdf.rar.hashes
Will run 128 OpenMP threads

this still uses all threads, but limits the cpu load to 25% per thread

[solardiz]

Hi @milahu. With OpenMP, you control this via the OMP_NUM_THREADS environment variable.

[solardiz]

Examples:

OMP_NUM_THREADS=32 john asdf.rar.hashes

or

OMP_NUM_THREADS=1 john --fork=32 asdf.rar.hashes

or

export OMP_NUM_THREADS=32
john asdf.rar.hashes