paratest - parametric test of program
paratest [OPTIONS] -r PROGRAM [PROGRAM-OPTIONS]
Repeatedly run arbitrary program with different arguments. The sequence of parameters supplied into arguments of program can be either linear or exponentially-growing for logarithmic x-axis plots.
To allow full utilization of the CPU and cache, tested programs are executed in sequence, not in parallel. However, multiple adequately configured paratests can be run in parallel to fully squeeze the multi-core CPU, when the performance available to the tested program is not important.
-h This help.
-v Extra verbose.
-d Debug output.
-q Query mode, only report what to run.
-a Include all individual tests results into log.
-n NAME Name of test, output file will be NAME.log.
-k KEYS Comma separated list of keys to log.
-r PROG ARGS Run program PROG with arguments ARGS.
-f5d Default numbers-format for logs (see KEYS).
-p=STR Parameter name (default A).
-A1=NUM Start.
-A2=NUM Stop.
-s=NUM Step, or log multiplayer, might use %.
-n=NUM Number of steps.
-log Logarithmic schedule (default linear).
-dp=NUM Decimal precision of A (default none).
-bs=NUM Next A only after surpassing this batch step (default none).
-bS=NUM Stop exactly on the batch steps (none).
-sl=NUM Sleep time between runs, decimal seconds (default none).
-eq=NUM Number of concurrent equal runs.
-rnd=NUM:NUM Number of concurrent randomized runs and radius.
Radius can be relative to the A parameter if given as %.
-jrn=PATH Paths to journal files to use.
-log=PATH Paths to log files to use.
Use the A letter to incorporate the tested parameter into
program arguments, like: nntrain -n test-A -t 146-A-3
The -k mcups:f2d,amae:maxmax,amae:maxavgmin specifies three columns to
store: the mcups and twice the amae variables from logs and journals.
The keys are comma separated, and optionally after a colon every key can
carry its specs. They can specify formatting of stored variable or
aggregation method. The aggregation has three levels:
level-1 in-file aggregation, for log-files generated by tested
programs,
level-2 between-several-files aggregation, if more logs/journals
are specified,
level-3 between-tests aggregation, if concurrent or randomized
runs were requested.
The maxavgmin by its order specifies the max aggregation on level-1,
the avg on level-2 and the min on level-3. If no level-3 method is
specified the level-2 will be reused. The min is default for level-1.
Available aggregation and formatting methods are:
min minimum, also as an alias l,
max maximum, also as an alias h,
avg average, also as an alias a,
f5d specifies max five decimal places to be stored,
f5D zero-padded exactly five decimal places.
paratest-0.1 (c) R.Jaksa 2019, GPLv3