20240215-benchmark-smallstack-refactors

[douzzer]

wolfssl/wolfcrypt/types.h:

• fix overallocation in WC_DECLARE_ARRAY() macro in the !WOLFSSL_SMALL_STACK path.
• rename WC_INIT_ARRAY() to WC_ALLOC_ARRAY() for clarity (it doesn't initialize any memory).
• rename WC_DECLARE_ARRAY_DYNAMIC_DEC(), WC_DECLARE_ARRAY_DYNAMIC_EXE(), and WC_FREE_ARRAY_DYNAMIC() to WC_DECLARE_HEAP_ARRAY(), WC_ALLOC_HEAP_ARRAY(), and WC_FREE_HEAP_ARRAY(), respectively, also for clarity, and refactor out the duplicate definitions.
• add WC_ALLOC_VAR(), and move the XMALLOC() in smallstack WC_DECLARE_VAR() into it. smallstack WC_DECLARE_VAR() now initializes the pointer to NULL, like smallstack WC_DECLARE_ARRAY(), assuring all pointers are valid upon shortcircuit to cleanup for a failed allocation (see WC_ALLOC_DO_ON_FAILURE below).
• add a new hook WC_ALLOC_DO_ON_FAILURE in WC_ALLOC_VAR(), WC_ALLOC_ARRAY(), and WC_ALLOC_HEAP_ARRAY(), which is invoked when an allocation fails. by default the hook is defined to WC_DO_NOTHING preserving the incumbent behavior.
• add basic safety to WC_*_HEAP_ARRAY() by recording/detecting allocation state via idx##VAR_NAME.
• add macros WC_ARRAY_OK() and WC_HEAP_ARRAY_OK() to test if allocation succeeded.
• add macros WC_CALLOC_ARRAY() and WC_CALLOC_HEAP_ARRAY() which zero the objects.
• add macro WC_CALLOC_VAR() which zeros the object.

ED448: smallstack refactor of ge448_scalarmult_base().

src/tls.c tests/api.c wolfcrypt/test/test.c: update WC_DECLARE_VAR()s with now-required matching WC_ALLOC_VAR()s.

wolfcrypt/benchmark/benchmark.c:

• no functional changes in default error-free behavior.
• add definition of WC_ALLOC_DO_ON_FAILURE() that prints error message, sets ret, and does goto exit.
• add BENCH_NTIMES and BENCH_AGREETIMES overrideeable macros, to allow fast sanitizer runs and slow high-precision runs.
• smallstack refactor of all declarations of stack arrays of the form foo[BENCH_MAX_PENDING], using WC_DECLARE_ARRAY()` (35 in all).
• additional smallstack refactors, using WC_DECLARE_VAR(), for bench_aesxts(), bench_ed448KeyGen(), bench_eccsi*(), and bench_sakke*().
• fixes for various unhandled error conditions around malloc failures.

wolfcrypt/test/test.c: opportunistically constify several (42) static constants, moving them to the readonly data segment.

linuxkm/Makefile: if ENABLED_LINUXKM_BENCHMARKS, add wolfcrypt/benchmark/benchmark.o to WOLFSSL_OBJ_FILES.

linuxkm/Kbuild: enable FPU for benchmark.o, and remove enablement for module_hooks.o.

linuxkm/module_hooks.c: remove inline include of benchmark.c.

tested with wolfssl-multi-test.sh ... super-quick-check all-max-func-stack-2k linuxkm-benchmarks sanitize-all-wolfcrypt-benchmark sanitize-all-wolfcrypt-benchmark-smallstack.

sanitize-all-wolfcrypt-benchmark and sanitize-all-wolfcrypt-benchmark-smallstack are new scenarios that run benchmark under the sanitizer. all-max-func-stack-2k is modified from --disable-benchmark to --enable-benchmark.

before fix to WC_DECLARE_ARRAY(), bench_aescbc_internal() (for example) allocated 1218816 bytes on stack, versus needed 1104 bytes.

[dgarske]

Looks like this still needs some work. Scan-build is unhappy:

Example:

./configure --enable-singlethreaded --enable-sp=small --enable-smallstack --enable-smallstackcache CPPFLAGS="-DECC_CACHE_CURVE -DHAVE_WOLF_BIGINT -DWOLFSSL_OLD_PRIME_CHECK"

wolfcrypt/benchmark/benchmark.c: In function ‘bench_dh’:
./wolfssl/wolfcrypt/types.h:575:97: error: ‘priv$’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
  575 |                     #define XFREE(p, h, t)       {void* xp = (p); (void)(h); (void)(t); if (xp) wolfSSL_Free(xp);}
      |                                                                                                 ^~~~~~~~~~~~
wolfcrypt/benchmark/benchmark.c:8938:22: note: ‘priv$’ was declared here
 8938 |     WC_DECLARE_ARRAY(priv, byte, BENCH_MAX_PENDING,
      |                      ^~~~
./wolfssl/wolfcrypt/types.h:588:19: note: in definition of macro ‘WC_DECLARE_HEAP_ARRAY’
  588 |         VAR_TYPE* VAR_NAME[VAR_ITEMS]; \
      |                   ^~~~~~~~
wolfcrypt/benchmark/benchmark.c:8938:5: note: in expansion of macro ‘WC_DECLARE_ARRAY’
 8938 |     WC_DECLARE_ARRAY(priv, byte, BENCH_MAX_PENDING,
      |     ^~~~~~~~~~~~~~~~

[douzzer]

@dgarske I addressed the apparently-false positives from scan-build with unconditional XMEMSET()s. clang-tidy doesn't report defects there, so this was probably just old-llvm symptoms.

[douzzer]

retest this please

[douzzer]

retest this please