wolfcrypt tests: improve file system gating for USE_CERT_BUFFERS by gojimmypi · Pull Request #7337 · wolfSSL/wolfssl

gojimmypi · 2024-03-14T22:22:40Z

Description

This PR has been retested (see commit history) and description updated since first submission.

/begin edit:

Former summary:

wolfcrypt tests: USE_CERT_BUFFERS_256 should be defined in user_settings.h

There's still the oddity of the USE_CERT_BUFFERS_256 defined locally in the wolfcrypt/test/test.c and used by the wolfssl/certs_test.h. This is not longer changed in this PR.

Instead, only the settings.h. is updated here to more robustly handle files or NO_FILESYSTEM.

I believe this code does not belong in test.c, but perhaps should be moved to settings.h as a default, or an error give that the user should select one in the respective user_settings.h:

#if defined(NO_FILESYSTEM) || defined(WC_NO_RNG)
    #if !defined(USE_CERT_BUFFERS_1024) && !defined(USE_CERT_BUFFERS_2048) && \
        !defined(USE_CERT_BUFFERS_3072) && !defined(USE_CERT_BUFFERS_4096)
        #define USE_CERT_BUFFERS_2048
    #endif
    #if !defined(USE_CERT_BUFFERS_256)
        #define USE_CERT_BUFFERS_256
    #endif
#endif

The problem is that the scope of USE_CERT_BUFFERS_256 for this PR is only in test.c, but should be applied throughout wolfcrypt.

I've confirmed that removing the #define USE_CERT_BUFFERS_256 is the cause of the current test failures.

I propose that the existing tests be modified to take this into account.

Although this PR addresses USE_CERT_BUFFERS_256, the #define USE_CERT_BUFFERS_2048 is probably needs to be similarly addressed.

/end edit

Similar to #7333, there are likely many nuances of the proposed changes herein that I may be missing.

In short, I think USE_CERT_BUFFERS_256 should be defined in the user_settings.h and not in test.c when needed. Doing this in test.c still leaves out the ecc definitions in wolfssl/certs_test.h, at least for embedded targets:

    #if !defined(USE_CERT_BUFFERS_256)
        #define USE_CERT_BUFFERS_256
    #endif

Of course, pulling that little string and a variety of other related dependencies became unwound, mainly surrounding having a filesystem or not.

I originally found this by plugging the Espressif wolfssl_client user_settings.h into the Espressif wolfssl_test.

There's probably a discussion to be had regarding my proposed revised gating logic, and if ASN_PARSE_E is the proper exit code to use.

I've also adjusted the indenting to make the code a bit more readable.

Current Behavior

Without the #define USE_CERT_BUFFERS_256 in the above-mentioned user_settings.h, here are the errors that occurred, and are fixed in this PR.

[112/118] Building C object esp-idf/wolfssl/CMakeFiles/__idf_wolfssl.dir/C_/workspace/wolfssl-gojimmypi-pr/wolfcrypt/test/test.c.obj
FAILED: esp-idf/wolfssl/CMakeFiles/__idf_wolfssl.dir/C_/workspace/wolfssl-gojimmypi-pr/wolfcrypt/test/test.c.obj 
C:\SysGCC\esp32\tools\xtensa-esp-elf\esp-13.2.0_20230928\xtensa-esp-elf\bin\xtensa-esp32-elf-gcc.exe -DESP_PLATFORM -DIDF_VER=\"v5.2-dev-3903-g66992aca7a-dirty\" -DLIBWOLFSSL_VERSION_GIT_BRANCH=\"PR-test-certbuf-256\" -DLIBWOLFSSL_VERSION_GIT_HASH=\"2adcb1b0df3da4dc342bb185233c3917bfec3097\" -DLIBWOLFSSL_VERSION_GIT_HASH_DATE="\"'Thu Mar 14 14:11:30 2024 -0700'\"" -DLIBWOLFSSL_VERSION_GIT_ORIGIN=\"https://github.com/gojimmypi/wolfssl.git\" -DLIBWOLFSSL_VERSION_GIT_SHORT_HASH=\"2adcb1b0d\" -DLIBWOLFSSL_VERSION_WOLFSSL_ROOT=\"C:/workspace/wolfssl-gojimmypi-pr\" -DSOC_MMU_PAGE_SIZE=CONFIG_MMU_PAGE_SIZE -DSOC_XTAL_FREQ_MHZ=CONFIG_XTAL_FREQ -DWOLFSSL_USER_SETTINGS_DIR=\"C://workspace//wolfssl-gojimmypi-pr//IDE//Espressif//ESP-IDF//examples//wolfssl_test//components//wolfssl//include//user_settings.h\" -D_GLIBCXX_HAVE_POSIX_SEMAPHORE -D_GLIBCXX_USE_POSIX_SEMAPHORE -D_GNU_SOURCE -D_POSIX_READER_WRITER_LOCKS -IC:/workspace/wolfssl-gojimmypi-pr/IDE/Espressif/ESP-IDF/examples/wolfssl_test/build/VisualGDB/Debug/config -IC:/workspace/wolfssl-gojimmypi-pr/IDE/Espressif/ESP-IDF/examples/wolfssl_test/components/wolfssl/include -IC:/workspace/wolfssl-gojimmypi-pr -IC:/workspace/wolfssl-gojimmypi-pr/wolfssl -IC:/workspace/wolfssl-gojimmypi-pr/wolfssl/wolfcrypt -IC:/workspace/wolfssl-gojimmypi-pr/wolfssl/wolfcrypt/port/Espressif -IC:/SysGCC/esp32/esp-idf/v5.2/components/freertos/FreeRTOS-Kernel/include/freertos -IC:/SysGCC/esp32/esp-idf/v5.2/components/newlib/platform_include -IC:/SysGCC/esp32/esp-idf/v5.2/components/freertos/config/include -IC:/SysGCC/esp32/esp-idf/v5.2/components/freertos/config/include/freertos -IC:/SysGCC/esp32/esp-idf/v5.2/components/freertos/config/xtensa/include -IC:/SysGCC/esp32/esp-idf/v5.2/components/freertos/FreeRTOS-Kernel/include -IC:/SysGCC/esp32/esp-idf/v5.2/components/freertos/FreeRTOS-Kernel/portable/xtensa/include -IC:/SysGCC/esp32/esp-idf/v5.2/components/freertos/FreeRTOS-Kernel/portable/xtensa/include/freertos -IC:/SysGCC/esp32/esp-idf/v5.2/components/freertos/esp_additions/include -IC:/SysGCC/esp32/esp-idf/v5.2/components/esp_hw_support/include -IC:/SysGCC/esp32/esp-idf/v5.2/components/esp_hw_support/include/soc -IC:/SysGCC/esp32/esp-idf/v5.2/components/esp_hw_support/include/soc/esp32 -IC:/SysGCC/esp32/esp-idf/v5.2/components/esp_hw_support/port/esp32/. -IC:/SysGCC/esp32/esp-idf/v5.2/components/esp_hw_support/port/esp32/private_include -IC:/SysGCC/esp32/esp-idf/v5.2/components/heap/include -IC:/SysGCC/esp32/esp-idf/v5.2/components/log/include -IC:/SysGCC/esp32/esp-idf/v5.2/components/soc/include -IC:/SysGCC/esp32/esp-idf/v5.2/components/soc/esp32 -IC:/SysGCC/esp32/esp-idf/v5.2/components/soc/esp32/include -IC:/SysGCC/esp32/esp-idf/v5.2/components/hal/platform_port/include -IC:/SysGCC/esp32/esp-idf/v5.2/components/hal/esp32/include -IC:/SysGCC/esp32/esp-idf/v5.2/components/hal/include -IC:/SysGCC/esp32/esp-idf/v5.2/components/esp_rom/include -IC:/SysGCC/esp32/esp-idf/v5.2/components/esp_rom/include/esp32 -IC:/SysGCC/esp32/esp-idf/v5.2/components/esp_rom/esp32 -IC:/SysGCC/esp32/esp-idf/v5.2/components/esp_common/include -IC:/SysGCC/esp32/esp-idf/v5.2/components/esp_system/include -IC:/SysGCC/esp32/esp-idf/v5.2/components/esp_system/port/soc -IC:/SysGCC/esp32/esp-idf/v5.2/components/esp_system/port/include/private -IC:/SysGCC/esp32/esp-idf/v5.2/components/xtensa/esp32/include -IC:/SysGCC/esp32/esp-idf/v5.2/components/xtensa/include -IC:/SysGCC/esp32/esp-idf/v5.2/components/xtensa/deprecated_include -IC:/SysGCC/esp32/esp-idf/v5.2/components/lwip/include -IC:/SysGCC/esp32/esp-idf/v5.2/components/lwip/include/apps -IC:/SysGCC/esp32/esp-idf/v5.2/components/lwip/include/apps/sntp -IC:/SysGCC/esp32/esp-idf/v5.2/components/lwip/lwip/src/include -IC:/SysGCC/esp32/esp-idf/v5.2/components/lwip/port/include -IC:/SysGCC/esp32/esp-idf/v5.2/components/lwip/port/freertos/include -IC:/SysGCC/esp32/esp-idf/v5.2/components/lwip/port/esp32xx/include -IC:/SysGCC/esp32/esp-idf/v5.2/components/lwip/port/esp32xx/include/arch -IC:/SysGCC/esp32/esp-idf/v5.2/components/lwip/port/esp32xx/include/sys -mlongcalls -Wno-frame-address  -DWOLFSSL_USER_SETTINGS -g -fdiagnostics-color=always -DWOLFSSL_ESP_NO_WATCHDOG=1 -ffunction-sections -fdata-sections -Wall -Werror=all -Wno-error=unused-function -Wno-error=unused-variable -Wno-error=unused-but-set-variable -Wno-error=deprecated-declarations -Wextra -Wno-unused-parameter -Wno-sign-compare -Wno-enum-conversion -gdwarf-4 -ggdb -Og -fno-shrink-wrap -fstack-protector -fmacro-prefix-map=C:/workspace/wolfssl-gojimmypi-pr/IDE/Espressif/ESP-IDF/examples/wolfssl_test=. -fmacro-prefix-map=C:/SysGCC/esp32/esp-idf/v5.2=/IDF -fstrict-volatile-bitfields -fno-jump-tables -fno-tree-switch-conversion -std=gnu17 -Wno-old-style-declaration -MD -MT esp-idf/wolfssl/CMakeFiles/__idf_wolfssl.dir/C_/workspace/wolfssl-gojimmypi-pr/wolfcrypt/test/test.c.obj -MF esp-idf\wolfssl\CMakeFiles\__idf_wolfssl.dir\C_\workspace\wolfssl-gojimmypi-pr\wolfcrypt\test\test.c.obj.d -o esp-idf/wolfssl/CMakeFiles/__idf_wolfssl.dir/C_/workspace/wolfssl-gojimmypi-pr/wolfcrypt/test/test.c.obj -c C:/workspace/wolfssl-gojimmypi-pr/wolfcrypt/test/test.c
In file included from C:/workspace/wolfssl-gojimmypi-pr/wolfcrypt/test/test.c:43:
C:/workspace/wolfssl-gojimmypi-pr/wolfcrypt/test/test.c: In function 'rsa_ecc_certgen_test':
C:/workspace/wolfssl-gojimmypi-pr/wolfcrypt/test/test.c:18755:18: error: 'ecc_key_pub_der_256' undeclared (first use in this function)
18755 |     XMEMCPY(tmp, ecc_key_pub_der_256, sizeof_ecc_key_pub_der_256);
      |                  ^~~~~~~~~~~~~~~~~~~
C:/workspace/wolfssl-gojimmypi-pr/wolfssl/wolfcrypt/types.h:710:51: note: in definition of macro 'XMEMCPY'
  710 |             #define XMEMCPY(d,s,l)    memcpy((d),(s),(l))
      |                                                   ^
C:/workspace/wolfssl-gojimmypi-pr/wolfcrypt/test/test.c:18755:18: note: each undeclared identifier is reported only once for each function it appears in
18755 |     XMEMCPY(tmp, ecc_key_pub_der_256, sizeof_ecc_key_pub_der_256);
      |                  ^~~~~~~~~~~~~~~~~~~
C:/workspace/wolfssl-gojimmypi-pr/wolfssl/wolfcrypt/types.h:710:51: note: in definition of macro 'XMEMCPY'
  710 |             #define XMEMCPY(d,s,l)    memcpy((d),(s),(l))
      |                                                   ^
C:/workspace/wolfssl-gojimmypi-pr/wolfcrypt/test/test.c:18755:39: error: 'sizeof_ecc_key_pub_der_256' undeclared (first use in this function); did you mean 'sizeof_ca_key_der_2048'?
18755 |     XMEMCPY(tmp, ecc_key_pub_der_256, sizeof_ecc_key_pub_der_256);
      |                                       ^~~~~~~~~~~~~~~~~~~~~~~~~~
C:/workspace/wolfssl-gojimmypi-pr/wolfssl/wolfcrypt/types.h:710:55: note: in definition of macro 'XMEMCPY'
  710 |             #define XMEMCPY(d,s,l)    memcpy((d),(s),(l))
      |                                                       ^
C:/workspace/wolfssl-gojimmypi-pr/wolfcrypt/test/test.c: In function 'ecc_test_make_pub':
C:/workspace/wolfssl-gojimmypi-pr/wolfcrypt/test/test.c:27348:18: error: 'ecc_key_der_256' undeclared (first use in this function); did you mean 'ca_key_der_2048'?
27348 |     XMEMCPY(tmp, ecc_key_der_256, (size_t)sizeof_ecc_key_der_256);
      |                  ^~~~~~~~~~~~~~~
C:/workspace/wolfssl-gojimmypi-pr/wolfssl/wolfcrypt/types.h:710:51: note: in definition of macro 'XMEMCPY'
  710 |             #define XMEMCPY(d,s,l)    memcpy((d),(s),(l))
      |                                                   ^
C:/workspace/wolfssl-gojimmypi-pr/wolfcrypt/test/test.c:27348:43: error: 'sizeof_ecc_key_der_256' undeclared (first use in this function); did you mean 'sizeof_ca_key_der_2048'?
27348 |     XMEMCPY(tmp, ecc_key_der_256, (size_t)sizeof_ecc_key_der_256);
      |                                           ^~~~~~~~~~~~~~~~~~~~~~
C:/workspace/wolfssl-gojimmypi-pr/wolfssl/wolfcrypt/types.h:710:55: note: in definition of macro 'XMEMCPY'
  710 |             #define XMEMCPY(d,s,l)    memcpy((d),(s),(l))
      |                                                       ^
C:/workspace/wolfssl-gojimmypi-pr/wolfcrypt/test/test.c: In function 'ecc_def_curve_test':
C:/workspace/wolfssl-gojimmypi-pr/wolfcrypt/test/test.c:28894:34: error: 'ecc_key_der_256' undeclared (first use in this function); did you mean 'ca_key_der_2048'?
28894 |     ret = wc_EccPrivateKeyDecode(ecc_key_der_256, &idx, key,
      |                                  ^~~~~~~~~~~~~~~
      |                                  ca_key_der_2048
C:/workspace/wolfssl-gojimmypi-pr/wolfcrypt/test/test.c:28895:9: error: 'sizeof_ecc_key_der_256' undeclared (first use in this function); did you mean 'sizeof_ca_key_der_2048'?
28895 |         sizeof_ecc_key_der_256);
      |         ^~~~~~~~~~~~~~~~~~~~~~
      |         sizeof_ca_key_der_2048
C:/workspace/wolfssl-gojimmypi-pr/wolfcrypt/test/test.c: In function 'ecc_test_cert_gen':
C:/workspace/wolfssl-gojimmypi-pr/wolfcrypt/test/test.c:29872:18: error: 'ca_ecc_key_der_384' undeclared (first use in this function)
29872 |     XMEMCPY(der, ca_ecc_key_der_384, sizeof_ca_ecc_key_der_384);
      |                  ^~~~~~~~~~~~~~~~~~
C:/workspace/wolfssl-gojimmypi-pr/wolfssl/wolfcrypt/types.h:710:51: note: in definition of macro 'XMEMCPY'
  710 |             #define XMEMCPY(d,s,l)    memcpy((d),(s),(l))
      |                                                   ^
C:/workspace/wolfssl-gojimmypi-pr/wolfcrypt/test/test.c:29872:38: error: 'sizeof_ca_ecc_key_der_384' undeclared (first use in this function); did you mean 'sizeof_ca_key_der_2048'?
29872 |     XMEMCPY(der, ca_ecc_key_der_384, sizeof_ca_ecc_key_der_384);
      |                                      ^~~~~~~~~~~~~~~~~~~~~~~~~
C:/workspace/wolfssl-gojimmypi-pr/wolfssl/wolfcrypt/types.h:710:55: note: in definition of macro 'XMEMCPY'
  710 |             #define XMEMCPY(d,s,l)    memcpy((d),(s),(l))
      |                                                       ^
C:/workspace/wolfssl-gojimmypi-pr/wolfcrypt/test/test.c:29964:38: error: 'ca_ecc_cert_der_384' undeclared (first use in this function); did you mean 'ca_cert_der_2048'?
29964 |     ret = wc_SetIssuerBuffer(myCert, ca_ecc_cert_der_384,
      |                                      ^~~~~~~~~~~~~~~~~~~
      |                                      ca_cert_der_2048
C:/workspace/wolfssl-gojimmypi-pr/wolfcrypt/test/test.c:29965:39: error: 'sizeof_ca_ecc_cert_der_384' undeclared (first use in this function); did you mean 'sizeof_ca_cert_der_2048'?
29965 |                                       sizeof_ca_ecc_cert_der_384);
      |                                       ^~~~~~~~~~~~~~~~~~~~~~~~~~
      |                                       sizeof_ca_cert_der_2048
C:/workspace/wolfssl-gojimmypi-pr/wolfcrypt/test/test.c: In function 'ecc_test_buffers':
C:/workspace/wolfssl-gojimmypi-pr/wolfcrypt/test/test.c:31705:21: error: 'sizeof_ecc_clikey_der_256' undeclared (first use in this function); did you mean 'sizeof_ca_key_der_2048'?
31705 |     bytes = (size_t)sizeof_ecc_clikey_der_256;
      |                     ^~~~~~~~~~~~~~~~~~~~~~~~~
      |                     sizeof_ca_key_der_2048
C:/workspace/wolfssl-gojimmypi-pr/wolfcrypt/test/test.c:31707:34: error: 'ecc_clikey_der_256' undeclared (first use in this function)
31707 |     ret = wc_EccPrivateKeyDecode(ecc_clikey_der_256, &idx, cliKey,
      |                                  ^~~~~~~~~~~~~~~~~~
C:/workspace/wolfssl-gojimmypi-pr/wolfcrypt/test/test.c:31713:21: error: 'sizeof_ecc_key_der_256' undeclared (first use in this function); did you mean 'sizeof_ca_key_der_2048'?
31713 |     bytes = (size_t)sizeof_ecc_key_der_256;
      |                     ^~~~~~~~~~~~~~~~~~~~~~
      |                     sizeof_ca_key_der_2048
C:/workspace/wolfssl-gojimmypi-pr/wolfcrypt/test/test.c:31716:34: error: 'ecc_key_der_256' undeclared (first use in this function); did you mean 'ca_key_der_2048'?
31716 |     ret = wc_EccPrivateKeyDecode(ecc_key_der_256, &idx, servKey,
      |                                  ^~~~~~~~~~~~~~~
      |                                  ca_key_der_2048
C:/workspace/wolfssl-gojimmypi-pr/wolfcrypt/test/test.c:31799:13: error: 'sizeof_ecc_clikeypub_der_256' undeclared (first use in this function)
31799 |     bytes = sizeof_ecc_clikeypub_der_256;
      |             ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
C:/workspace/wolfssl-gojimmypi-pr/wolfcrypt/test/test.c:31801:33: error: 'ecc_clikeypub_der_256' undeclared (first use in this function)
31801 |     ret = wc_EccPublicKeyDecode(ecc_clikeypub_der_256, &idx, cliKey,
      |                                 ^~~~~~~~~~~~~~~~~~~~~
ninja: build stopped: subcommand failed.
-------------------------------------------------------------
Command exited with code 1
Executable: C:\SysGCC\esp32\tools\ninja\1.11.1\ninja.exe
Arguments: 
Directory: C:\workspace\wolfssl-gojimmypi-pr\IDE\Espressif\ESP-IDF\examples\wolfssl_test\VisualGDB/../build/VisualGDB/Debug
Command-line action failed

New Behavior

With this update, here's how the missing USE_CERT_BUFFERS_256 error is now presented with DEBUG_WOLFSSL enabled (on the ESP32):

I (71074) wolfssl: Getting Cert Name
I (71078) wolfssl: wolfSSL Entering GetAlgoId
I (71083) wolfssl: wolfSSL Entering DecodeCertExtensions
I (71089) wolfssl: wolfSSL Entering DecodeSubjKeyId
I (71094) wolfssl: wolfSSL Entering DecodeAuthKeyId
I (71100) wolfssl: wolfSSL Entering DecodeBasicCaConstraint
I (71106) wolfssl: wolfSSL Entering DecodeAltNames
I (71111) wolfssl:      Unsupported name type, skipping
I (71117) wolfssl: wolfSSL Entering DecodeExtKeyUsage
I (75221) wolfssl: mp_to_unsigned_bin_len_ct...
I (75222) wolfssl: No file system and USE_CERT_BUFFERS_256 not defined.(1)
E (75223) wc_test: Failed: Error = -140 during rsa_ecc_certgen_test, line 18773

Fixes zd# n/a

Testing

How did you test?

Tested primarily in Espressif with various user_settings.h values, as well as a general --enable-all comandline.

Checklist

added tests
updated/added doxygen
updated appropriate READMEs
Updated manual and documentation

gojimmypi · 2024-03-24T22:56:34Z

Jenkins retest this please

gojimmypi added the enhancement label Mar 14, 2024

gojimmypi requested review from JacobBarthelmeh and dgarske March 14, 2024 22:22

gojimmypi assigned dgarske, JacobBarthelmeh and gojimmypi Mar 14, 2024

gojimmypi added the Not For This Release Not for release 5.9.1 label Mar 14, 2024

dgarske assigned wolfSSL-Bot and unassigned dgarske Mar 18, 2024

gojimmypi force-pushed the PR-test-certbuf-256 branch from 2adcb1b to e80f951 Compare March 22, 2024 15:39

gojimmypi removed the Not For This Release Not for release 5.9.1 label Mar 22, 2024

Improve wolfcrypt test NO_FILESYSTEM checks

bf97758

gojimmypi force-pushed the PR-test-certbuf-256 branch from 84e4843 to bf97758 Compare March 22, 2024 20:00

gojimmypi changed the title ~~wolfcrypt tests: USE_CERT_BUFFERS_256 should be defined in user_settings.h~~ wolfcrypt tests: improve file system gating for USE_CERT_BUFFERS Mar 25, 2024

douzzer approved these changes Mar 30, 2024

View reviewed changes

douzzer merged commit 03d7eac into wolfSSL:master Mar 30, 2024

gojimmypi deleted the PR-test-certbuf-256 branch October 9, 2024 18:09

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

wolfcrypt tests: improve file system gating for USE_CERT_BUFFERS#7337

wolfcrypt tests: improve file system gating for USE_CERT_BUFFERS#7337
douzzer merged 1 commit intowolfSSL:masterfrom
gojimmypi:PR-test-certbuf-256

gojimmypi commented Mar 14, 2024 •

edited

Loading

Uh oh!

gojimmypi commented Mar 24, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Conversation

gojimmypi commented Mar 14, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Current Behavior

New Behavior

Testing

Checklist

Uh oh!

gojimmypi commented Mar 24, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

gojimmypi commented Mar 14, 2024 •

edited

Loading