Current master fails test 30/74 in SM2 on Apple Silicon

[mouse07410]

MacOS Sonoma 14.2.1, Xcode-15.2. Current master.

30-test_engine.t ........................ ok

        # INFO:  @ test/testutil/stanza.c:21
        # Reading ../../test/recipes/30-test_evp_data/evppkey_sm2.txt
        # INFO:  @ test/testutil/stanza.c:122
        # Starting "SM2 tests" tests at line 19
        # INFO:  @ test/evp_test.c:3720
        # ../../test/recipes/30-test_evp_data/evppkey_sm2.txt:63: Source of above error; unexpected error KEYOP_ERROR
        # 005003E401000000:error:1A800066:SM2 routines:ossl_sm2_decrypt:reason(102):crypto/sm2/sm2_crypt.c:384:
        # INFO:  @ test/testutil/stanza.c:122
        # Starting "SM2 key generation tests" tests at line 78
        # INFO:  @ test/testutil/stanza.c:33
        # Completed 10 tests with 1 errors and 0 skipped
        # OPENSSL_TEST_RAND_SEED=1705934423
        not ok 1 - iteration 1
# ------------------------------------------------------------------------------
    # OPENSSL_TEST_RAND_SEED=1705934423
    not ok 1 - run_file_tests
# ------------------------------------------------------------------------------
../../util/wrap.pl ../../test/evp_test -config ../../test/default-and-legacy.cnf ../../test/recipes/30-test_evp_data/evppkey_sm2.txt => 1
not ok 74 - running evp_test -config ../../test/default-and-legacy.cnf evppkey_sm2.txt
# ------------------------------------------------------------------------------
# Looks like you failed 1 test of 80.30-test_evp.t ........................... 
Dubious, test returned 1 (wstat 256, 0x100)
Failed 1/80 subtests 
	(less 3 skipped subtests: 76 okay)
30-test_evp_extra.t ..................... ok

[t8m]

What is the CPU architecture of the machine? Could you please dump the build configuration with ./configdata.pm -d?

[mouse07410]

@t8m forgot to mention, sorry. Apple Silicon M2 Max chip.

$ ./configdata.pm -d

Command line (with current working directory = .):

    /opt/local/bin/perl5.34 ./Configure --prefix=/Users/ur20980/openssl-3 --debug --openssldir=/Users/ur20980/openssl-3/etc --with-rand-seed=os enable-ec_nistp_64_gcc_128 enable-md2 enable-rc5 enable-weak-ssl-ciphers zlib-dynamic enable-ssl-trace enable-trace threads enable-buildtest-c++ enable-quic

Perl information:

    /opt/local/bin/perl5.34
    5.34.3 for darwin-thread-multi-2level

Enabled features:

    apps
    argon2
    aria
    asm
    async
    autoalginit
    autoerrinit
    autoload-config
    bf
    blake2
    buildtest-c++
    bulk
    cached-fetch
    camellia
    capieng
    cast
    chacha
    cmac
    cmp
    cms
    comp
    ct
    default-thread-pool
    deprecated
    des
    dgram
    dh
    docs
    dsa
    dso
    dtls
    dynamic-engine
    ec
    ec2m
    ec_nistp_64_gcc_128
    ecdh
    ecdsa
    ecx
    engine
    err
    filenames
    gost
    http
    idea
    legacy
    loadereng
    makedepend
    md2
    md4
    mdc2
    module
    multiblock
    nextprotoneg
    ocb
    ocsp
    padlockeng
    pic
    pinshared
    poly1305
    posix-io
    psk
    quic
    rc2
    rc4
    rc5
    rdrand
    rfc3779
    rmd160
    scrypt
    secure-memory
    seed
    shared
    siphash
    siv
    sm2
    sm2-precomp
    sm3
    sm4
    sock
    srp
    srtp
    sse2
    ssl
    ssl-trace
    static-engine
    stdio
    tests
    thread-pool
    threads
    tls
    trace
    ts
    ui-console
    weak-ssl-ciphers
    whirlpool
    zlib
    zlib-dynamic
    tls1
    tls1-method
    tls1_1
    tls1_1-method
    tls1_2
    tls1_2-method
    tls1_3
    dtls1
    dtls1-method
    dtls1_2
    dtls1_2-method

Disabled features:

    acvp-tests          [cascade]        OPENSSL_NO_ACVP_TESTS
    afalgeng            [not-linux]      OPENSSL_NO_AFALGENG
    asan                [default]        OPENSSL_NO_ASAN
    brotli              [default]        OPENSSL_NO_BROTLI
    brotli-dynamic      [default]        OPENSSL_NO_BROTLI_DYNAMIC
    winstore            [not-windows]    OPENSSL_NO_WINSTORE
    crypto-mdebug       [default]        OPENSSL_NO_CRYPTO_MDEBUG
    devcryptoeng        [default]        OPENSSL_NO_DEVCRYPTOENG
    egd                 [default]        OPENSSL_NO_EGD
    external-tests      [default]        OPENSSL_NO_EXTERNAL_TESTS
    fips                [default]        
    fips-securitychecks [cascade]        OPENSSL_NO_FIPS_SECURITYCHECKS
    fuzz-afl            [default]        OPENSSL_NO_FUZZ_AFL
    fuzz-libfuzzer      [default]        OPENSSL_NO_FUZZ_LIBFUZZER
    ktls                [default]        OPENSSL_NO_KTLS
    msan                [default]        OPENSSL_NO_MSAN
    sctp                [default]        OPENSSL_NO_SCTP
    tfo                 [default]        OPENSSL_NO_TFO
    ubsan               [default]        OPENSSL_NO_UBSAN
    unit-test           [default]        OPENSSL_NO_UNIT_TEST
    uplink              [no uplink_arch] OPENSSL_NO_UPLINK
    zstd                [default]        OPENSSL_NO_ZSTD
    zstd-dynamic        [default]        OPENSSL_NO_ZSTD_DYNAMIC
    ssl3                [default]        OPENSSL_NO_SSL3
    ssl3-method         [default]        OPENSSL_NO_SSL3_METHOD

Config target attributes:

    AR => "ar",
    ARFLAGS => "qc",
    CC => "cc",
    CFLAGS => "-g -O0 -Wall",
    HASHBANGPERL => "/usr/bin/env perl",
    RANLIB => "ranlib -c",
    RC => "windres",
    asm_arch => "aarch64",
    bn_ops => "SIXTY_FOUR_BIT_LONG",
    build_file => "Makefile",
    build_scheme => [ "unified", "unix" ],
    cflags => "-arch arm64",
    cppflags => "-D_REENTRANT",
    defines => [ "OPENSSL_BUILDING_OPENSSL", "ZLIB", "ZLIB_SHARED" ],
    disable => [  ],
    dso_scheme => "dlfcn",
    enable => [  ],
    ex_libs => "",
    includes => [  ],
    lflags => "-Wl,-search_paths_first",
    lib_cflags => "",
    lib_cppflags => "-DL_ENDIAN",
    lib_defines => [  ],
    module_cflags => "-fPIC",
    module_cxxflags => undef,
    module_ldflags => "-bundle",
    perl_platform => "Unix",
    perlasm_scheme => "ios64",
    shared_cflag => "-fPIC",
    shared_defines => [  ],
    shared_extension => ".\$(SHLIB_VERSION_NUMBER).dylib",
    shared_ldflag => "-dynamiclib -current_version \$(SHLIB_VERSION_NUMBER) -compatibility_version \$(SHLIB_VERSION_NUMBER)",
    shared_rcflag => "",
    shared_sonameflag => "-install_name \$(libdir)/",
    shared_target => "darwin-shared",
    sys_id => "MACOSX",
    thread_defines => [  ],
    thread_scheme => "pthreads",
    unistd => "<unistd.h>",

Recorded environment:

    AR = 
    ARFLAGS = 
    AS = 
    ASFLAGS = 
    BUILDFILE = 
    CC = clang
    CFLAGS = -O3 -std=gnu18 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk -I//Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include -g
    CPP = 
    CPPDEFINES = 
    CPPFLAGS = 
    CPPINCLUDES = 
    CROSS_COMPILE = 
    CXX = clang++
    CXXFLAGS = -std=gnu++20 -O3 -stdlib=libc++ -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk
    HASHBANGPERL = 
    LD = 
    LDFLAGS = 
    LDLIBS = 
    MT = 
    MTFLAGS = 
    OPENSSL_LOCAL_CONFIG_DIR = 
    PERL = 
    RANLIB = 
    RC = 
    RCFLAGS = 
    RM = 
    WINDRES = 
    __CNF_CFLAGS = 
    __CNF_CPPDEFINES = 
    __CNF_CPPFLAGS = 
    __CNF_CPPINCLUDES = 
    __CNF_CXXFLAGS = 
    __CNF_LDFLAGS = 
    __CNF_LDLIBS = 

Makevars:

    AR              = ar
    ARFLAGS         = qc
    ASFLAGS         = 
    CC              = clang
    CFLAGS          = -O3 -std=gnu18 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk -I//Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include -g
    CPPDEFINES      = 
    CPPFLAGS        = 
    CPPINCLUDES     = 
    CXX             = clang++
    CXXFLAGS        = -std=gnu++20 -O3 -stdlib=libc++ -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk
    HASHBANGPERL    = /usr/bin/env perl
    LDFLAGS         = 
    LDLIBS          = 
    PERL            = /opt/local/bin/perl5.34
    RANLIB          = ranlib -c
    RC              = windres
    RCFLAGS         = 

NOTE: These variables only represent the configuration view.  The build file
template may have processed these variables further, please have a look at the
build file for more exact data:
    Makefile

build file:

    Makefile

build file templates:

    Configurations/common0.tmpl
    Configurations/unix-Makefile.tmpl

[jamuir]

I can confirm this.

test_evp_extra fails intermittently for me on Apple M2 Pro.

I repeatedly ran the test like so:

#!/bin/bash
x=0
set -e
while [ true ]; do
  let x=x+1
  echo $x
  make TESTS='test_evp_extra' test &> sm2-log.txt
done

It usually fails quickly (after five or less iterations), but occasionally takes longer (e.g. 27 iterations).

Unfortunately, the error is not the same each time.

[jamuir]

Here is a log showing different line numbers for the test failure:

% cat test.bash
x=0
set -e
while [ true ]; do
    let x=x+1
    echo $x
    make TESTS='test_evp_extra' test &> sm2-log.txt
done
set +e
% bash test.bash; grep ERROR sm2-log.txt
1
2
3
4
5
6
7
8
    # ERROR: (int) 'EVP_PKEY_decrypt(cctx, plaintext, &ptext_len, ciphertext, ctext_len) > 0' failed @ test/evp_extra_test.c:2306
% bash test.bash; grep ERROR sm2-log.txt
1
2
3
4
5
6
7
8
9
10
    # ERROR: (int) 'EVP_DigestVerifyFinal(md_ctx_verify, sig, sig_len) > 0' failed @ test/evp_extra_test.c:2243
% bash test.bash; grep ERROR sm2-log.txt
1
2
    # ERROR: (int) 'EVP_DigestVerifyFinal(mctx, signature, sizeof(signature)) > 0' failed @ test/evp_extra_test.c:2133
% bash test.bash; grep ERROR sm2-log.txt
1
2
3
4
    # ERROR: (int) 'EVP_PKEY_decrypt(cctx, plaintext, &ptext_len, ciphertext, ctext_len) > 0' failed @ test/evp_extra_test.c:2306

[t8m]

I assume if you build with no-asm the issue disappears?

[t8m]

Also do I assume correctly that this issue is not present on the 3.2 branch?

[mouse07410]

I assume if you build with no-asm the issue disappears?

Haven't tried.

Also do I assume correctly that this issue is not present on the 3.2 branch?

I install OpenSSL-3.2 binaries-only (system-wide via Macports), so I don't run any tests on it.

[jamuir]

Also do I assume correctly that this issue is not present on the 3.2 branch?

No, that is not correct.

I am able to reproduce it against openssl-3.2:

% git log --oneline -n1
e554f9f249 (HEAD -> openssl-3.2, origin/openssl-3.2) Uninitialized array variable
% bash test.bash; grep ERROR sm2-log.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
    # ERROR: (int) 'EVP_DigestVerifyFinal(mctx, signature, sizeof(signature)) > 0' failed @ test/evp_extra_test.c:2133

[tom-cosgrove-arm]

@docularxu Could this be related to something in #20754?

[jamuir]

I assume if you build with no-asm the issue disappears?

Yes, the failure appears to go away if you build with no-asm.

I have done over 60 iterations without hitting it.

With no-asm, that test is painfully slow.

[t8m]

@jamuir Could you please try to build instead of no-asm with no-sm2-precomp?

[t8m]

@xu-yi-zhou Any idea if #20754 could be causing this?

[jamuir]

@jamuir Could you please try to build instead of no-asm with no-sm2-precomp?

It still fails:

% cat doit.bash
./Configure -pedantic -Werror --strict-warnings --debug enable-trace no-sm2-precomp
make -j6
# make test
% source doit.bash &> log.txt
% bash test.bash; grep ERROR sm2-log.txt
1
2
    # ERROR: (int) 'EVP_PKEY_decrypt(cctx, plaintext, &ptext_len, ciphertext, ctext_len) > 0' failed @ test/evp_extra_test.c:2306

[xu-yi-zhou]

Thanks for the report, I'm trying to reproduce this issue.