Cannot set --openssldir to Windows path when cross compiling on Linux

[ethanreece-rtechnics]

Issue

I am trying to cross compile OpenSSL for Windows on Linux, but when I set set --openssldir to a Windows path, it appends it to the --prefix Unix path and fails.

Commands and output

Configure command: ./Configure --cross-compile-prefix=/usr/bin/i686-w64-mingw32- mingw --prefix=$(pwd)/build --openssldir="\"C:/Program Files/Common Files/SSL\""

(I have also tried ./Configure --cross-compile-prefix=/usr/bin/i686-w64-mingw32- mingw --prefix=$(pwd)/build --openssldir='C:\\Program Files\\Common Files\\SSL\\')

perl configdata.pm --dump

Command line (with current working directory = .):

    /usr/bin/perl ./Configure --cross-compile-prefix=/usr/bin/i686-w64-mingw32- mingw --prefix=/mnt/d/openssl/openssl/build --openssldir="C:/Program Files/Common Files/SSL"

Perl information:

    /usr/bin/perl
    5.32.1 for x86_64-linux-gnu-thread-multi

Enabled features:

    apps
    argon2
    aria
    asm
    async
    autoalginit
    autoerrinit
    autoload-config
    bf
    blake2
    bulk
    cached-fetch
    camellia
    capieng
    winstore
    cast
    chacha
    cmac
    cmp
    cms
    comp
    ct
    default-thread-pool
    deprecated
    des
    dgram
    dh
    docs
    dsa
    dso
    dtls
    dynamic-engine
    ec
    ec2m
    ecdh
    ecdsa
    ecx
    engine
    err
    filenames
    gost
    http
    idea
    legacy
    loadereng
    makedepend
    md4
    mdc2
    module
    multiblock
    nextprotoneg
    ocb
    ocsp
    padlockeng
    pic
    pinshared
    poly1305
    posix-io
    psk
    rc2
    rc4
    rdrand
    rfc3779
    rmd160
    scrypt
    secure-memory
    seed
    shared
    siphash
    siv
    sm2
    sm3
    sm4
    sock
    srp
    srtp
    sse2
    ssl
    ssl-trace
    static-engine
    stdio
    tests
    thread-pool
    threads
    tls
    ts
    ui-console
    uplink
    whirlpool
    tls1
    tls1-method
    tls1_1
    tls1_1-method
    tls1_2
    tls1_2-method
    tls1_3
    dtls1
    dtls1-method
    dtls1_2
    dtls1_2-method

Disabled features:

    acvp-tests          [cascade]   OPENSSL_NO_ACVP_TESTS
    afalgeng            [not-linux] OPENSSL_NO_AFALGENG
    asan                [default]   OPENSSL_NO_ASAN
    brotli              [default]   OPENSSL_NO_BROTLI
    brotli-dynamic      [default]   OPENSSL_NO_BROTLI_DYNAMIC
    buildtest-c++       [default]
    crypto-mdebug       [default]   OPENSSL_NO_CRYPTO_MDEBUG
    devcryptoeng        [default]   OPENSSL_NO_DEVCRYPTOENG
    ec_nistp_64_gcc_128 [default]   OPENSSL_NO_EC_NISTP_64_GCC_128
    egd                 [default]   OPENSSL_NO_EGD
    external-tests      [default]   OPENSSL_NO_EXTERNAL_TESTS
    fips                [default]
    fips-securitychecks [cascade]   OPENSSL_NO_FIPS_SECURITYCHECKS
    fuzz-afl            [default]   OPENSSL_NO_FUZZ_AFL
    fuzz-libfuzzer      [default]   OPENSSL_NO_FUZZ_LIBFUZZER
    ktls                [default]   OPENSSL_NO_KTLS
    md2                 [default]   OPENSSL_NO_MD2 (skip crypto/md2)
    msan                [default]   OPENSSL_NO_MSAN
    quic                [default]   OPENSSL_NO_QUIC
    rc5                 [default]   OPENSSL_NO_RC5 (skip crypto/rc5)
    sctp                [default]   OPENSSL_NO_SCTP
    tfo                 [default]   OPENSSL_NO_TFO
    trace               [default]   OPENSSL_NO_TRACE
    ubsan               [default]   OPENSSL_NO_UBSAN
    unit-test           [default]   OPENSSL_NO_UNIT_TEST
    weak-ssl-ciphers    [default]   OPENSSL_NO_WEAK_SSL_CIPHERS
    zlib                [default]   OPENSSL_NO_ZLIB
    zlib-dynamic        [default]   OPENSSL_NO_ZLIB_DYNAMIC
    zstd                [default]   OPENSSL_NO_ZSTD
    zstd-dynamic        [default]   OPENSSL_NO_ZSTD_DYNAMIC
    ssl3                [default]   OPENSSL_NO_SSL3
    ssl3-method         [default]   OPENSSL_NO_SSL3_METHOD

Config target attributes:

    AR => "ar",
    ARFLAGS => "qc",
    CC => "gcc",
    CFLAGS => "-Wall -O3 -fomit-frame-pointer",
    HASHBANGPERL => "/usr/bin/env perl",
    RANLIB => "",
    RC => "windres",
    asm_arch => "x86",
    bn_ops => "BN_LLONG",
    build_file => "Makefile",
    build_scheme => [ "unified", "unix" ],
    cflags => "-m32",
    cppflags => "-DUNICODE -D_UNICODE -DWIN32_LEAN_AND_MEAN -D_MT",
    defines => [ "OPENSSL_BUILDING_OPENSSL" ],
    disable => [  ],
    dso_scheme => "win32",
    enable => [  ],
    ex_libs => "-lws2_32 -lgdi32 -lcrypt32",
    includes => [  ],
    lflags => "",
    lib_cflags => "",
    lib_cppflags => "-DL_ENDIAN",
    lib_defines => [  ],
    module_cflags => "",
    module_cxxflags => undef,
    module_ldflags => "-static-libgcc -shared -Wl,--enable-auto-image-base",
    multilib => "",
    perl_platform => "mingw",
    perlasm_scheme => "coff",
    shared_argfileflag => "\@",
    shared_cflag => "",
    shared_cppflags => "_WINDLL",
    shared_defflag => "",
    shared_defines => [  ],
    shared_impflag => "-Wl,--out-implib=",
    shared_ldflag => "-static-libgcc -shared -Wl,--enable-auto-image-base",
    shared_rcflag => "--target=pe-i386",
    shared_target => "mingw-shared",
    sys_id => "MINGW32",
    thread_defines => [  ],
    thread_scheme => "winthreads",
    unistd => "<unistd.h>",
    uplink_arch => "x86",

Recorded environment:

    AR =
    ARFLAGS =
    AS =
    ASFLAGS =
    BUILDFILE =
    CC =
    CFLAGS =
    CPP =
    CPPDEFINES =
    CPPFLAGS =
    CPPINCLUDES =
    CROSS_COMPILE =
    CXX =
    CXXFLAGS =
    HASHBANGPERL =
    LD =
    LDFLAGS =
    LDLIBS =
    MT =
    MTFLAGS =
    OPENSSL_LOCAL_CONFIG_DIR =
    PERL =
    RANLIB =
    RC =
    RCFLAGS =
    RM =
    WINDRES =
    __CNF_CFLAGS =
    __CNF_CPPDEFINES =
    __CNF_CPPFLAGS =
    __CNF_CPPINCLUDES =
    __CNF_CXXFLAGS =
    __CNF_LDFLAGS =
    __CNF_LDLIBS =

Makevars:

    AR              = /usr/bin/i686-w64-mingw32-ar
    ARFLAGS         = qc
    ASFLAGS         =
    CC              = /usr/bin/i686-w64-mingw32-gcc
    CFLAGS          = -Wall -O3 -fomit-frame-pointer
    CPPDEFINES      =
    CPPFLAGS        =
    CPPINCLUDES     =
    CROSS_COMPILE   = /usr/bin/i686-w64-mingw32-
    CXXFLAGS        =
    HASHBANGPERL    = /usr/bin/env perl
    LDFLAGS         =
    LDLIBS          =
    PERL            = /usr/bin/perl
    RC              = /usr/bin/i686-w64-mingw32-windres
    RCFLAGS         =

NOTE: These variables only represent the configuration view.  The build file
template may have processed these variables further, please have a look at the
build file for more exact data:
    Makefile

build file:

    Makefile

build file templates:

    Configurations/common0.tmpl
    Configurations/unix-Makefile.tmpl

make:

/usr/bin/perl "-I." "-Iutil/perl" "-Mconfigdata" "-MOpenSSL::paramnames" "util/dofile.pl" "-oMakefile" crypto/params_idx.c.in > crypto/params_idx.c
/usr/bin/perl "-I." "-Mconfigdata" "util/dofile.pl" "-oMakefile" include/crypto/bn_conf.h.in > include/crypto/bn_conf.h
/usr/bin/perl "-I." "-Mconfigdata" "util/dofile.pl" "-oMakefile" include/crypto/dso_conf.h.in > include/crypto/dso_conf.h
/usr/bin/perl "-I." "-Iutil/perl" "-Mconfigdata" "-MOpenSSL::paramnames" "util/dofile.pl" "-oMakefile" include/internal/param_names.h.in > include/internal/param_names.h
/usr/bin/perl "-I." "-Mconfigdata" "util/dofile.pl" "-oMakefile" include/openssl/asn1.h.in > include/openssl/asn1.h
/usr/bin/perl "-I." "-Mconfigdata" "util/dofile.pl" "-oMakefile" include/openssl/asn1t.h.in > include/openssl/asn1t.h
/usr/bin/perl "-I." "-Mconfigdata" "util/dofile.pl" "-oMakefile" include/openssl/bio.h.in > include/openssl/bio.h
/usr/bin/perl "-I." "-Mconfigdata" "util/dofile.pl" "-oMakefile" include/openssl/cmp.h.in > include/openssl/cmp.h
/usr/bin/perl "-I." "-Mconfigdata" "util/dofile.pl" "-oMakefile" include/openssl/cms.h.in > include/openssl/cms.h
/usr/bin/perl "-I." "-Mconfigdata" "util/dofile.pl" "-oMakefile" include/openssl/conf.h.in > include/openssl/conf.h
/usr/bin/perl "-I." "-Iutil/perl" "-Mconfigdata" "-MOpenSSL::paramnames" "util/dofile.pl" "-oMakefile" include/openssl/core_names.h.in > include/openssl/core_names.h
/usr/bin/perl "-I." "-Mconfigdata" "util/dofile.pl" "-oMakefile" include/openssl/crmf.h.in > include/openssl/crmf.h
/usr/bin/perl "-I." "-Mconfigdata" "util/dofile.pl" "-oMakefile" include/openssl/crypto.h.in > include/openssl/crypto.h
/usr/bin/perl "-I." "-Mconfigdata" "util/dofile.pl" "-oMakefile" include/openssl/ct.h.in > include/openssl/ct.h
/usr/bin/perl "-I." "-Mconfigdata" "util/dofile.pl" "-oMakefile" include/openssl/err.h.in > include/openssl/err.h
/usr/bin/perl "-I." "-Mconfigdata" "util/dofile.pl" "-oMakefile" include/openssl/ess.h.in > include/openssl/ess.h
/usr/bin/perl "-I." "-Mconfigdata" "util/dofile.pl" "-oMakefile" include/openssl/fipskey.h.in > include/openssl/fipskey.h
/usr/bin/perl "-I." "-Mconfigdata" "util/dofile.pl" "-oMakefile" include/openssl/lhash.h.in > include/openssl/lhash.h
/usr/bin/perl "-I." "-Mconfigdata" "util/dofile.pl" "-oMakefile" include/openssl/ocsp.h.in > include/openssl/ocsp.h
/usr/bin/perl "-I." "-Mconfigdata" "util/dofile.pl" "-oMakefile" include/openssl/opensslv.h.in > include/openssl/opensslv.h
/usr/bin/perl "-I." "-Mconfigdata" "util/dofile.pl" "-oMakefile" include/openssl/pkcs12.h.in > include/openssl/pkcs12.h
/usr/bin/perl "-I." "-Mconfigdata" "util/dofile.pl" "-oMakefile" include/openssl/pkcs7.h.in > include/openssl/pkcs7.h
/usr/bin/perl "-I." "-Mconfigdata" "util/dofile.pl" "-oMakefile" include/openssl/safestack.h.in > include/openssl/safestack.h
/usr/bin/perl "-I." "-Mconfigdata" "util/dofile.pl" "-oMakefile" include/openssl/srp.h.in > include/openssl/srp.h
/usr/bin/perl "-I." "-Mconfigdata" "util/dofile.pl" "-oMakefile" include/openssl/ssl.h.in > include/openssl/ssl.h
/usr/bin/perl "-I." "-Mconfigdata" "util/dofile.pl" "-oMakefile" include/openssl/ui.h.in > include/openssl/ui.h
/usr/bin/perl "-I." "-Mconfigdata" "util/dofile.pl" "-oMakefile" include/openssl/x509.h.in > include/openssl/x509.h
/usr/bin/perl "-I." "-Mconfigdata" "util/dofile.pl" "-oMakefile" include/openssl/x509_vfy.h.in > include/openssl/x509_vfy.h
/usr/bin/perl "-I." "-Mconfigdata" "util/dofile.pl" "-oMakefile" include/openssl/x509v3.h.in > include/openssl/x509v3.h
/usr/bin/perl "-I." "-Mconfigdata" "util/dofile.pl" "-oMakefile" test/provider_internal_test.cnf.in > test/provider_internal_test.cnf
make depend && make _build_sw
make[1]: Entering directory '/mnt/d/openssl/openssl'
make[1]: Leaving directory '/mnt/d/openssl/openssl'
make[1]: Entering directory '/mnt/d/openssl/openssl'
/usr/bin/i686-w64-mingw32-gcc  -I. -Iinclude -Iapps/include  -m32 -Wall -O3 -fomit-frame-pointer -DL_ENDIAN -DOPENSSL_PIC -DOPENSSLDIR="\"/mnt/d/openssl/openssl/build/"C:/Program Files/Common Files/SSL"\"" -DENGINESDIR="\"/mnt/d/openssl/openssl/build/lib/engines-3\"" -DMODULESDIR="\"/mnt/d/openssl/openssl/build/lib/ossl-modules\"" -DUNICODE -D_UNICODE -DWIN32_LEAN_AND_MEAN -D_MT -DOPENSSL_BUILDING_OPENSSL -DNDEBUG  -MMD -MF apps/lib/libapps-lib-app_libctx.d.tmp -MT apps/lib/libapps-lib-app_libctx.obj -c -o apps/lib/libapps-lib-app_libctx.obj apps/lib/app_libctx.c
i686-w64-mingw32-gcc: error: Files/Common: No such file or directory
i686-w64-mingw32-gcc: error: Files/SSL": No such file or directory
make[1]: *** [Makefile:5116: apps/lib/libapps-lib-app_libctx.obj] Error 1
make[1]: Leaving directory '/mnt/d/openssl/openssl'
make: *** [Makefile:3436: build_sw] Error 2

Tested Platforms

This fails on Ubuntu on GitHub Actions and Debian on WSL2.

Workarounds

Using --openssldir="/Program Files/Common Files/SSL/" works and appears to read from the corresponding Windows directory (although by default it uses the --prefix directory which seems like a security issue). Also, the openssl version -d command gives the Unix path output, which is not ideal.

Expected behavior

It compiles it with the Windows path, and openssl version -d outputs a Windows path.

[t8m]

It is wrong to use --prefix the way you're using it. Do you actually want to specify DESTDIR when running make install instead?

[ethanreece-rtechnics]

It is wrong to use --prefix the way you're using it. Do you actually want to specify DESTDIR when running make install instead?

Probably, although how do I do that?

For ./Configure --cross-compile-prefix=/usr/bin/i686-w64-mingw32- mingw --openssldir="\"C:/Program Files/Common Files/SSL\"", running make uses -DOPENSSLDIR="\"/usr/local/"C:/Program Files/Common Files/SSL"\"" which fails.

For ./Configure --cross-compile-prefix=/usr/bin/i686-w64-mingw32- mingw --prefix="\"C:/Program Files/OpenSSL\"" --openssldir="\"C:/Program Files/Common Files/SSL\"", the command immediately fails saying Directory given with --prefix MUST be absolute.

[bernd-edlinger]

Maybe you could try to work around this by using
--prefix="//./C:/Program Files/OpenSSL" --openssldir="//./C:/Program Files/Common Files/SSL"

[vszakats]

An ages old issue, ref #9520.

Patch was offered for this but rejected on bureaucratic grounds: #16445

It's a two-liner fix, including the closing curly-brace. Apparently "non-trivial".

[t8m]

Patch was offered for this but rejected on bureaucratic grounds: #16445

I am sorry but you've closed the PR. I could have amended the patch for you if you asked me to do that as you already have an ICLA signed.

[vszakats]

I closed the patch because after me stepping up to fix this issue (after OpenSSL ignoring it for years) you requested a change, then followed-up by promoting the patch to "non-trivial" (a rather unfriendly step, I felt), then requested to sign a new CLA because my commit email address changed. This felt unreasonable and unacceptable to me, esp considering how petty (yet annoying) bug this is.

Please feel free to commit this under anybody's name, I release this patch to the public domain (I might have already done so back then).

Aside: OpenSSL is also apparently not willing to address the underlying issue of using a vulnerable openssldir by default. 3.1.3 fixed the no-autoload-config option, but it remains murky if OpenSSL is still trying to pick up DLLs or configuration from an inherently insecure (world-writable) baked-in openssldir location on Windows.

[t8m]

Fixed by #22723

[vszakats]

Thank you @t8m!

After noticing the patch in 3.2.0, I started using this live. It turned out to be somewhat complicated and fragile, because our merged solution is relying on filling CROSS_COMPILE (which was so far not mandatory for cross-builds). This is what settled it for curl builds:

export CROSS_COMPILE
CROSS_COMPILE="$(dirname "$(command -v "$(echo "${CC}" | cut -d ' ' -f 1)")")/"

A possible improvement is to make the absolute path detection function enforce the rules for the target OS instead of the host OS the build is running on. I have no patch ready for this though. (The build knows we're building for Windows, so the necessary information should be there.)

The reason to change --prefix for Windows builds, is that the default used by OpenSSL is world-writable on a sizable portion of Windows installs (i.e. localized ones, where C:\Program Files*\ isn't present). As suggested earlier C:\WINDOWS\... would cover more installs. The cover all installs, runtime detection of the Windows (or Program Files) directory would be necessary, because on Windows there is no universally protected path that OpenSSL can safely hardcode into its binary at build-time.