[BUG] - Github Actions CI for Windows is broken

[newhoggy]

Internal/External
Internal if an IOHK staff member.

Area
CI

Summary
The Github Actions CI for Windows has stopped working

Steps to reproduce
Create a PR and observe the Windows CI is broken.

The error in the "Build" step is as follows:

Failed to build HsOpenSSL-0.11.7.2. The failure occurred during the configure
step.
Build log (
C:\cabal\logs\ghc-8.10.7\HsOpenSSL-0.11.7.2-20704ddca06ada5f5f5da43cd972c8ee33724e64.log
):
[1 of 1] Compiling Main             ( D:\a\cardano-node\cardano-node\dist-newstyle\tmp\src-2804\HsOpenSSL-0.11.7.2\dist\setup\setup.hs, D:\a\cardano-node\cardano-node\dist-newstyle\tmp\src-2804\HsOpenSSL-0.11.7.2\dist\setup\Main.o )
Linking D:\a\cardano-node\cardano-node\dist-newstyle\tmp\src-2804\HsOpenSSL-0.11.7.2\dist\setup\setup.exe ...
cabal-3.6.2.0.exe: Failed to build HsOpenSSL-0.11.7.2 (which is required by
test:chairman-tests from cardano-node-chairman-1.36.0, test:cardano-node-test
Configuring HsOpenSSL-0.11.7.2...
from cardano-node-1.36.0 and others). See the build log above for details.
setup.exe: Missing dependencies on foreign libraries:

* Missing (or bad) header file: openssl/asn1.h
* Missing (or bad) C libraries: ssl, crypto
This problem can usually be solved by installing the system packages that
provide these libraries (you may need the "-dev" versions). If the libraries
are already installed but in a non-standard location then you can use the
flags --extra-include-dirs= and --extra-lib-dirs= to specify where they are.If
the library files do exist, it may contain errors that are caught by the C
compiler at the preprocessing stage. In this case you can re-run configure
with the verbosity flag -v3 to see the error messages.
If the header file does exist, it may contain errors that are caught by the C
compiler at the preprocessing stage. In this case you can re-run configure
with the verbosity flag -v3 to see the error messages.

Expected behavior
It should pass

System info (please complete the following information):

• OS Name: Windows
• OS Version N/A
• Node version N/A
• CLI version N/A

[newhoggy]

The error can also manifest in this manner:

Failed to build secp256k1-haskell-0.6.1. The failure occurred during the
configure step.
Build log (
C:\cabal\logs\ghc-8.10.7\secp256k1-has_-0.6.1-685d43eb3dd44322adb734f9f7d60d64cfdc366d.log
):
Configuring library for secp256k1-haskell-0.6.1..
cabal-3.6.2.0.exe: Missing dependency on a foreign library:
* Missing (or bad) C library: secp256k1
This problem can usually be solved by installing the system package that
provides this library (you may need the "-dev" version). If the library is
cabal-3.6.2.0.exe: Failed to build secp256k1-haskell-0.6.1 (which is required
by cardano-ledger-babbage-test-0.1.0.0 and cardano-crypto-tests-2.0.0). See
already installed but in a non-standard location then you can use the flags
the build log above for details.
--extra-include-dirs= and --extra-lib-dirs= to specify where it is.If the

library file does exist, it may contain errors that are caught by the C
compiler at the preprocessing stage. In this case you can re-run configure
with the verbosity flag -v3 to see the error messages.

See https://github.com/input-output-hk/ouroboros-network/actions/runs/3387418765/jobs/5628140073

[newhoggy]

Checking how secp256k is installed in the "Install secp256k1" step for Windows:

libtool: install: /usr/bin/install -c .libs/libsecp256k1.dll.a /mingw64/lib/libsecp256k1.dll.a
libtool: install: base_file=`basename libsecp256k1.la`
libtool: install:  dlpath=`/bin/sh 2>&1 -c '. .libs/'libsecp256k1.la'i; echo libsecp256k1-0.dll'`
libtool: install:  dldir=/mingw64/lib/`dirname ../bin/libsecp256k1-0.dll`
libtool: install:  test -d /mingw64/lib/../bin || mkdir -p /mingw64/lib/../bin
libtool: install:  /usr/bin/install -c .libs/libsecp256k1-0.dll /mingw64/lib/../bin/libsecp256k1-0.dll
libtool: install:  chmod a+x /mingw64/lib/../bin/libsecp256k1-0.dll
libtool: install:  if test -n '' && test -n 'strip --strip-unneeded'; then eval 'strip --strip-unneeded /mingw64/lib/../bin/libsecp256k1-0.dll' || exit 0; fi
libtool: install: /usr/bin/install -c .libs/libsecp256k1.lai /mingw64/lib/libsecp256k1.la
libtool: install: /usr/bin/install -c .libs/libsecp256k1.a /mingw64/lib/libsecp256k1.a
libtool: install: chmod 644 /mingw64/lib/libsecp256k1.a
libtool: install: /mingw64/bin/ranlib /mingw64/lib/libsecp256k1.a
 /usr/bin/mkdir -p '/mingw64/include'
 /usr/bin/install -c -m 644 include/secp256k1.h include/secp256k1_preallocated.h include/secp256k1_extrakeys.h include/secp256k1_schnorrsig.h '/mingw64/include'
 /usr/bin/mkdir -p '/mingw64/lib/pkgconfig'
 /usr/bin/install -c -m 644 libsecp256k1.pc '/mingw64/lib/pkgconfig'
make[2]: Leaving directory '/d/a/ouroboros-network/ouroboros-network/secp256k1'
make[1]: Leaving directory '/d/a/ouroboros-network/ouroboros-network/secp256k1'

The install directories are:

/mingw64/include
/mingw64/lib/pkgconfig
/mingw64/include

These are POSIX paths but don't really indicate where the files are on the Windows filesystem.

After some searching I found that they live here:

D:/a/_temp/msys64/mingw64/lib/pkgconfig/expat.pc
D:/a/_temp/msys64/mingw64/lib/pkgconfig/formw.pc
D:/a/_temp/msys64/mingw64/lib/pkgconfig/gmp.pc
D:/a/_temp/msys64/mingw64/lib/pkgconfig/gmpxx.pc
D:/a/_temp/msys64/mingw64/lib/pkgconfig/gnurx.pc
D:/a/_temp/msys64/mingw64/lib/pkgconfig/iconv.pc
D:/a/_temp/msys64/mingw64/lib/pkgconfig/isl.pc
D:/a/_temp/msys64/mingw64/lib/pkgconfig/libcrypto.pc
D:/a/_temp/msys64/mingw64/lib/pkgconfig/libffi.pc
D:/a/_temp/msys64/mingw64/lib/pkgconfig/liblzma.pc
D:/a/_temp/msys64/mingw64/lib/pkgconfig/libpkgconf.pc
D:/a/_temp/msys64/mingw64/lib/pkgconfig/libsecp256k1.pc
D:/a/_temp/msys64/mingw64/lib/pkgconfig/libssl.pc
D:/a/_temp/msys64/mingw64/lib/pkgconfig/libtasn1.pc
D:/a/_temp/msys64/mingw64/lib/pkgconfig/libxxhash.pc
D:/a/_temp/msys64/mingw64/lib/pkgconfig/libzstd.pc
D:/a/_temp/msys64/mingw64/lib/pkgconfig/menuw.pc
D:/a/_temp/msys64/mingw64/lib/pkgconfig/mpfr.pc
D:/a/_temp/msys64/mingw64/lib/pkgconfig/ncurses++w.pc
D:/a/_temp/msys64/mingw64/lib/pkgconfig/ncursesw.pc
D:/a/_temp/msys64/mingw64/lib/pkgconfig/oniguruma.pc
D:/a/_temp/msys64/mingw64/lib/pkgconfig/openssl.pc
D:/a/_temp/msys64/mingw64/lib/pkgconfig/p11-kit-1.pc
D:/a/_temp/msys64/mingw64/lib/pkgconfig/panelw.pc
D:/a/_temp/msys64/mingw64/lib/pkgconfig/python-3.10-embed.pc
D:/a/_temp/msys64/mingw64/lib/pkgconfig/python-3.10.pc
D:/a/_temp/msys64/mingw64/lib/pkgconfig/python3-embed.pc
D:/a/_temp/msys64/mingw64/lib/pkgconfig/python3.pc
D:/a/_temp/msys64/mingw64/lib/pkgconfig/readline.pc
D:/a/_temp/msys64/mingw64/lib/pkgconfig/regex.pc
D:/a/_temp/msys64/mingw64/lib/pkgconfig/sqlite3.pc
D:/a/_temp/msys64/mingw64/lib/pkgconfig/termcap.pc
D:/a/_temp/msys64/mingw64/lib/pkgconfig/tre.pc
D:/a/_temp/msys64/mingw64/lib/pkgconfig/zlib.pc

From this I deduce that /mingw64 corresponds to D:/a/_temp/msys64/mingw64 on the Windows filesystem.

This means the files are installed to these locations.

D:/a/_temp/msys64/mingw64/include
D:/a/_temp/msys64/mingw64/lib
D:/a/_temp/msys64/mingw64/lib/pkgconfig

[newhoggy]

Then I noticed this like in the "WIN: Setup Haskell" step:

# Modified cabal config extra-lib-dirs
extra-lib-dirs: C:\ghcup\ghc\8.10.7\mingw\x86_64-w64-mingw32\lib, C:\msys64\mingw64\lib

See https://github.com/newhoggy/secp256k1-haskell/actions/runs/3385765285/jobs/5624303772

This shows that there another location where system libraries are installed on the C drive:

C:\msys64\mingw64\include
C:\msys64\mingw64\lib
C:\msys64\mingw64\lib\pkgconfig

These are the directories that cabal is actually configured to use.

[newhoggy]

I tried solving the problem by telling cabal where the D: installation location like this:

package secp256k1-haskell
  extra-include-dirs: D:/a/_temp/msys64/mingw64/include
  extra-lib-dirs: D:/a/_temp/msys64/mingw64/lib

Using this method, the secp256k1-haskell library builds fine:

https://github.com/newhoggy/secp256k1-haskell/actions/runs/3385490495/jobs/5623693479

[newhoggy]

I tried an alternate way to fix the problem:

    - name: Setup additional gcc paths
      run: |
        echo "C_INCLUDE_PATH=D:/a/_temp/msys64/mingw64/include;$C_INCLUDE_PATH" >> $GITHUB_ENV
        echo "LIBRARY_PATH=D:/a/_temp/msys64/mingw64/lib;$LIBRARY_PATH" >> $GITHUB_ENV

This failed in this manner:

db stack: ["C:\\cabal\\store\\ghc-8.10.7\\package.db","C:\\Users\\runneradmin\\AppData\\Roaming\\ghc\\x86_64-mingw32-8.10.7\\package.conf.d","C:\\ghcup\\ghc\\8.10.7\\lib\\package.conf.d"]
modifying: C:\cabal\store\ghc-8.10.7\package.db
flag db stack: ["C:\\cabal\\store\\ghc-8.10.7\\package.db"]
writing cache C:\cabal\store\ghc-8.10.7\package.db\package.cache
Installed store entry
ghc-8.10.7\unliftio-core-0.2.0.1-812e2f03ca36773ac83ffa358a6780387cd3061d
Completed    unliftio-core-0.2.0.1 (lib)
packages definitely up to date: 
packages previously probably up to date: secp256k1-haskell-0.6.1-inplace, secp256k1-haskell-0.6.1-inplace-spec
packages now probably up to date: 
packages newly up to date: 
packages out to date: secp256k1-haskell-0.6.1-inplace, secp256k1-haskell-0.6.1-inplace-spec
packages invalid due to dep change: QuickCheck-2.14.2-c4024bcba1960207213b5c6afe2164dd5c6ab148, abstract-deque-0.3-f31b7d4360341d561cd810a04a3efa6e112223c2, base16-0.3.2.0-bd4761c7858798c071a6ee8cdfaf220fa639512c, entropy-0.4.1.10-624b7762bff648dccb044307f50c0f986fa25a1a, hashable-1.4.1.0-f678339d380c1ae95524eb8bee8b65e2b5462218, hspec-2.10.6-69ae98da7194d21120dbd639caf320585d173bdc, hspec-core-2.10.6-b9010f80a1388ddd4675a3b967e3de6e949ba94e, math-functions-0.3.4.2-440f1e6012ad5bf637f9a3e8e842212c96e7d64b, monad-par-0.3.5-fdca88dec172d36828cd8a2b1d6fae2773473350, monad-par-ext_-0.3.3-451c268446cf3110b95cee847d6d6424246609f2, mwc-random-0.15.0.2-bbda3a23212bc55d8d22f5351c47333065d776b5, quickcheck-io-0.2.0-ef8b4739e20ac41c72e22247906b685ffcff910b, random-1.2.1.1-153631b804c32218f972326bc1df71a326b08f87, secp256k1-haskell-0.6.1-inplace, secp256k1-haskell-0.6.1-inplace-spec, text-short-0.1.5-6f496764959f28a04e11eed9ea133fa5c34d17df, tf-random-0.5-1148d11836c8407c5e84c80f264e065787e7da7e, vector-0.13.0.0-99ac30ff503d66695483f91e08a27a8057cc5bac

packages invalid due to build failure: entropy-0.4.1.10-624b7762bff648dccb044307f50c0f986fa25a1a
cabal-3.6.2.0.exe: Failed to build
entropy-0.4.1.10-624b7762bff648dccb044307f50c0f986fa25a1a. The failure
occurred during the configure step. The exception was:
D:\a\secp256k1-haskell\secp256k1-haskell\dist-newstyle\tmp\src-6100\entropy-0.4.1.10\dist\setup\setup.exe:
createProcess: invalid argument (Exec format error)
Failed to build secp256k1-haskell-0.6.1 because it depends on
secp256k1-haskell-0.6.1 which itself failed to build.

And also this:

    - name: Setup additional gcc paths
      run: |
        echo 'C_INCLUDE_PATH=D:\a\_temp\msys64\mingw64\include;'"$C_INCLUDE_PATH" >> $GITHUB_ENV
        echo 'LIBRARY_PATH=D:\a\_temp\msys64\mingw64\lib;'"$LIBRARY_PATH" >> $GITHUB_ENV

Which fails in this manner:

cabal-3.6.2.0.exe: Failed to build entropy-0.4.1.10 (which is required by
secp256k1-haskell-0.6.1). The failure occurred during the configure step. The
exception was:
D:\a\secp256k1-haskell\secp256k1-haskell\dist-newstyle\tmp\src-4636\entropy-0.4.1.10\dist\setup\setup.exe:
createProcess: invalid argument (Exec format error)