Skip to content

Commit

Permalink
GHA: add MSYS, mingw-w64, Cygwin jobs
Browse files Browse the repository at this point in the history
- re-implement autotools MSYS and Cygwin AppVeyor jobs in GHA.
  Now build with SSL and PSL to improve test coverage.
- re-implement MSYS2 mingw-w64 gcc 13 AppVeyor job in GHA.
  `CMake, mingw-w64, gcc 13, Debug, x64, Schannel, Static, Unicode`
- add new cmake Cygwin job (build-only).
- enable `-j14` parallelism when running tests.
- delete the 5 migrated jobs from AppVeyor CI.
- add 2 build-only mingw-w64 builds, gcc Release and clang OpenSSL.
- also enable brotli, libssh2, nghttp2 for more test coverage.

These jobs offer better performance, more flexibility and
parallelization compared to the AppVeyor ones they replace. It also
offloads AppVeyor, allowing to iterate faster. They also appear more
reliable than e.g. Azure Windows jobs, where runners are prone to fail
[1].

Closes #13599

[1]:
`Exit code 143 returned from process: file name 'C:\Windows\system32\docker.EXE',
arguments 'exec -i   6b13a669c6dfe7fb9f59414369872fd64d61c7182f880c3d39c135cb4c115c8f
C:\__a\externals\node\bin\node.exe C:\__w\_temp\containerHandlerInvoker.js'.`
  • Loading branch information
vszakats committed May 17, 2024
1 parent 3455572 commit 36fd2dd
Show file tree
Hide file tree
Showing 4 changed files with 281 additions and 60 deletions.
2 changes: 1 addition & 1 deletion .github/labeler.yml
Original file line number Diff line number Diff line change
Expand Up @@ -255,4 +255,4 @@ Windows:
- all:
- changed-files:
- any-glob-to-all-files:
- '{appveyor.*,CMake/Platforms/WindowsCache.cmake,lib/*win32*,lib/curl_multibyte.*,lib/rename.*,lib/vtls/schannel*,m4/curl-schannel.m4,projects/**,src/tool_doswin.c,winbuild/**,libcurl.def}'
- '{appveyor.*,.github/workflows/windows.yml,CMake/Platforms/WindowsCache.cmake,lib/*win32*,lib/curl_multibyte.*,lib/rename.*,lib/vtls/schannel*,m4/curl-schannel.m4,projects/**,src/tool_doswin.c,winbuild/**,libcurl.def}'
280 changes: 280 additions & 0 deletions .github/workflows/windows.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,280 @@
# Copyright (C) Viktor Szakats
#
# SPDX-License-Identifier: curl

name: windows

on:
push:
branches:
- master
- '*/ci'
paths-ignore:
- '**/*.md'
- '.azure-pipelines.yml'
- '.circleci/**'
- '.cirrus.yml'
- 'appveyor.*'
- 'packages/**'
- 'plan9/**'
- 'projects/**'
- 'winbuild/**'
pull_request:
branches:
- master
paths-ignore:
- '**/*.md'
- '.azure-pipelines.yml'
- '.circleci/**'
- '.cirrus.yml'
- 'appveyor.*'
- 'packages/**'
- 'plan9/**'
- 'projects/**'
- 'winbuild/**'

concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.sha }}
cancel-in-progress: true

permissions: {}

jobs:
cygwin:
name: "cygwin (${{ matrix.build }}, ${{ matrix.platform }}, ${{ matrix.config }})"
runs-on: windows-latest
timeout-minutes: 45
env:
SHELLOPTS: 'igncr'
strategy:
matrix:
include:
- { build: 'automake', platform: 'x86_64', tflags: '', config: '--enable-debug --disable-threaded-resolver' }
- { build: 'cmake' , platform: 'x86_64', tflags: '', config: '-DCURL_USE_OPENSSL=ON' }
fail-fast: false
steps:
- run: git config --global core.autocrlf input
- uses: actions/checkout@v4
- uses: cygwin/cygwin-install-action@v4
with:
platform: ${{ matrix.platform }}
# https://cygwin.com/cgi-bin2/package-grep.cgi
packages: autoconf libtool ${{ matrix.build }} gcc-core gcc-g++ binutils make libssl-devel zlib-devel libbrotli-devel libnghttp2-devel libpsl-devel libssh2-devel
site: https://mirrors.kernel.org/sourceware/cygwin/

- name: 'autotools configure'
if: ${{ matrix.build == 'automake' }}
timeout-minutes: 5
shell: C:\cygwin\bin\bash.exe '{0}'
run: |
export PATH="/usr/bin:$(cygpath ${SYSTEMROOT})/system32"
autoreconf -fi
mkdir bld && cd bld && ../configure --enable-warnings --enable-werror \
--prefix="${HOME}"/install \
--enable-websockets \
--with-openssl \
--with-libssh2 \
${{ matrix.config }} || { tail -n 1200 config.log; false; }
- name: 'autotools build'
if: ${{ matrix.build == 'automake' }}
timeout-minutes: 10
shell: C:\cygwin\bin\bash.exe '{0}'
run: |
make -C bld -j3 V=1 install
find . -name '*.exe' -o -name '*.dll'
bld/src/curl.exe --disable --version
- name: 'autotools build examples'
if: ${{ matrix.build == 'automake' }}
timeout-minutes: 5
shell: C:\cygwin\bin\bash.exe '{0}'
run: |
make -C bld -j3 V=1 examples
- name: 'autotools build tests'
if: ${{ matrix.build == 'automake' }}
timeout-minutes: 10
shell: C:\cygwin\bin\bash.exe '{0}'
run: |
make -C bld -j3 -C tests V=1
- name: 'autotools run tests'
if: ${{ matrix.build == 'automake' && matrix.tflags != 'skip' }}
timeout-minutes: 40
shell: C:\cygwin\bin\bash.exe '{0}'
run: |
export TFLAGS='-j8 ${{ matrix.tflags }}'
if [ -x "$(cygpath -u "${SYSTEMROOT}/System32/curl.exe")" ]; then
TFLAGS+=" -ac $(cygpath -u "${SYSTEMROOT}/System32/curl.exe")"
fi
make -C bld -j3 V=1 test-ci
- name: 'cmake configure'
if: ${{ matrix.build == 'cmake' }}
timeout-minutes: 5
shell: C:\cygwin\bin\bash.exe '{0}'
run: |
export PATH="/usr/bin:$(cygpath ${SYSTEMROOT})/system32"
cmake -B bld ${options} ${{ matrix.config }} \
"-DCMAKE_C_FLAGS=${cflags}" \
-DCMAKE_UNITY_BUILD=ON \
-DCURL_WERROR=ON \
-DBUILD_EXAMPLES=ON \
-DENABLE_WEBSOCKETS=ON \
-DCURL_BROTLI=ON -DUSE_NGHTTP2=ON
- name: 'cmake build'
if: ${{ matrix.build == 'cmake' }}
timeout-minutes: 10
shell: C:\cygwin\bin\bash.exe '{0}'
run: |
cmake --build bld --config '${{ matrix.type }}' --parallel 3
[[ '${{ matrix.config }}' != *'BUILD_SHARED_LIBS=OFF'* ]] && cp -f -p bld/lib/*.dll bld/src/
find . -name '*.exe' -o -name '*.dll'
bld/src/curl.exe --disable --version
msys2:
name: "msys2 (${{ matrix.build }}, ${{ matrix.sys }}, ${{ matrix.env }}, ${{ matrix.config }})"
runs-on: windows-latest
timeout-minutes: 45
strategy:
matrix:
include:
- { build: 'autotools', sys: 'msys' , env: 'x86_64' , tflags: '!19 !1233' , config: '--enable-debug --disable-threaded-resolver --disable-proxy' }
- { build: 'autotools', sys: 'msys' , env: 'x86_64' , tflags: '!19 !504 !704 !705 !1233' , config: '--enable-debug --disable-threaded-resolver' }
- { build: 'autotools', sys: 'msys' , env: 'x86_64' , tflags: '!19 !504 !704 !705 !1233' , config: '' }
# FIXME: WebSockets test results ignored due to frequent failures on native Windows:
- { build: 'cmake' , sys: 'mingw64', env: 'x86_64' , tflags: '!19 !504 !704 !705 !1233 ~2301 ~2305', config: '-DENABLE_DEBUG=ON -DBUILD_SHARED_LIBS=OFF -DCURL_USE_SCHANNEL=ON -DENABLE_UNICODE=ON', type: 'Debug' }
- { build: 'cmake' , sys: 'mingw64', env: 'x86_64' , tflags: 'skip' , config: '-DENABLE_DEBUG=ON -DBUILD_SHARED_LIBS=OFF -DCURL_USE_SCHANNEL=ON -DENABLE_UNICODE=ON', type: 'Release' }
- { build: 'cmake' , sys: 'clang64', env: 'clang-x86_64', tflags: 'skip' , config: '-DENABLE_DEBUG=ON -DBUILD_SHARED_LIBS=OFF -DCURL_USE_OPENSSL=ON -DENABLE_UNICODE=OFF', type: 'Release' }
fail-fast: false
steps:
- run: git config --global core.autocrlf input
- uses: actions/checkout@v4
- uses: msys2/setup-msys2@v2
if: ${{ matrix.sys == 'msys' }}
with:
msystem: ${{ matrix.sys }}
# https://packages.msys2.org/search
install: gcc ${{ matrix.build }} make openssl-devel zlib-devel brotli-devel libnghttp2-devel libpsl-devel libssh2-devel
- uses: msys2/setup-msys2@v2
if: ${{ matrix.sys != 'msys' }}
with:
msystem: ${{ matrix.sys }}
install: >-
mingw-w64-${{ matrix.env }}-cc
mingw-w64-${{ matrix.env }}-${{ matrix.build }} make
mingw-w64-${{ matrix.env }}-openssl
- name: 'autotools configure'
if: ${{ matrix.build == 'autotools' }}
timeout-minutes: 5
shell: msys2 {0}
run: |
autoreconf -fi
mkdir bld && cd bld && ../configure --enable-warnings --enable-werror \
--prefix="${HOME}"/install \
--enable-websockets \
--with-openssl \
--with-libssh2 \
${{ matrix.config }} || { tail -n 1200 config.log; false; }
- name: 'autotools build'
if: ${{ matrix.build == 'autotools' }}
timeout-minutes: 10
shell: msys2 {0}
run: |
make -C bld -j3 V=1 install
find . -name '*.exe' -o -name '*.dll'
bld/src/curl.exe --disable --version
- name: 'autotools build examples'
if: ${{ matrix.build == 'autotools' }}
timeout-minutes: 5
shell: msys2 {0}
run: |
make -C bld -j3 V=1 examples
- name: 'autotools build tests'
if: ${{ matrix.build == 'autotools' }}
timeout-minutes: 10
shell: msys2 {0}
run: |
make -C bld -j3 -C tests V=1
- name: 'autotools run tests'
if: ${{ matrix.build == 'autotools' && matrix.tflags != 'skip' }}
timeout-minutes: 30
shell: msys2 {0}
run: |
export TFLAGS='-j14 ${{ matrix.tflags }}'
if [ -x "$(cygpath -u 'C:/msys64/usr/bin/curl.exe')" ]; then
TFLAGS+=" -ac $(cygpath -u 'C:/msys64/usr/bin/curl.exe')"
fi
make -C bld -j3 V=1 test-ci
- name: 'cmake configure'
if: ${{ matrix.build == 'cmake' }}
timeout-minutes: 5
shell: msys2 {0}
run: |
if [[ '${{ matrix.env }}' = 'clang'* ]]; then
options='-DCMAKE_C_COMPILER=clang'
else
options='-DCMAKE_C_COMPILER=gcc'
fi
cflags='-Wno-deprecated-declarations'
if [ '${{ matrix.test }}' = 'uwp' ]; then
options+=' -DCMAKE_SYSTEM_NAME=WindowsStore -DCMAKE_SYSTEM_VERSION=10.0'
pacman --noconfirm --ask 20 --noprogressbar --sync --needed 'mingw-w64-${{ matrix.env }}-winstorecompat-git'
specs="$(realpath gcc-specs-uwp)"
gcc -dumpspecs | sed -e 's/-lmingwex/-lwindowsapp -lmingwex -lwindowsapp -lwindowsappcompat/' -e 's/-lmsvcrt/-lmsvcr120_app/' > "${specs}"
cflags+=" -specs=${specs} -DWINSTORECOMPAT -DWINAPI_FAMILY=WINAPI_FAMILY_APP"
# CMake (as of v3.26.4) gets confused and applies the MSVC rc.exe command-line
# template to windres. Reset it to the windres template manually:
rcopts='<CMAKE_RC_COMPILER> -O coff <DEFINES> <INCLUDES> <FLAGS> <SOURCE> <OBJECT>'
else
rcopts=''
fi
[ '${{ matrix.type }}' = 'Debug' ] && options+=' -DCMAKE_RUNTIME_OUTPUT_DIRECTORY_DEBUG='
[ '${{ matrix.type }}' = 'Release' ] && options+=' -DCMAKE_RUNTIME_OUTPUT_DIRECTORY_RELEASE='
[ '${{ matrix.type }}' = 'Release' ] && [[ '${{ matrix.config }}' = *'ENABLE_DEBUG=ON'* ]] && cflags+=' -DDEBUGBUILD'
cmake -B bld ${options} ${{ matrix.config }} \
"-DCMAKE_C_FLAGS=${cflags}" \
"-DCMAKE_RC_COMPILE_OBJECT=${rcopts}" \
"-DCMAKE_BUILD_TYPE=${{ matrix.type }}" \
-DCMAKE_UNITY_BUILD=ON \
-DCURL_WERROR=ON \
-DBUILD_EXAMPLES=ON \
-DENABLE_WEBSOCKETS=ON \
-DCURL_BROTLI=ON -DUSE_NGHTTP2=ON
- name: 'cmake build'
if: ${{ matrix.build == 'cmake' }}
timeout-minutes: 10
shell: msys2 {0}
run: |
cmake --build bld --config '${{ matrix.type }}' --parallel 3
[[ '${{ matrix.config }}' != *'BUILD_SHARED_LIBS=OFF'* ]] && cp -f -p bld/lib/*.dll bld/src/
find . -name '*.exe' -o -name '*.dll'
bld/src/curl.exe --disable --version
- name: 'cmake build tests'
if: ${{ matrix.build == 'cmake' }}
timeout-minutes: 10
shell: msys2 {0}
run: |
cmake --build bld --config '${{ matrix.type }}' --parallel 3 --target testdeps
- name: 'cmake run tests'
if: ${{ matrix.build == 'cmake' && matrix.tflags != 'skip' }}
timeout-minutes: 40
shell: msys2 {0}
run: |
export TFLAGS='-j14 ${{ matrix.tflags }}'
if [ -x "$(cygpath -u 'C:/msys64/usr/bin/curl.exe')" ]; then
TFLAGS+=" -ac $(cygpath -u 'C:/msys64/usr/bin/curl.exe')"
fi
ls bld/lib/*.dll >/dev/null 2>&1 && cp -f -p bld/lib/*.dll bld/tests/libtest/
cmake --build bld --config '${{ matrix.type }}' --target test-ci
15 changes: 0 additions & 15 deletions appveyor.sh
Original file line number Diff line number Diff line change
Expand Up @@ -102,19 +102,6 @@ EOF
rm _make.bat
)
curl="builds/libcurl-vc14.10-x64-${PATHPART}-dll-ssl-dll-ipv6-sspi/bin/curl.exe"
elif [ "${BUILD_SYSTEM}" = 'autotools' ]; then
autoreconf -fi
(
mkdir _bld
cd _bld
# shellcheck disable=SC2086
../configure ${CONFIG_ARGS:-}
make -j2 V=1
make -j2 V=1 examples
cd tests
make -j2 V=1
)
curl='_bld/src/curl.exe'
fi

find . -name '*.exe' -o -name '*.dll'
Expand Down Expand Up @@ -144,8 +131,6 @@ if [ "${TESTING}" = 'ON' ]; then
cmake --build _bld --config "${PRJ_CFG}" --parallel 2 --target testdeps
ls _bld/lib/*.dll >/dev/null 2>&1 && cp -f -p _bld/lib/*.dll _bld/tests/libtest/
cmake --build _bld --config "${PRJ_CFG}" --target test-ci
elif [ "${BUILD_SYSTEM}" = 'autotools' ]; then
make -C _bld -j2 V=1 test-ci
else
(
TFLAGS="-a -p !flaky -r -rm ${TFLAGS}"
Expand Down
44 changes: 0 additions & 44 deletions appveyor.yml
Original file line number Diff line number Diff line change
Expand Up @@ -131,19 +131,6 @@ environment:

# generated CMake-based MSYS Makefiles builds (mingw cross-compiling)

- job_name: 'CMake, mingw-w64, gcc 13, Debug, x64, Schannel, Static, Unicode'
APPVEYOR_BUILD_WORKER_IMAGE: 'Visual Studio 2022'
BUILD_SYSTEM: CMake
PRJ_GEN: 'MSYS Makefiles'
PRJ_CFG: Debug
SCHANNEL: 'ON'
ENABLE_UNICODE: 'ON'
HTTP_ONLY: 'OFF'
TESTING: 'ON'
DISABLED_TESTS: '!1086 !1139 !1451 !1501 !1177 !1477'
ADD_PATH: 'C:/msys64/mingw64/bin'
MSYS2_ARG_CONV_EXCL: '/*'
BUILD_OPT: -k
- job_name: 'CMake, mingw-w64, gcc 7, Debug, x64, Schannel, Static, Unicode'
APPVEYOR_BUILD_WORKER_IMAGE: 'Visual Studio 2017'
BUILD_SYSTEM: CMake
Expand Down Expand Up @@ -255,37 +242,6 @@ environment:
TESTING: 'OFF'
VC_VERSION: VC12

# autotools-based builds (NOT mingw cross-compiling, but msys2 native)

- job_name: 'autotools, msys2, Debug, x86_64, no Proxy, no SSL'
APPVEYOR_BUILD_WORKER_IMAGE: 'Visual Studio 2017'
BUILD_SYSTEM: autotools
TESTING: 'ON'
DISABLED_TESTS: '!19 !1233'
CONFIG_ARGS: '--enable-warnings --enable-werror --without-ssl --enable-websockets --without-libpsl --enable-debug --disable-threaded-resolver --disable-proxy'
- job_name: 'autotools, msys2, Debug, x86_64, no SSL'
APPVEYOR_BUILD_WORKER_IMAGE: 'Visual Studio 2017'
BUILD_SYSTEM: autotools
TESTING: 'ON'
DISABLED_TESTS: '!19 !504 !704 !705 !1233'
CONFIG_ARGS: '--enable-warnings --enable-werror --without-ssl --enable-websockets --without-libpsl --enable-debug --disable-threaded-resolver'
- job_name: 'autotools, msys2, Release, x86_64, no SSL'
APPVEYOR_BUILD_WORKER_IMAGE: 'Visual Studio 2017'
BUILD_SYSTEM: autotools
TESTING: 'ON'
DISABLED_TESTS: '!19 !504 !704 !705 !1233'
CONFIG_ARGS: '--enable-warnings --enable-werror --without-ssl --enable-websockets --without-libpsl'

# autotools-based Cygwin build

- job_name: 'autotools, cygwin, Debug, x86_64, no SSL'
APPVEYOR_BUILD_WORKER_IMAGE: 'Visual Studio 2022'
BUILD_SYSTEM: autotools
TESTING: 'ON'
DISABLED_TESTS: ''
ADD_SHELL: 'C:/cygwin64/bin'
CONFIG_ARGS: '--enable-warnings --enable-werror --without-ssl --enable-websockets --without-libpsl --enable-debug --disable-threaded-resolver'

install:
- ps: |
if($env:ADD_SHELL -ne $null) {
Expand Down

0 comments on commit 36fd2dd

Please sign in to comment.