Skip to content

Commit

Permalink
tests: add HTTP/3 test case, custom location for proper nghttpx
Browse files Browse the repository at this point in the history
- adding support for HTTP/3 test cases via a nghttpx server that is
  build with ngtcp2 and nghttp3.
- test2500 is the first test case, performing a simple GET.
- nghttpx is checked for support and the 'feature' nghttpx-h3
  is set accordingly. test2500 will only run, when supported.
- a specific nghttpx location can be given in the environment
  variable NGHTTPX or via the configure option
    --with-test-nghttpx=<path>

Extend NGHTTPX config to H2 tests as well

* use $ENV{NGHTTPX} and the configured default also in http2 server starts
* always provide the empty test/nghttpx.conf to nghttpx. as it defaults to
  reading /etc/nghttpx/nghttpx.conf otherwise.

Added nghttpx to CI ngtcp2 jobs to run h3 tests.

Closes #9031
  • Loading branch information
icing authored and bagder committed Nov 30, 2022
1 parent 0186ec4 commit ca15b75
Show file tree
Hide file tree
Showing 16 changed files with 582 additions and 24 deletions.
41 changes: 34 additions & 7 deletions .github/workflows/ngtcp2-gnutls.yml
Original file line number Diff line number Diff line change
Expand Up @@ -28,9 +28,21 @@ jobs:
matrix:
build:
- name: gnutls
install: nettle-dev libp11-kit-dev libtspi-dev libunistring-dev guile-2.2-dev libtasn1-bin libtasn1-6-dev libidn2-0-dev gawk gperf libtss2-dev dns-root-data bison gtk-doc-tools texinfo texlive texlive-extra-utils autopoint libev-dev
configure: LDFLAGS="-Wl,-rpath,$HOME/all/lib" --with-gnutls=$HOME/all --enable-debug
gnutls-configure: --with-included-libtasn1 --with-included-unistring --disable-guile --disable-doc --disable-tests
install: >-
libpsl-dev libbrotli-dev libzstd-dev zlib1g-dev libev-dev libc-ares-dev
nettle-dev libp11-kit-dev libtspi-dev libunistring-dev guile-2.2-dev libtasn1-bin
libtasn1-6-dev libidn2-0-dev gawk gperf libtss2-dev dns-root-data bison gtk-doc-tools
texinfo texlive texlive-extra-utils autopoint libev-dev
configure: >-
PKG_CONFIG_PATH="$HOME/all/lib/pkgconfig" LDFLAGS="-Wl,-rpath,$HOME/all/lib"
--with-ngtcp2=$HOME/all --enable-warnings --enable-werror --enable-debug
--with-test-nghttpx="$HOME/all/bin/nghttpx"
ngtcp2-configure: >-
--prefix=$HOME/all PKG_CONFIG_PATH="$HOME/all/lib/pkgconfig" --enable-lib-only
gnutls-configure: >-
PKG_CONFIG_PATH="$HOME/all/lib/pkgconfig" LDFLAGS="-Wl,-rpath,$HOME/all/lib -L$HOME/all/lib"
--with-included-libtasn1 --with-included-unistring
--disable-guile --disable-doc --disable-tests --disable-tools
steps:
- run: |
Expand All @@ -39,6 +51,13 @@ jobs:
sudo python3 -m pip install impacket
name: 'install prereqs and impacket'
- run: |
git clone --depth=1 -b openssl-3.0.7+quic https://github.com/quictls/openssl
cd openssl
./config --prefix=$HOME/all --libdir=$HOME/all/lib
make install_sw
name: 'install quictls'
- run: |
git clone --depth=1 https://gitlab.com/gnutls/nettle.git
cd nettle
Expand All @@ -51,32 +70,40 @@ jobs:
git clone --depth=1 -b 3.7.7 https://github.com/gnutls/gnutls.git
cd gnutls
./bootstrap
./configure PKG_CONFIG_PATH="$HOME/all/lib/pkgconfig" LDFLAGS="-Wl,-rpath,$HOME/all/lib -L$HOME/all/lib" --prefix=$HOME/all ${{ matrix.build.gnutls-configure }} --disable-tools
./configure ${{ matrix.build.gnutls-configure }} --prefix=$HOME/all
make install
name: 'install gnutls'
- run: |
git clone --depth=1 https://github.com/ngtcp2/nghttp3
cd nghttp3
autoreconf -fi
./configure --prefix=$HOME/all --enable-lib-only
./configure --prefix=$HOME/all PKG_CONFIG_PATH="$HOME/all/lib/pkgconfig" --enable-lib-only
make install
name: 'install nghttp3'
- run: |
git clone --depth=1 https://github.com/ngtcp2/ngtcp2
cd ngtcp2
autoreconf -fi
./configure PKG_CONFIG_PATH=$HOME/all/lib/pkgconfig LDFLAGS="-Wl,-rpath,$HOME/all/lib" --prefix=$HOME/all --enable-lib-only --with-gnutls=$HOME/all
./configure ${{ matrix.build.ngtcp2-configure }} --with-openssl --with-gnutls
make install
name: 'install ngtcp2'
- run: |
git clone --depth=1 https://github.com/nghttp2/nghttp2
cd nghttp2
autoreconf -fi
./configure --prefix=$HOME/all PKG_CONFIG_PATH="$HOME/all/lib/pkgconfig" --enable-http3
make install
name: 'install nghttp2'
- uses: actions/checkout@v3

- run: autoreconf -fi
name: 'autoreconf'

- run: ./configure --enable-warnings --enable-werror ${{ matrix.build.configure }}
- run: ./configure --with-gnutls=$HOME/all ${{ matrix.build.configure }}
name: 'configure'

- run: make V=1
Expand Down
98 changes: 98 additions & 0 deletions .github/workflows/ngtcp2-quictls.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,98 @@
# Copyright (C) 2000 - 2022 Daniel Stenberg, <daniel@haxx.se>, et al.
#
# SPDX-License-Identifier: curl

name: ngtcp2

on:
push:
branches:
- master
- '*/ci'
pull_request:
branches:
- master

concurrency:
# Hardcoded workflow filename as workflow name above is just Linux again
group: ngtcp2-openssl-${{ github.event.pull_request.number || github.sha }}
cancel-in-progress: true

jobs:
autotools:
name: ${{ matrix.build.name }}
runs-on: 'ubuntu-latest'
timeout-minutes: 60
strategy:
fail-fast: false
matrix:
build:
- name: quictls
install: >-
libpsl-dev libbrotli-dev libzstd-dev zlib1g-dev libev-dev libc-ares-dev
configure: >-
PKG_CONFIG_PATH="$HOME/all/lib/pkgconfig" LDFLAGS="-Wl,-rpath,$HOME/all/lib"
--with-ngtcp2=$HOME/all --enable-warnings --enable-werror --enable-debug
--with-test-nghttpx="$HOME/all/bin/nghttpx"
ngtcp2-configure: >-
--prefix=$HOME/all PKG_CONFIG_PATH="$HOME/all/lib/pkgconfig" --enable-lib-only
steps:
- run: |
sudo apt-get update
sudo apt-get install libtool autoconf automake pkg-config stunnel4 ${{ matrix.build.install }}
sudo python3 -m pip install impacket
name: 'install prereqs and impacket'
- run: |
git clone --depth=1 -b openssl-3.0.7+quic https://github.com/quictls/openssl
cd openssl
./config --prefix=$HOME/all --libdir=$HOME/all/lib
make install_sw
name: 'install quictls'
- run: |
git clone --depth=1 https://github.com/ngtcp2/nghttp3
cd nghttp3
autoreconf -fi
./configure --prefix=$HOME/all PKG_CONFIG_PATH="$HOME/all/lib/pkgconfig" --enable-lib-only
make install
name: 'install nghttp3'
- run: |
git clone --depth=1 https://github.com/ngtcp2/ngtcp2
cd ngtcp2
autoreconf -fi
./configure ${{ matrix.build.ngtcp2-configure }} --with-openssl
make install
name: 'install ngtcp2'
- run: |
git clone --depth=1 https://github.com/nghttp2/nghttp2
cd nghttp2
autoreconf -fi
./configure --prefix=$HOME/all PKG_CONFIG_PATH="$HOME/all/lib/pkgconfig" --enable-http3
make install
name: 'install nghttp2'
- uses: actions/checkout@v3

- run: autoreconf -fi
name: 'autoreconf'

- run: ./configure --with-openssl=$HOME/all ${{ matrix.build.configure }}
name: 'configure'

- run: make V=1
name: 'make'

- run: make V=1 examples
name: 'make examples'

- run: make V=1 -C tests
name: 'make tests'

- run: make V=1 test-ci
name: 'run tests'
env:
TFLAGS: "${{ matrix.build.tflags }}"
39 changes: 31 additions & 8 deletions .github/workflows/ngtcp2-wolfssl.yml
Original file line number Diff line number Diff line change
Expand Up @@ -28,9 +28,17 @@ jobs:
matrix:
build:
- name: wolfssl
install:
configure: LDFLAGS="-Wl,-rpath,$HOME/all/lib" --with-wolfssl=$HOME/all --enable-debug
wolfssl-configure: --enable-quic --enable-session-ticket --enable-earlydata --enable-psk --enable-harden --enable-altcertchains
install: >-
libpsl-dev libbrotli-dev libzstd-dev zlib1g-dev libev-dev libc-ares-dev
configure: >-
PKG_CONFIG_PATH="$HOME/all/lib/pkgconfig" LDFLAGS="-Wl,-rpath,$HOME/all/lib"
--with-ngtcp2=$HOME/all --enable-warnings --enable-werror --enable-debug
--with-test-nghttpx="$HOME/all/bin/nghttpx"
ngtcp2-configure: >-
--prefix=$HOME/all PKG_CONFIG_PATH="$HOME/all/lib/pkgconfig" --enable-lib-only
wolfssl-configure: >-
--enable-quic --enable-session-ticket --enable-earlydata --enable-psk
--enable-harden --enable-altcertchains
steps:
- run: |
Expand All @@ -48,27 +56,42 @@ jobs:
name: 'install wolfssl'
- run: |
git clone https://github.com/ngtcp2/nghttp3
git clone --depth=1 -b openssl-3.0.7+quic https://github.com/quictls/openssl
cd openssl
./config --prefix=$HOME/all --libdir=$HOME/all/lib
make install_sw
name: 'install quictls'
- run: |
git clone --depth=1 https://github.com/ngtcp2/nghttp3
cd nghttp3
autoreconf -fi
./configure --prefix=$HOME/all --enable-lib-only
./configure --prefix=$HOME/all PKG_CONFIG_PATH="$HOME/all/lib/pkgconfig" --enable-lib-only
make install
name: 'install nghttp3'
- run: |
git clone https://github.com/ngtcp2/ngtcp2
git clone --depth=1 https://github.com/ngtcp2/ngtcp2
cd ngtcp2
autoreconf -fi
./configure PKG_CONFIG_PATH=$HOME/all/lib/pkgconfig LDFLAGS="-Wl,-rpath,$HOME/all/lib" --prefix=$HOME/all --enable-lib-only --with-wolfssl=$HOME/all
./configure ${{ matrix.build.ngtcp2-configure }} --with-openssl --with-wolfssl
make install
name: 'install ngtcp2'
- run: |
git clone --depth=1 https://github.com/nghttp2/nghttp2
cd nghttp2
autoreconf -fi
./configure --prefix=$HOME/all PKG_CONFIG_PATH="$HOME/all/lib/pkgconfig" --enable-http3
make install
name: 'install nghttp2'
- uses: actions/checkout@v3

- run: autoreconf -fi
name: 'autoreconf'

- run: ./configure --enable-warnings --enable-werror ${{ matrix.build.configure }}
- run: ./configure --with-wolfssl=$HOME/all ${{ matrix.build.configure }}
name: 'configure'

- run: make V=1
Expand Down
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -64,3 +64,5 @@ curl_fuzzer
curl_fuzzer_seed_corpus.zip
libstandaloneengine.a
tests/string
tests/config

11 changes: 11 additions & 0 deletions configure.ac
Original file line number Diff line number Diff line change
Expand Up @@ -301,6 +301,16 @@ AS_HELP_STRING([--with-nss=PATH],[where to look for NSS, PATH points to the inst
fi
)

TEST_NGHTTPX=nghttpx
AC_ARG_WITH(test-nghttpx,dnl
AS_HELP_STRING([--with-test-nghttpx=PATH],[where to find nghttpx for testing]),
TEST_NGHTTPX=$withval
if test X"$OPT_TEST_NGHTTPX" = "Xno" ; then
TEST_NGHTTPX=""
fi
)
AC_SUBST(TEST_NGHTTPX)

dnl If no TLS choice has been made, check if it was explicitly disabled or
dnl error out to force the user to decide.
if test -z "$TLSCHOICE"; then
Expand Down Expand Up @@ -4568,6 +4578,7 @@ AC_CONFIG_FILES([Makefile \
lib/libcurl.vers \
lib/libcurl.plist \
tests/Makefile \
tests/config \
tests/certs/Makefile \
tests/certs/scripts/Makefile \
tests/data/Makefile \
Expand Down
1 change: 1 addition & 0 deletions tests/.gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -24,3 +24,4 @@ runtests.pdf
testcurl.html
testcurl.pdf
*.port
config
7 changes: 6 additions & 1 deletion tests/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ SPDX-License-Identifier: curl
- diff (when a test fails, a diff is shown)
- stunnel (for HTTPS and FTPS tests)
- OpenSSH or SunSSH (for SCP, SFTP and SOCKS4/5 tests)
- nghttpx (for HTTP/2 tests)
- nghttpx (for HTTP/2 and HTTP/3 tests)
- nroff (for --manual tests)
- An available `en_US.UTF-8` locale

Expand Down Expand Up @@ -69,6 +69,11 @@ SPDX-License-Identifier: curl

The HTTP server supports listening on a Unix domain socket, the default
location is 'http.sock'.

For HTTP/2 and HTTP/3 testing an installed `nghttpx` is used. HTTP/3
tests check if nghttpx supports the protocol. To override the nghttpx
used, set the environment variable `NGHTTPX`. The default can also be
changed by specifying `--with-test-nghttpx=<path>` as argument to `configure`.

### Run

Expand Down
24 changes: 24 additions & 0 deletions tests/config.in
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
#***************************************************************************
# _ _ ____ _
# Project ___| | | | _ \| |
# / __| | | | |_) | |
# | (__| |_| | _ <| |___
# \___|\___/|_| \_\_____|
#
# Copyright (C) 1998 - 2022, Daniel Stenberg, <daniel@haxx.se>, et al.
#
# This software is licensed as described in the file COPYING, which
# you should have received as part of this distribution. The terms
# are also available at https://curl.se/docs/copyright.html.
#
# You may opt to use, copy, modify, merge, publish, distribute and/or sell
# copies of the Software, and permit persons to whom the Software is
# furnished to do so, under the terms of the COPYING file.
#
# This software is distributed on an "AS IS" basis, WITHOUT WARRANTY OF ANY
# KIND, either express or implied.
#
# SPDX-License-Identifier: curl
#
###########################################################################
NGHTTPX: @TEST_NGHTTPX@
2 changes: 2 additions & 0 deletions tests/data/Makefile.inc
Original file line number Diff line number Diff line change
Expand Up @@ -243,6 +243,8 @@ test2200 test2201 test2202 test2203 test2204 test2205 \
\
test2300 test2301 test2302 test2303 \
\
test2500 \
\
test3000 test3001 test3002 test3003 test3004 test3005 test3006 test3007 \
test3008 test3009 test3010 test3011 test3012 test3013 test3014 test3015 \
test3016 test3017 test3018 test3019 test3020 test3021 test3022 test3023 \
Expand Down

1 comment on commit ca15b75

@MarcelRaad
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately, the changes to runtests.pl seem to lead to red autobuilds if nghttpx is unavailable:
https://curl.se/dev/log.cgi?id=20221205052054-623463#prob1
Can we avoid that?

Please sign in to comment.