Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HTTP Lua: upgrade ngx_http_lua_module v0.10.26rc1 for lua-resty-core v0.1.28rc1 #1899

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ jobs:
uses: actions/checkout@v3
with:
repository: openresty/lua-resty-core
ref: v0.1.27
ref: v0.1.28rc1
path: lua-resty-core
- name: 'build lua-resty-core'
working-directory: lua-resty-core
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/test-nginx-core.yml
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@ jobs:
uses: actions/checkout@v3
with:
repository: openresty/lua-resty-core
ref: v0.1.27
ref: v0.1.28rc1
path: lua-resty-core
- name: 'build lua-resty-core'
working-directory: lua-resty-core
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/test-ntls.yml
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ jobs:
uses: actions/checkout@v3
with:
repository: openresty/lua-resty-core
ref: v0.1.27
ref: v0.1.28rc1
path: lua-resty-core
- name: 'build lua-resty-core'
working-directory: lua-resty-core
Expand Down
58 changes: 35 additions & 23 deletions modules/ngx_http_lua_module/.travis.yml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
dist: bionic
dist: focal

branches:
only:
Expand All @@ -24,6 +24,11 @@ addons:
- libtest-longstring-perl
- liblist-moreutils-perl
- libgd-dev
- time
- cmake
- libunwind-dev
- wget
- libbrotli1

cache:
directories:
Expand All @@ -38,9 +43,13 @@ env:
- LUAJIT_INC=$LUAJIT_PREFIX/include/luajit-2.1
- LUA_INCLUDE_DIR=$LUAJIT_INC
- PCRE_VER=8.45
- PCRE2_VER=10.37
- PCRE_PREFIX=/opt/pcre
- PCRE2_PREFIX=/opt/pcre2
- PCRE_LIB=$PCRE_PREFIX/lib
- PCRE2_LIB=$PCRE2_PREFIX/lib
- PCRE_INC=$PCRE_PREFIX/include
- PCRE2_INC=$PCRE2_PREFIX/include
- OPENSSL_PREFIX=/opt/ssl
- OPENSSL_LIB=$OPENSSL_PREFIX/lib
- OPENSSL_INC=$OPENSSL_PREFIX/include
Expand All @@ -50,26 +59,33 @@ env:
- LD_LIBRARY_PATH=$LUAJIT_LIB:$LD_LIBRARY_PATH
- DRIZZLE_VER=2011.07.21
- TEST_NGINX_SLEEP=0.006
- MALLOC_PERTURB_=9
jobs:
- NGINX_VERSION=1.21.4 OPENSSL_VER=1.1.0l OPENSSL_PATCH_VER=1.1.0d
#- NGINX_VERSION=1.21.4 OPENSSL_VER=1.1.0l OPENSSL_PATCH_VER=1.1.0d
- NGINX_VERSION=1.21.4 OPENSSL_VER=1.1.1s OPENSSL_PATCH_VER=1.1.1f
#- NGINX_VERSION=1.25.1 OPENSSL_VER=1.1.0l OPENSSL_PATCH_VER=1.1.0d
- NGINX_VERSION=1.25.1 OPENSSL_VER=1.1.1s OPENSSL_PATCH_VER=1.1.1f USE_PCRE2=Y
- NGINX_VERSION=1.25.1 BORINGSSL=1 TEST_NGINX_USE_HTTP3=1 USE_PCRE2=Y
#- NGINX_VERSION=1.25.1 OPENSSL_VER=1.1.1s TEST_NGINX_USE_HTTP2=1

services:
- memcached
- redis
- mysql

before_install:
- sudo apt update
- sudo apt install --only-upgrade ca-certificates
- '! grep -n -P ''(?<=.{80}).+'' --color `find src -name ''*.c''` `find . -name ''*.h''` || (echo "ERROR: Found C source lines exceeding 80 columns." > /dev/stderr; exit 1)'
- '! grep -n -P ''\t+'' --color `find src -name ''*.c''` `find . -name ''*.h''` || (echo "ERROR: Cannot use tabs." > /dev/stderr; exit 1)'
- /usr/bin/env perl $(command -v cpanm) --sudo --notest Test::Nginx IPC::Run > build.log 2>&1 || (cat build.log && exit 1)
- pyenv global 2.7

install:
- if [ ! -f download-cache/drizzle7-$DRIZZLE_VER.tar.gz ]; then wget -P download-cache http://openresty.org/download/drizzle7-$DRIZZLE_VER.tar.gz; fi
- if [ ! -f download-cache/pcre-$PCRE_VER.tar.gz ]; then wget -P download-cache https://downloads.sourceforge.net/project/pcre/pcre/${PCRE_VER}/pcre-${PCRE_VER}.tar.gz; fi
- if [ ! -f download-cache/openssl-$OPENSSL_VER.tar.gz ]; then wget -P download-cache https://www.openssl.org/source/openssl-$OPENSSL_VER.tar.gz || wget -P download-cache https://www.openssl.org/source/old/${OPENSSL_VER//[a-z]/}/openssl-$OPENSSL_VER.tar.gz; fi
- if [ ! -f download-cache/drizzle7-$DRIZZLE_VER.tar.gz ]; then wget -P download-cache https://github.com/openresty/openresty-deps-prebuild/releases/download/v20230902/drizzle7-$DRIZZLE_VER.tar.gz; fi
- if [ "$USE_PCRE2" != "Y" ] && [ ! -f download-cache/pcre-$PCRE_VER.tar.gz ]; then wget -P download-cache https://downloads.sourceforge.net/project/pcre/pcre/${PCRE_VER}/pcre-${PCRE_VER}.tar.gz; fi
- if [ "$USE_PCRE2" = "Y" ] && [ ! -f download-cache/pcre2-$PCRE2_VER.tar.gz ]; then wget -P download-cache https://downloads.sourceforge.net/project/pcre/pcre2/${PCRE2_VER}/pcre2-${PCRE2_VER}.tar.gz; fi
- if [ -n "$OPENSSL_VER" ] && [ ! -f download-cache/openssl-$OPENSSL_VER.tar.gz ]; then wget -P download-cache https://www.openssl.org/source/openssl-$OPENSSL_VER.tar.gz || wget -P download-cache https://www.openssl.org/source/old/${OPENSSL_VER//[a-z]/}/openssl-$OPENSSL_VER.tar.gz; fi
- if [ -n "$OPENSSL_VER" ] && [ ! -f download-cache/openssl-$OPENSSL_VER.tar.gz ]; then wget -P download-cache https://www.openssl.org/source/openssl-$OPENSSL_VER.tar.gz || wget -P download-cache https://www.openssl.org/source/old/${OPENSSL_VER//[a-z]/}/openssl-$OPENSSL_VER.tar.gz; fi
- wget https://github.com/openresty/openresty-deps-prebuild/releases/download/v20230902/boringssl-20230902-x64-focal.tar.gz
- wget https://github.com/openresty/openresty-deps-prebuild/releases/download/v20230902/curl-h3-x64-focal.tar.gz
- git clone https://github.com/openresty/test-nginx.git
- git clone https://github.com/openresty/openresty.git ../openresty
- git clone https://github.com/openresty/no-pool-nginx.git ../no-pool-nginx
Expand All @@ -91,21 +107,24 @@ install:
- git clone https://github.com/openresty/lua-resty-core.git ../lua-resty-core
- git clone https://github.com/openresty/lua-resty-lrucache.git ../lua-resty-lrucache
- git clone https://github.com/openresty/lua-resty-mysql.git ../lua-resty-mysql
- git clone https://github.com/spacewander/lua-resty-rsa.git ../lua-resty-rsa
- git clone https://github.com/openresty/lua-resty-string.git ../lua-resty-string
- git clone https://github.com/openresty/stream-lua-nginx-module.git ../stream-lua-nginx-module
- git clone -b v2.1-agentzh https://github.com/openresty/luajit2.git luajit2

before_script:
- mysql -uroot -e 'create database ngx_test; grant all on ngx_test.* to "ngx_test"@"%" identified by "ngx_test"; flush privileges;'
- mysql -uroot -e "create database ngx_test; CREATE USER 'ngx_test'@'%' IDENTIFIED WITH mysql_native_password BY 'ngx_test'; grant all on ngx_test.* to 'ngx_test'@'%'; flush privileges;"

script:
- export PATH=$PWD/work/nginx/sbin:$PWD/openresty-devel-utils:$PATH
- sudo tar -C / -xf curl-h3-x64-focal.tar.gz
- export PATH=$PWD/work/nginx/sbin:$PWD/openresty-devel-utils:/opt/curl-h3/bin:$PATH
- ngx-releng > check.txt || true
- lines=`wc -l check.txt | awk '{print $1}'`; if [ $lines -gt 5 ]; then cat check.txt; exit 1; fi
- sudo iptables -I OUTPUT 1 -p udp --dport 10086 -j REJECT
- sudo iptables -I OUTPUT -p tcp --dst 127.0.0.2 --dport 12345 -j DROP
- sudo iptables -I OUTPUT -p udp --dst 127.0.0.2 --dport 12345 -j DROP
- sudo ip route add prohibit 0.0.0.1/32
- sudo sysctl -w kernel.pid_max=10000
- cd luajit2/
- make -j$JOBS CCDEBUG=-g Q= PREFIX=$LUAJIT_PREFIX CC=$CC XCFLAGS='-DLUA_USE_APICHECK -DLUA_USE_ASSERT -msse4.2' > build.log 2>&1 || (cat build.log && exit 1)
- sudo make install PREFIX=$LUAJIT_PREFIX > build.log 2>&1 || (cat build.log && exit 1)
Expand All @@ -116,19 +135,10 @@ script:
- sudo make install-libdrizzle-1.0 > build.log 2>&1 || (cat build.log && exit 1)
- cd ../mockeagain/ && make CC=$CC -j$JOBS && cd ..
- cd lua-cjson/ && make -j$JOBS && sudo make install && cd ..
- tar zxf download-cache/pcre-$PCRE_VER.tar.gz
- cd pcre-$PCRE_VER/
- ./configure --prefix=$PCRE_PREFIX --enable-jit --enable-utf --enable-unicode-properties > build.log 2>&1 || (cat build.log && exit 1)
- make -j$JOBS > build.log 2>&1 || (cat build.log && exit 1)
- sudo PATH=$PATH make install > build.log 2>&1 || (cat build.log && exit 1)
- cd ..
- tar zxf download-cache/openssl-$OPENSSL_VER.tar.gz
- cd openssl-$OPENSSL_VER/
- patch -p1 < ../../openresty/patches/openssl-$OPENSSL_PATCH_VER-sess_set_get_cb_yield.patch
- ./config shared enable-ssl3 enable-ssl3-method -g --prefix=$OPENSSL_PREFIX -DPURIFY > build.log 2>&1 || (cat build.log && exit 1)
- make -j$JOBS > build.log 2>&1 || (cat build.log && exit 1)
- sudo make PATH=$PATH install_sw > build.log 2>&1 || (cat build.log && exit 1)
- cd ..
- if [ "$USE_PCRE2" != "Y" ]; then tar zxf download-cache/pcre-$PCRE_VER.tar.gz; cd pcre-$PCRE_VER/; ./configure --prefix=$PCRE_PREFIX --enable-jit --enable-utf --enable-unicode-properties > build.log 2>&1 || (cat build.log && exit 1); make -j$JOBS > build.log 2>&1 || (cat build.log && exit 1); sudo PATH=$PATH make install > build.log 2>&1 || (cat build.log && exit 1); cd ..; fi
- if [ "$USE_PCRE2" = "Y" ]; then tar zxf download-cache/pcre2-$PCRE2_VER.tar.gz; cd pcre2-$PCRE2_VER/; ./configure --prefix=$PCRE2_PREFIX --enable-jit --enable-utf > build.log 2>&1 || (cat build.log && exit 1); make -j$JOBS > build.log 2>&1 || (cat build.log && exit 1); sudo PATH=$PATH make install > build.log 2>&1 || (cat build.log && exit 1); cd ..; fi
- if [ -n "$OPENSSL_VER" ]; then tar zxf download-cache/openssl-$OPENSSL_VER.tar.gz; cd openssl-$OPENSSL_VER/; patch -p1 < ../../openresty/patches/openssl-$OPENSSL_PATCH_VER-sess_set_get_cb_yield.patch; ./config shared enable-ssl3 enable-ssl3-method -g --prefix=$OPENSSL_PREFIX -DPURIFY > build.log 2>&1 || (cat build.log && exit 1); make -j$JOBS > build.log 2>&1 || (cat build.log && exit 1); sudo make PATH=$PATH install_sw > build.log 2>&1 || (cat build.log && exit 1); cd ..; fi
- if [ -n "$BORINGSSL" ]; then sudo mkdir -p /opt/ssl && sudo tar -C /opt/ssl -xf boringssl-20230902-x64-focal.tar.gz --strip-components=1; fi
- export NGX_BUILD_CC=$CC
- sh util/build-without-ssl.sh $NGINX_VERSION > build.log 2>&1 || (cat build.log && exit 1)
- sh util/build-with-dd.sh $NGINX_VERSION > build.log 2>&1 || (cat build.log && exit 1)
Expand All @@ -139,6 +149,8 @@ script:
- ldd `which nginx`|grep -E 'luajit|ssl|pcre'
- export LD_PRELOAD=$PWD/mockeagain/mockeagain.so
- export LD_LIBRARY_PATH=$PWD/mockeagain:$LD_LIBRARY_PATH
- export TEST_NGINX_HTTP3_CRT=$PWD/t/cert/http3/http3.crt
- export TEST_NGINX_HTTP3_KEY=$PWD/t/cert/http3/http3.key
- export TEST_NGINX_RESOLVER=8.8.4.4
- dig +short myip.opendns.com @resolver1.opendns.com || exit 0
- dig +short @$TEST_NGINX_RESOLVER openresty.org || exit 0
Expand Down
76 changes: 59 additions & 17 deletions modules/ngx_http_lua_module/README.markdown
Original file line number Diff line number Diff line change
Expand Up @@ -4,13 +4,11 @@ Name
ngx_http_lua_module - Embed the power of Lua into Nginx HTTP Servers.

This module is a core component of [OpenResty](https://openresty.org). If you are using this module,
then you are essentially using OpenResty.
then you are essentially using OpenResty :)

*This module is not distributed with the Nginx source.* See
[the installation instructions](#installation).

This is a core component of OpenResty. If you are using this module, then you are essentially using OpenResty :)

Table of Contents
=================

Expand Down Expand Up @@ -65,8 +63,8 @@ Version
=======

This document describes ngx_lua
[v0.10.19](https://github.com/openresty/lua-nginx-module/tags), which was released
on 3 Nov, 2020.
[v0.10.25](https://github.com/openresty/lua-nginx-module/tags), which was released
on 19 June 2023.

Videos
======
Expand Down Expand Up @@ -309,6 +307,8 @@ Nginx Compatibility

The latest version of this module is compatible with the following versions of Nginx:

* 1.25.x (last tested: 1.25.1)
* 1.21.x (last tested: 1.21.4)
* 1.19.x (last tested: 1.19.3)
* 1.17.x (last tested: 1.17.8)
* 1.15.x (last tested: 1.15.8)
Expand Down Expand Up @@ -964,7 +964,6 @@ TODO

* cosocket: implement LuaSocket's unconnected UDP API.
* cosocket: add support in the context of [init_by_lua*](#init_by_lua).
* cosocket: implement the `bind()` method for stream-typed cosockets.
* cosocket: review and merge aviramc's [patch](https://github.com/openresty/lua-nginx-module/pull/290) for adding the `bsdrecv` method.
* cosocket: add configure options for different strategies of handling the cosocket connection exceeding in the pools.
* review and apply vadim-pavlov's patch for [ngx.location.capture](#ngxlocationcapture)'s `extra_headers` option
Expand Down Expand Up @@ -1166,6 +1165,8 @@ Directives
* [lua_ssl_ciphers](#lua_ssl_ciphers)
* [lua_ssl_crl](#lua_ssl_crl)
* [lua_ssl_protocols](#lua_ssl_protocols)
* [lua_ssl_certificate](#lua_ssl_certificate)
* [lua_ssl_certificate_key](#lua_ssl_certificate_key)
* [lua_ssl_trusted_certificate](#lua_ssl_trusted_certificate)
* [lua_ssl_verify_depth](#lua_ssl_verify_depth)
* [lua_ssl_conf_command](#lua_ssl_conf_command)
Expand Down Expand Up @@ -2721,6 +2722,8 @@ lua_need_request_body

**phase:** *depends on usage*

Due to the stream processing feature of HTTP/2 or HTTP/3, this configuration could potentially block the entire request. Therefore, this configuration is effective only when HTTP/2 or HTTP/3 requests send content-length header. For requests with versions lower than HTTP/2, this configuration can still be used without any problems.

Determines whether to force the request body data to be read before running rewrite/access/content_by_lua* or not. The Nginx core does not read the client request body by default and if request body data is required, then this directive should be turned `on` or the [ngx.req.read_body](#ngxreqread_body) function should be called within the Lua code.

To read the request body data within the [$request_body](http://nginx.org/en/docs/http/ngx_http_core_module.html#var_request_body) variable,
Expand Down Expand Up @@ -3321,24 +3324,63 @@ lua_ssl_protocols

**syntax:** *lua_ssl_protocols \[SSLv2\] \[SSLv3\] \[TLSv1\] [TLSv1.1] [TLSv1.2] [TLSv1.3]*

**default:** *lua_ssl_protocols SSLv3 TLSv1 TLSv1.1 TLSv1.2*
**default:** *lua_ssl_protocols TLSv1 TLSv1.1 TLSv1.2 TLSv1.3*

**context:** *http, server, location*

Enables the specified protocols for requests to a SSL/TLS server in the [tcpsock:sslhandshake](#tcpsocksslhandshake) method.

The support for the `TLSv1.3` parameter requires version `v0.10.12` *and* OpenSSL 1.1.1.
From version v0.10.25, the default value change from `SSLV3 TLSv1 TLSv1.1 TLSv1.2` to `TLSv1 TLSv1.1 TLSv1.2 TLSv1.3`.

This directive was first introduced in the `v0.9.11` release.

[Back to TOC](#directives)

lua_ssl_certificate
-------------------

**syntax:** *lua_ssl_certificate &lt;file&gt;*

**default:** *none*

**context:** *http, server, location*

Specifies the file path to the SSL/TLS certificate in PEM format used for the [tcpsock:sslhandshake](#tcpsocksslhandshake) method.

This directive allows you to specify the SSL/TLS certificate that will be presented to server during the SSL/TLS handshake process.

This directive was first introduced in the `v0.10.26` release.

See also [lua_ssl_certificate_key](#lua_ssl_certificate_key) and [lua_ssl_verify_depth](#lua_ssl_verify_depth).

[Back to TOC](#directives)

lua_ssl_certificate_key
-----------------------

**syntax:** *lua_ssl_certificate_key &lt;file&gt;*

**default:** *none*

**context:** *http, server, location*

Specifies the file path to the private key associated with the SSL/TLS certificate used in the [tcpsock:sslhandshake](#tcpsocksslhandshake) method.

This directive allows you to specify the private key file corresponding to the SSL/TLS certificate specified by lua_ssl_certificate. The private key should be in PEM format and must match the certificate.

This directive was first introduced in the `v0.10.26` release.

See also [lua_ssl_certificate](#lua_ssl_certificate) and [lua_ssl_verify_depth](#lua_ssl_verify_depth).

[Back to TOC](#directives)

lua_ssl_trusted_certificate
---------------------------

**syntax:** *lua_ssl_trusted_certificate &lt;file&gt;*

**default:** *no*
**default:** *none*

**context:** *http, server, location*

Expand All @@ -3363,7 +3405,7 @@ Sets the verification depth in the server certificates chain.

This directive was first introduced in the `v0.9.11` release.

See also [lua_ssl_trusted_certificate](#lua_ssl_trusted_certificate).
See also [lua_ssl_certificate](#lua_ssl_certificate), [lua_ssl_certificate_key](#lua_ssl_certificate_key) and [lua_ssl_trusted_certificate](#lua_ssl_trusted_certificate).

[Back to TOC](#directives)

Expand Down Expand Up @@ -5384,6 +5426,8 @@ Reads the client request body synchronously without blocking the Nginx event loo
local args = ngx.req.get_post_args()
```

Due to the stream processing feature of HTTP/2 or HTTP/3, this api could potentially block the entire request. Therefore, this api is effective only when HTTP/2 or HTTP/3 requests send content-length header. For requests with versions lower than HTTP/2, this api can still be used without any problems.

If the request body is already read previously by turning on [lua_need_request_body](#lua_need_request_body) or by using other modules, then this function does not run and returns immediately.

If the request body has already been explicitly discarded, either by the [ngx.req.discard_body](#ngxreqdiscard_body) function or other modules, this function does not run and returns immediately.
Expand Down Expand Up @@ -5423,12 +5467,14 @@ See also [ngx.req.read_body](#ngxreqread_body).
ngx.req.get_body_data
---------------------

**syntax:** *data = ngx.req.get_body_data()*
**syntax:** *data = ngx.req.get_body_data(max_bytes?)*

**context:** *rewrite_by_lua&#42;, access_by_lua&#42;, content_by_lua&#42;, log_by_lua&#42;*

Retrieves in-memory request body data. It returns a Lua string rather than a Lua table holding all the parsed query arguments. Use the [ngx.req.get_post_args](#ngxreqget_post_args) function instead if a Lua table is required.

The optional `max_bytes` argument can be used when you don't need the entire body.

This function returns `nil` if

1. the request body has not been read,
Expand Down Expand Up @@ -5597,6 +5643,8 @@ Returns a read-only cosocket object that wraps the downstream connection. Only [

In case of error, `nil` will be returned as well as a string describing the error.

Due to the streaming nature of HTTP2 and HTTP3, this API cannot be used when the downstream connection is HTTP2 and HTTP3.

The socket object returned by this method is usually used to read the current request's body in a streaming fashion. Do not turn on the [lua_need_request_body](#lua_need_request_body) directive, and do not mix this call with [ngx.req.read_body](#ngxreqread_body) and [ngx.req.discard_body](#ngxreqdiscard_body).

If any request body data has been pre-read into the Nginx core request header buffer, the resulting cosocket object will take care of this to avoid potential data loss resulting from such pre-reading.
Expand Down Expand Up @@ -9321,12 +9369,6 @@ Only the following ngx_lua APIs could be used in `function_name` function of the
* `ngx.decode_args`
* `ngx.quote_sql_str`

* `ngx.re.match`
* `ngx.re.find`
* `ngx.re.gmatch`
* `ngx.re.sub`
* `ngx.re.gsub`

* `ngx.crc32_short`
* `ngx.crc32_long`
* `ngx.hmac_sha1`
Expand All @@ -9353,7 +9395,7 @@ The second argument `module_name` specifies the lua module name to execute in th

The third argument `func_name` specifies the function field in the module table as the second argument.

The type of `arg`s must be one of type below:
The type of `args` must be one of type below:

* boolean
* number
Expand Down