Incorrectly decoding query parameters #749
Assignees
Labels
Comments
|
After a quick look through the Kong source, my guess is that percent-encoding needs to be performed in the Technically both the param names and values should be percent-encoded. (See RFC 3986 section 3.4. The query string is made up of |
|
The method you pointed out is only used when tests are running ( Let me quickly explain what is going on here: since Kong needs to eventually modify some query parameters (like the request-transformation plugin), they are detached from the URI, processed by plugins, and re-attached to the URI at the very last moment before proxying the request. The problem seems to be the result of several assumptions made by ngx_lua:
There is an opened pull request which addresses this on ngx_lua: openresty/lua-nginx-module#542 which seems promising, but it is hanging. local args = {foo = ",$@|`"} -- percent-decoded by ngx.get_uri_args()
-- current behavior
ngx.encode_args(plain) -- foo=,$@|`
-- expected behavior
ngx.encode_args(plain) -- foo=%2C%24%40%7C%60For now, I see no choice but to iterate through the query values and percent-encode each one of them with another function, such as LuaSocket's url.escape. |
thibaultcha
added a commit
that referenced
this issue
Dec 1, 2015
Percent-encode query args when re-attaching them to the `upstream_uri`. Since `ngx.encode_args` does not perform percent-encoding on various reserved characters, this implements a custom `utils.encode_args` function which uses LuaSocket's `url.encode` function. It tries to mimic the `ngx.encode_uri` behaviour 100%. Ideally, `ngx.encode_args` would proceed to the percent-encoding itself (see openresty/lua-nginx-module#542). This also makes some perf and style changes. Fix #749
|
#752 should solve that, and should be included in the next major release ( |
|
Thanks! Feels good to wake up in the morning and find that both my issues have been fixed overnight :) |
kikito
pushed a commit
that referenced
this issue
Apr 23, 2024
### Summary
#### 2.6.0
```
Release 2.6.0 Tue February 6 2024
Security fixes:
#789 #814 CVE-2023-52425 -- Fix quadratic runtime issues with big tokens
that can cause denial of service, in partial where
dealing with compressed XML input. Applications
that parsed a document in one go -- a single call to
functions XML_Parse or XML_ParseBuffer -- were not affected.
The smaller the chunks/buffers you use for parsing
previously, the bigger the problem prior to the fix.
Backporters should be careful to no omit parts of
pull request #789 and to include earlier pull request #771,
in order to not break the fix.
#777 CVE-2023-52426 -- Fix billion laughs attacks for users
compiling *without* XML_DTD defined (which is not common).
Users with XML_DTD defined have been protected since
Expat >=2.4.0 (and that was CVE-2013-0340 back then).
Bug fixes:
#753 Fix parse-size-dependent "invalid token" error for
external entities that start with a byte order mark
#780 Fix NULL pointer dereference in setContext via
XML_ExternalEntityParserCreate for compilation with
XML_DTD undefined
#812 #813 Protect against closing entities out of order
Other changes:
#723 Improve support for arc4random/arc4random_buf
#771 #788 Improve buffer growth in XML_GetBuffer and XML_Parse
#761 #770 xmlwf: Support --help and --version
#759 #770 xmlwf: Support custom buffer size for XML_GetBuffer and read
#744 xmlwf: Improve language and URL clickability in help output
#673 examples: Add new example "element_declarations.c"
#764 Be stricter about macro XML_CONTEXT_BYTES at build time
#765 Make inclusion to expat_config.h consistent
#726 #727 Autotools: configure.ac: Support --disable-maintainer-mode
#678 #705 ..
#706 #733 #792 Autotools: Sync CMake templates with CMake 3.26
#795 Autotools: Make installation of shipped man page doc/xmlwf.1
independent of docbook2man availability
#815 Autotools|CMake: Add missing -DXML_STATIC to pkg-config file
section "Cflags.private" in order to fix compilation
against static libexpat using pkg-config on Windows
#724 #751 Autotools|CMake: Require a C99 compiler
(a de-facto requirement already since Expat 2.2.2 of 2017)
#793 Autotools|CMake: Fix PACKAGE_BUGREPORT variable
#750 #786 Autotools|CMake: Make test suite require a C++11 compiler
#749 CMake: Require CMake >=3.5.0
#672 CMake: Lowercase off_t and size_t to help a bug in Meson
#746 CMake: Sort xmlwf sources alphabetically
#785 CMake|Windows: Fix generation of DLL file version info
#790 CMake: Build tests/benchmark/benchmark.c as well for
a build with -DEXPAT_BUILD_TESTS=ON
#745 #757 docs: Document the importance of isFinal + adjust tests
accordingly
#736 docs: Improve use of "NULL" and "null"
#713 docs: Be specific about version of XML (XML 1.0r4)
and version of C (C99); (XML 1.0r5 will need a sponsor.)
#762 docs: reference.html: Promote function XML_ParseBuffer more
#779 docs: reference.html: Add HTML anchors to XML_* macros
#760 docs: reference.html: Upgrade to OK.css 1.2.0
#763 #739 docs: Fix typos
#696 docs|CI: Use HTTPS URLs instead of HTTP at various places
#669 #670 ..
#692 #703 ..
#733 #772 Address compiler warnings
#798 #800 Address clang-tidy warnings
#775 #776 Version info bumped from 9:10:8 (libexpat*.so.1.8.10)
to 10:0:9 (libexpat*.so.1.9.0); see https://verbump.de/
for what these numbers do
Infrastructure:
#700 #701 docs: Document security policy in file SECURITY.md
#766 docs: Improve parse buffer variables in-code documentation
#674 #738 ..
#740 #747 ..
#748 #781 #782 Refactor coverage and conformance tests
#714 #716 Refactor debug level variables to unsigned long
#671 Improve handling of empty environment variable value
in function getDebugLevel (without visible user effect)
#755 #774 ..
#758 #783 ..
#784 #787 tests: Improve test coverage with regard to parse chunk size
#660 #797 #801 Fuzzing: Improve fuzzing coverage
#367 #799 Fuzzing|CI: Start running OSS-Fuzz fuzzing regression tests
#698 #721 CI: Resolve some Travis CI leftovers
#669 CI: Be robust towards absence of Git tags
#693 #694 CI: Set permissions to "contents: read" for security
#709 CI: Pin all GitHub Actions to specific commits for security
#739 CI: Reject spelling errors using codespell
#798 CI: Enforce clang-tidy clean code
#773 #808 ..
#809 #810 CI: Upgrade Clang from 15 to 18
#796 CI: Start using Clang's Control Flow Integrity sanitizer
#675 #720 #722 CI: Adapt to breaking changes in GitHub Actions Ubuntu images
#689 CI: Adapt to breaking changes in Clang/LLVM Debian packaging
#763 CI: Adapt to breaking changes in codespell
#803 CI: Adapt to breaking changes in Cppcheck
Special thanks to:
Ivan Galkin
Joyce Brum
Philippe Antoine
Rhodri James
Snild Dolkow
spookyahell
Steven Garske
and
Clang AddressSanitizer
Clang UndefinedBehaviorSanitizer
codespell
GCC Farm Project
OSS-Fuzz
Sony Mobile
```
#### 2.6.1
```
Release 2.6.1 Thu February 29 2024
Bug fixes:
#817 Make tests independent of CPU speed, and thus more robust
#828 #836 Expose billion laughs API with XML_DTD defined and
XML_GE undefined, regression from 2.6.0
Other changes:
#829 Hide test-only code behind new internal macro
#833 Autotools: Reject expat_config.h.in defining SIZEOF_VOID_P
#819 Address compiler warnings
#832 #834 Version info bumped from 10:0:9 (libexpat*.so.1.9.0)
to 10:1:9 (libexpat*.so.1.9.1); see https://verbump.de/
for what these numbers do
Infrastructure:
#818 CI: Adapt to breaking changes in clang-format
Special thanks to:
David Hall
Snild Dolkow
```
#### 2.6.2
```
Release 2.6.2 Wed March 13 2024
Security fixes:
#839 #842 CVE-2024-28757 -- Prevent billion laughs attacks with
isolated use of external parsers. Please see the commit
message of commit 1d50b80cf31de87750103656f6eb693746854aa8
for details.
Bug fixes:
#839 #841 Reject direct parameter entity recursion
and avoid the related undefined behavior
Other changes:
#847 Autotools: Fix build for DOCBOOK_TO_MAN containing spaces
#837 Add missing #821 and #824 to 2.6.1 change log
#838 #843 Version info bumped from 10:1:9 (libexpat*.so.1.9.1)
to 10:2:9 (libexpat*.so.1.9.2); see https://verbump.de/
for what these numbers do
Special thanks to:
Philippe Antoine
Tomas Korbar
and
Clang UndefinedBehaviorSanitizer
OSS-Fuzz / ClusterFuzz
```
Signed-off-by: Aapo Talvensaari <aapo.talvensaari@gmail.com>
bungle
added a commit
that referenced
this issue
Apr 23, 2024
### Summary
#### 2.6.0
```
Release 2.6.0 Tue February 6 2024
Security fixes:
#789 #814 CVE-2023-52425 -- Fix quadratic runtime issues with big tokens
that can cause denial of service, in partial where
dealing with compressed XML input. Applications
that parsed a document in one go -- a single call to
functions XML_Parse or XML_ParseBuffer -- were not affected.
The smaller the chunks/buffers you use for parsing
previously, the bigger the problem prior to the fix.
Backporters should be careful to no omit parts of
pull request #789 and to include earlier pull request #771,
in order to not break the fix.
#777 CVE-2023-52426 -- Fix billion laughs attacks for users
compiling *without* XML_DTD defined (which is not common).
Users with XML_DTD defined have been protected since
Expat >=2.4.0 (and that was CVE-2013-0340 back then).
Bug fixes:
#753 Fix parse-size-dependent "invalid token" error for
external entities that start with a byte order mark
#780 Fix NULL pointer dereference in setContext via
XML_ExternalEntityParserCreate for compilation with
XML_DTD undefined
#812 #813 Protect against closing entities out of order
Other changes:
#723 Improve support for arc4random/arc4random_buf
#771 #788 Improve buffer growth in XML_GetBuffer and XML_Parse
#761 #770 xmlwf: Support --help and --version
#759 #770 xmlwf: Support custom buffer size for XML_GetBuffer and read
#744 xmlwf: Improve language and URL clickability in help output
#673 examples: Add new example "element_declarations.c"
#764 Be stricter about macro XML_CONTEXT_BYTES at build time
#765 Make inclusion to expat_config.h consistent
#726 #727 Autotools: configure.ac: Support --disable-maintainer-mode
#678 #705 ..
#706 #733 #792 Autotools: Sync CMake templates with CMake 3.26
#795 Autotools: Make installation of shipped man page doc/xmlwf.1
independent of docbook2man availability
#815 Autotools|CMake: Add missing -DXML_STATIC to pkg-config file
section "Cflags.private" in order to fix compilation
against static libexpat using pkg-config on Windows
#724 #751 Autotools|CMake: Require a C99 compiler
(a de-facto requirement already since Expat 2.2.2 of 2017)
#793 Autotools|CMake: Fix PACKAGE_BUGREPORT variable
#750 #786 Autotools|CMake: Make test suite require a C++11 compiler
#749 CMake: Require CMake >=3.5.0
#672 CMake: Lowercase off_t and size_t to help a bug in Meson
#746 CMake: Sort xmlwf sources alphabetically
#785 CMake|Windows: Fix generation of DLL file version info
#790 CMake: Build tests/benchmark/benchmark.c as well for
a build with -DEXPAT_BUILD_TESTS=ON
#745 #757 docs: Document the importance of isFinal + adjust tests
accordingly
#736 docs: Improve use of "NULL" and "null"
#713 docs: Be specific about version of XML (XML 1.0r4)
and version of C (C99); (XML 1.0r5 will need a sponsor.)
#762 docs: reference.html: Promote function XML_ParseBuffer more
#779 docs: reference.html: Add HTML anchors to XML_* macros
#760 docs: reference.html: Upgrade to OK.css 1.2.0
#763 #739 docs: Fix typos
#696 docs|CI: Use HTTPS URLs instead of HTTP at various places
#669 #670 ..
#692 #703 ..
#733 #772 Address compiler warnings
#798 #800 Address clang-tidy warnings
#775 #776 Version info bumped from 9:10:8 (libexpat*.so.1.8.10)
to 10:0:9 (libexpat*.so.1.9.0); see https://verbump.de/
for what these numbers do
Infrastructure:
#700 #701 docs: Document security policy in file SECURITY.md
#766 docs: Improve parse buffer variables in-code documentation
#674 #738 ..
#740 #747 ..
#748 #781 #782 Refactor coverage and conformance tests
#714 #716 Refactor debug level variables to unsigned long
#671 Improve handling of empty environment variable value
in function getDebugLevel (without visible user effect)
#755 #774 ..
#758 #783 ..
#784 #787 tests: Improve test coverage with regard to parse chunk size
#660 #797 #801 Fuzzing: Improve fuzzing coverage
#367 #799 Fuzzing|CI: Start running OSS-Fuzz fuzzing regression tests
#698 #721 CI: Resolve some Travis CI leftovers
#669 CI: Be robust towards absence of Git tags
#693 #694 CI: Set permissions to "contents: read" for security
#709 CI: Pin all GitHub Actions to specific commits for security
#739 CI: Reject spelling errors using codespell
#798 CI: Enforce clang-tidy clean code
#773 #808 ..
#809 #810 CI: Upgrade Clang from 15 to 18
#796 CI: Start using Clang's Control Flow Integrity sanitizer
#675 #720 #722 CI: Adapt to breaking changes in GitHub Actions Ubuntu images
#689 CI: Adapt to breaking changes in Clang/LLVM Debian packaging
#763 CI: Adapt to breaking changes in codespell
#803 CI: Adapt to breaking changes in Cppcheck
Special thanks to:
Ivan Galkin
Joyce Brum
Philippe Antoine
Rhodri James
Snild Dolkow
spookyahell
Steven Garske
and
Clang AddressSanitizer
Clang UndefinedBehaviorSanitizer
codespell
GCC Farm Project
OSS-Fuzz
Sony Mobile
```
#### 2.6.1
```
Release 2.6.1 Thu February 29 2024
Bug fixes:
#817 Make tests independent of CPU speed, and thus more robust
#828 #836 Expose billion laughs API with XML_DTD defined and
XML_GE undefined, regression from 2.6.0
Other changes:
#829 Hide test-only code behind new internal macro
#833 Autotools: Reject expat_config.h.in defining SIZEOF_VOID_P
#819 Address compiler warnings
#832 #834 Version info bumped from 10:0:9 (libexpat*.so.1.9.0)
to 10:1:9 (libexpat*.so.1.9.1); see https://verbump.de/
for what these numbers do
Infrastructure:
#818 CI: Adapt to breaking changes in clang-format
Special thanks to:
David Hall
Snild Dolkow
```
#### 2.6.2
```
Release 2.6.2 Wed March 13 2024
Security fixes:
#839 #842 CVE-2024-28757 -- Prevent billion laughs attacks with
isolated use of external parsers. Please see the commit
message of commit 1d50b80cf31de87750103656f6eb693746854aa8
for details.
Bug fixes:
#839 #841 Reject direct parameter entity recursion
and avoid the related undefined behavior
Other changes:
#847 Autotools: Fix build for DOCBOOK_TO_MAN containing spaces
#837 Add missing #821 and #824 to 2.6.1 change log
#838 #843 Version info bumped from 10:1:9 (libexpat*.so.1.9.1)
to 10:2:9 (libexpat*.so.1.9.2); see https://verbump.de/
for what these numbers do
Special thanks to:
Philippe Antoine
Tomas Korbar
and
Clang UndefinedBehaviorSanitizer
OSS-Fuzz / ClusterFuzz
```
Signed-off-by: Aapo Talvensaari <aapo.talvensaari@gmail.com>
tysoekong
pushed a commit
that referenced
this issue
Apr 26, 2024
```
Release 2.6.0 Tue February 6 2024
Security fixes:
#789 #814 CVE-2023-52425 -- Fix quadratic runtime issues with big tokens
that can cause denial of service, in partial where
dealing with compressed XML input. Applications
that parsed a document in one go -- a single call to
functions XML_Parse or XML_ParseBuffer -- were not affected.
The smaller the chunks/buffers you use for parsing
previously, the bigger the problem prior to the fix.
Backporters should be careful to no omit parts of
pull request #789 and to include earlier pull request #771,
in order to not break the fix.
#777 CVE-2023-52426 -- Fix billion laughs attacks for users
compiling *without* XML_DTD defined (which is not common).
Users with XML_DTD defined have been protected since
Expat >=2.4.0 (and that was CVE-2013-0340 back then).
Bug fixes:
#753 Fix parse-size-dependent "invalid token" error for
external entities that start with a byte order mark
#780 Fix NULL pointer dereference in setContext via
XML_ExternalEntityParserCreate for compilation with
XML_DTD undefined
#812 #813 Protect against closing entities out of order
Other changes:
#723 Improve support for arc4random/arc4random_buf
#771 #788 Improve buffer growth in XML_GetBuffer and XML_Parse
#761 #770 xmlwf: Support --help and --version
#759 #770 xmlwf: Support custom buffer size for XML_GetBuffer and read
#744 xmlwf: Improve language and URL clickability in help output
#673 examples: Add new example "element_declarations.c"
#764 Be stricter about macro XML_CONTEXT_BYTES at build time
#765 Make inclusion to expat_config.h consistent
#726 #727 Autotools: configure.ac: Support --disable-maintainer-mode
#678 #705 ..
#706 #733 #792 Autotools: Sync CMake templates with CMake 3.26
#795 Autotools: Make installation of shipped man page doc/xmlwf.1
independent of docbook2man availability
#815 Autotools|CMake: Add missing -DXML_STATIC to pkg-config file
section "Cflags.private" in order to fix compilation
against static libexpat using pkg-config on Windows
#724 #751 Autotools|CMake: Require a C99 compiler
(a de-facto requirement already since Expat 2.2.2 of 2017)
#793 Autotools|CMake: Fix PACKAGE_BUGREPORT variable
#750 #786 Autotools|CMake: Make test suite require a C++11 compiler
#749 CMake: Require CMake >=3.5.0
#672 CMake: Lowercase off_t and size_t to help a bug in Meson
#746 CMake: Sort xmlwf sources alphabetically
#785 CMake|Windows: Fix generation of DLL file version info
#790 CMake: Build tests/benchmark/benchmark.c as well for
a build with -DEXPAT_BUILD_TESTS=ON
#745 #757 docs: Document the importance of isFinal + adjust tests
accordingly
#736 docs: Improve use of "NULL" and "null"
#713 docs: Be specific about version of XML (XML 1.0r4)
and version of C (C99); (XML 1.0r5 will need a sponsor.)
#762 docs: reference.html: Promote function XML_ParseBuffer more
#779 docs: reference.html: Add HTML anchors to XML_* macros
#760 docs: reference.html: Upgrade to OK.css 1.2.0
#763 #739 docs: Fix typos
#696 docs|CI: Use HTTPS URLs instead of HTTP at various places
#669 #670 ..
#692 #703 ..
#733 #772 Address compiler warnings
#798 #800 Address clang-tidy warnings
#775 #776 Version info bumped from 9:10:8 (libexpat*.so.1.8.10)
to 10:0:9 (libexpat*.so.1.9.0); see https://verbump.de/
for what these numbers do
Infrastructure:
#700 #701 docs: Document security policy in file SECURITY.md
#766 docs: Improve parse buffer variables in-code documentation
#674 #738 ..
#740 #747 ..
#748 #781 #782 Refactor coverage and conformance tests
#714 #716 Refactor debug level variables to unsigned long
#671 Improve handling of empty environment variable value
in function getDebugLevel (without visible user effect)
#755 #774 ..
#758 #783 ..
#784 #787 tests: Improve test coverage with regard to parse chunk size
#660 #797 #801 Fuzzing: Improve fuzzing coverage
#367 #799 Fuzzing|CI: Start running OSS-Fuzz fuzzing regression tests
#698 #721 CI: Resolve some Travis CI leftovers
#669 CI: Be robust towards absence of Git tags
#693 #694 CI: Set permissions to "contents: read" for security
#709 CI: Pin all GitHub Actions to specific commits for security
#739 CI: Reject spelling errors using codespell
#798 CI: Enforce clang-tidy clean code
#773 #808 ..
#809 #810 CI: Upgrade Clang from 15 to 18
#796 CI: Start using Clang's Control Flow Integrity sanitizer
#675 #720 #722 CI: Adapt to breaking changes in GitHub Actions Ubuntu images
#689 CI: Adapt to breaking changes in Clang/LLVM Debian packaging
#763 CI: Adapt to breaking changes in codespell
#803 CI: Adapt to breaking changes in Cppcheck
Special thanks to:
Ivan Galkin
Joyce Brum
Philippe Antoine
Rhodri James
Snild Dolkow
spookyahell
Steven Garske
and
Clang AddressSanitizer
Clang UndefinedBehaviorSanitizer
codespell
GCC Farm Project
OSS-Fuzz
Sony Mobile
```
```
Release 2.6.1 Thu February 29 2024
Bug fixes:
#817 Make tests independent of CPU speed, and thus more robust
#828 #836 Expose billion laughs API with XML_DTD defined and
XML_GE undefined, regression from 2.6.0
Other changes:
#829 Hide test-only code behind new internal macro
#833 Autotools: Reject expat_config.h.in defining SIZEOF_VOID_P
#819 Address compiler warnings
#832 #834 Version info bumped from 10:0:9 (libexpat*.so.1.9.0)
to 10:1:9 (libexpat*.so.1.9.1); see https://verbump.de/
for what these numbers do
Infrastructure:
#818 CI: Adapt to breaking changes in clang-format
Special thanks to:
David Hall
Snild Dolkow
```
```
Release 2.6.2 Wed March 13 2024
Security fixes:
#839 #842 CVE-2024-28757 -- Prevent billion laughs attacks with
isolated use of external parsers. Please see the commit
message of commit 1d50b80cf31de87750103656f6eb693746854aa8
for details.
Bug fixes:
#839 #841 Reject direct parameter entity recursion
and avoid the related undefined behavior
Other changes:
#847 Autotools: Fix build for DOCBOOK_TO_MAN containing spaces
#837 Add missing #821 and #824 to 2.6.1 change log
#838 #843 Version info bumped from 10:1:9 (libexpat*.so.1.9.1)
to 10:2:9 (libexpat*.so.1.9.2); see https://verbump.de/
for what these numbers do
Special thanks to:
Philippe Antoine
Tomas Korbar
and
Clang UndefinedBehaviorSanitizer
OSS-Fuzz / ClusterFuzz
```
KAG-4331
Signed-off-by: Aapo Talvensaari <aapo.talvensaari@gmail.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Exactly the same problem as #688 (which I can confirm was fixed in Kong 0.5.3), but this time for query parameters.
If I hit
/foo?param=abc%7Cdefvia Kong, the%7C(pipe character) gets decoded and so the URL is passed through to the backend as/foo?param=abc|def. This causes my backend server (Spray) to reject the request.The text was updated successfully, but these errors were encountered: