http inspector: use http_parser in http inspector #8808
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: crazyxy <yxyan@google.com>
|
Considering that http-parser needs to be replaced (#5155) should we increase its usage? Or will this code be portable / usable by llhttp (if that is what we use to replace http-parser)? |
|
The problem here is that current parsing logic in http inspector is different from http-parser. There might be some case that traffic is detected as non http, but http-parser can parse successfully, e.g. I didn't do investigation and don't know if llhttp will give the same result as http-parser when inspecting the incoming packets and I think we need to migrate this code to llhttp when http-parser is completely replaced by llhttp for http/1x codec. |
|
@moderation we can migrate both place to llhttp later if that's the choice, at this point the focus of this PR is use same parser so we get consistent result. llhttp has pretty similar interface to http_parser so increasing usage here shouldn't affect too much on migration. |
Signed-off-by: crazyxy <yxyan@google.com>
jplevyak
reviewed
Oct 30, 2019
| @@ -108,6 +128,7 @@ ParseState Filter::onRead() { | |||
|
|
|||
There was a problem hiding this comment.
Using MSG_PEEK repetitively leads to O(N^2) cost if the attacker sends 1 byte messages. The 8K limit is AFAICT only for Apache, other servers can accept longer URLs.
In many cases a URL can be padded out in a way that the server will still accept, for example by providing unnecessary query key-value pairs. This makes it possible to fool the parser and still have the URL accepted by the backend.
You can also use a callback to abort further parsing in on_headers_complete by returning an error and checking for CB_headers_complete failure before declaring a ParseState::Error.
There was a problem hiding this comment.
I updated the PR to validate the request line only as discussed in the below comment.
source/extensions/filters/listener/http_inspector/http_inspector.cc
Outdated
Show resolved
Hide resolved
lizan
suggested changes
Oct 30, 2019
source/extensions/filters/listener/http_inspector/http_inspector.cc
Outdated
Show resolved
Hide resolved
Signed-off-by: crazyxy <yxyan@google.com>
Signed-off-by: crazyxy <yxyan@google.com>
|
Hmm.. I added a test for larger request line and the test failed under both asan and tasn. asan failed with error tsan failed silently Working on it |
|
It seems the ci failure is not related with the PR |
lambdai
suggested changes
Nov 1, 2019
There was a problem hiding this comment.
http_parser support stream interface
The semantic of http_parser_execute is feeding new data.
We should use the assumption that the data we peek last time should remain the same. Suppose last time we see 6k data, and this time we see 8k, we should feed the slice of (6k, 8k) using http_parser_execute(&parser_/last state/, &setting_, request_line.data() + 6k, request_line.length()-6k);
source/extensions/filters/listener/http_inspector/http_inspector.cc
Outdated
Show resolved
Hide resolved
lizan
reviewed
Nov 1, 2019
source/extensions/filters/listener/http_inspector/http_inspector.cc
Outdated
Show resolved
Hide resolved
source/extensions/filters/listener/http_inspector/http_inspector.h
Outdated
Show resolved
Hide resolved
source/extensions/filters/listener/http_inspector/http_inspector.h
Outdated
Show resolved
Hide resolved
test/extensions/filters/listener/http_inspector/http_inspector_test.cc
Outdated
Show resolved
Hide resolved
Signed-off-by: Yan Xue <yxyan@google.com>
lizan
reviewed
Nov 1, 2019
| .WillOnce( | ||
| Invoke([&data, len](int, void* buffer, size_t length, int) -> Api::SysCallSizeResult { | ||
| ASSERT(length >= data.size()); | ||
| memcpy(buffer, data.data(), data.size()); |
There was a problem hiding this comment.
len instead of data.size()? (ditto for below)
There was a problem hiding this comment.
@yxue I meant len here exactly 😅to copy less memory, not size() vs length().
There was a problem hiding this comment.
😳 let me change it back
lizan
approved these changes
Nov 3, 2019
PiotrSikora
pushed a commit
to PiotrSikora/envoy
that referenced
this pull request
Nov 9, 2019
Description: use http_parser in http inspector Risk Level: Med Testing: Unit test Docs Changes: N/A Release Notes: N/A Signed-off-by: crazyxy <yxyan@google.com> Signed-off-by: Piotr Sikora <piotrsikora@google.com>
howardjohn
pushed a commit
to istio/envoy
that referenced
this pull request
Nov 9, 2019
…120) Description: use http_parser in http inspector Risk Level: Med Testing: Unit test Docs Changes: N/A Release Notes: N/A Signed-off-by: crazyxy <yxyan@google.com> Signed-off-by: Piotr Sikora <piotrsikora@google.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Signed-off-by: crazyxy yxyan@google.com
Description: use http_parser in http inspector
Risk Level: Med
Testing: Unit test
Docs Changes: N/A
Release Notes: N/A
[Optional Fixes #Issue]
[Optional Deprecated:]