New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Exclude %2F(/) from decoding of percents in a request path.
#3855
Conversation
Motivation: Currently, `%2F` and `%2f` are converted into `/` while decoding request paths. It might route a path to an unexpected service if users really want to send `%2F` as data itself. Furthermore, other specifications such as gRPC transcoding do not expect to decode `%2F` as `/`. https://github.com/googleapis/googleapis/blob/02710fa0ea5312d79d7fb986c9c9823fb41049a9/google/api/http.proto#L257-L258 It should be better to exclude `%2F` for less confusion and better interop with other echo systems. Modifications: - Exclude `%2F` and `%2f` from decoding percent-encoded characters in paths. Result: `%2F` and `%2f` are no longer converted to `/` when decoding a request path.
| wasSlash = decoded == '/'; | ||
| if (decoded == '/') { | ||
| // Do not decode '%2F' and '%2f' in the path to '/' for compatibility with | ||
| // other echo systems, e.g. HTTP/JSON to gRPC transcoding. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| // other echo systems, e.g. HTTP/JSON to gRPC transcoding. | |
| // other implementations in the ecosystem, e.g. HTTP/JSON to gRPC transcoding. |
| // Do not decode '%2F' and '%2f' in the path to '/' for compatibility with | ||
| // other echo systems, e.g. HTTP/JSON to gRPC transcoding. | ||
| // https://github.com/googleapis/googleapis/blob/02710fa0ea5312d79d7fb986c9c9823fb41049a9/google/api/http.proto#L257-L258 | ||
| final byte marker = RAW_CHAR_TO_MARKER['/']; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
How about adding the comment like we did for the other branch below:
// Insert a special mark so we can distinguish a raw character ('/') and
// percent-encoded character ('%2F') in a path string.
// We will encode this mark back into a percent-encoded character later.
|
@ikhoon Could you check the test failures? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Codecov Report
@@ Coverage Diff @@
## master #3855 +/- ##
============================================
+ Coverage 73.20% 73.31% +0.10%
- Complexity 15041 15285 +244
============================================
Files 1323 1332 +9
Lines 57902 58511 +609
Branches 7342 7410 +68
============================================
+ Hits 42387 42895 +508
- Misses 11775 11846 +71
- Partials 3740 3770 +30
Continue to review full report at Codecov.
|
Motivation: - We changed how `PathAndQuery` handles `%2F` (/) in 1.12.0 via #3855. This change introduces an unexpected hole in its double-dot detection logic. - Since we decided not to decode `%2F`, we should not decode it whereever possible. Modifications: - Hardened the double-dot detection logic in `PathAndQuery`. - `Bytes.data` now always store the bytes in their decoded form. We keep whether the byte has to be encoded in a separate `BitSet`. - Split `ArmeriaHttpUtil.decodePath()` into `decodePath()` and `decodePathParam()`. - We don't decode `%2F` in `decodePath()` but we do in `decodePathParam()`. - `RoutingResultBuilder.rawParam()` now uses `decodePathParam()` because `decodePath()` doesn't decode `%2F` anymore. Result: - A path that contains double dots with `%2F`, such as `/files/..%2Fsecrets.txt`, are now rejected correctly.
Motivation:
Currently,
%2Fand%2fare converted into/while decodingrequest paths. It might route a path to an unexpected service if users
really want to send
%2Fas data itself.Furthermore, other specifications such as gRPC transcoding do not
expect to decode
%2Fto/.https://github.com/googleapis/googleapis/blob/02710fa0ea5312d79d7fb986c9c9823fb41049a9/google/api/http.proto#L257-L258
It should be better to exclude
%2Ffor less confusion and betterinterop with other echo systems.
Modifications:
%2Fand%2ffrom decoding percent-encoded characters in paths.Result:
%2Fand%2fare no longer converted to/when decoding a request path.