Kong is adding URL encoding before sending upstream

[tecnobrat]

Summary

In Kong 0.10.2 it is adding additional URL encoding before sending a request upstream.

Steps To Reproduce

1. Make a request to kong with /auth%7C123 (/auth|123, url-encoded).
2. Kong will make the request to your upstream as: /auth%257C123

Additional Details & Logs

• Kong version 0.10.2

This was not the case in Kong 0.9.9, it didn't further encode these URLs.

[thibaultcha]

Hi,

Thanks for the report. That might indeed very well be the case, we'll investigate.

@bungle Did you see this?

[bungle]

The relevant tests:
https://github.com/Mashape/kong/blob/master/spec/01-unit/11-router_spec.lua#L819-L833
https://github.com/Mashape/kong/blob/master/spec/02-integration/05-proxy/01-router_spec.lua#L175-L184

The possible bug:
https://github.com/Mashape/kong/blob/master/kong/core/router.lua#L687

I guess that you may need to url decode the uri before supplying it to ngx.req.set_uri.

Docs say:

Rewrite the current request's (parsed) URI by the uri argument. 

So either it is a bug in OpenResty, or it is expected behavior and requires you to decode before setting it. I suspect it is the latter.

[bungle]

@tecnobrat If you have time, mind you changing the line:
https://github.com/Mashape/kong/blob/master/kong/core/router.lua#L687

that has code:

ngx.req.set_uri(new_uri)

with

ngx.req.set_uri(ngx.unescape_uri(new_uri))

And report back if it had any effect.

[tecnobrat]

@bungle I'll try giving it a shot, I'll need to do some docker-fu and report back :)

[tecnobrat]

Hey @bungle yep, that worked.

[3dbrows]

@bungle Your fix also worked for me, thanks.

[thibaultcha]

I have created a regression test at #2519, but this test exposes that the currently proposed fix is incomplete. I believe that with this fix, Kong is making the upstream request with unescaped URIs (/auth|123) instead. Could you check? Thanks!

[p0pr0ck5]

@thibaultcha isn't the point here that the upstream request should be unescaped? @tecnobrat isn't that what you're looking for?

[thibaultcha]

No, the point is that Kong needs to be as transparent as possible, and an escaped request from downstream should be forwarded upstream as an escaped request as well. This is the expectation we set when we qualify Kong as a "transparent proxy".

[tecnobrat]

@thibaultcha is correct.

The request was pre-escaped. It should be escaped when it hits the resulting upstream. Currently its escaping it TWICE. But it most certainly should not be unescaped completely.

However the request was made to kong, it should be made to the upstream.

[thibaultcha]

Thanks for confirming that @tecnobrat.

However the request was made to kong, it should be made to the upstream.

That is indeed the expectation one might have, and certainly the one we'd like to promise, but we are limited by the current APIs that Nginx/OpenResty are offering us. We are investigating fixes, but so far we don't have a definitive one.

[pklingem]

is there any news on this? it's preventing us from upgrading from 0.9.x.

[thibaultcha]

I just updated #2519 with a fix that ensures we transparently proxy request URIs (without further escaping/un-escaping). Could you give it a try on your side and let us know? Thanks!