Feat: Pass encoded characters in path unchanged

[valentin-krasontovitsch]

We enable passing substrings that represent encoded reserved symbols, like
%2f for /, in the path of a request.

NB: This does not enable using encoded characters in rewrite rules (e.g. in the
urlprefix directive).

Cf. #347 and #486

[CLAassistant]

All committers have signed the CLA.

[valentin-krasontovitsch]

So it turns out that one can utilize the internal mechanisms of net/url to handle encoded parts of an uri. One just has to keep track of RawPath, whenever necessary, which is what this commit attempts to do...

As a side note, I got carried away and tried enabling encoded characters in rewrite rules as well, and in the strip prefix, which worked fine, until you combined say a rule like

route add svc / http://bar.com/a%2fc/$path

with a request for

/%20

the space in the request won't get escaped, as it's unambiguous on its own (I guess, cf. docs for function that sets Path and RawPath in net/url package), but would get escaped in the context of %2f, i.e. if one looked at the whole url http://bar.com/a%2fc/%20.

One may inspect the more involved changes on my corresponding branch - in particular, a link to the failing test case.

Since I didn't see an easy and quick fix for that, I decided that it's maybe not the most needed use-case and that supporting what we have here might be enough.

[magiconair]

Does that mean this fixes #347 and #486?

[valentin-krasontovitsch]

I'm so sorry, saw your comment, but kinda ignored it O_o - let me see, I honestly don't know anymore, gotta check out the code... did I add some test cases? : )

... yes, I did. Please take a look at those, I'm sure you're more familiar with the code base and can say with more confidence which cases I have covered, and whether I forgot any cases / should add some cases.

But at least the test cases are covered! ... and I think, to answer your original question, yes, that solves those issues, as far as I understand.

[scalp42]

Any chance to see this merged @magiconair ?

[makepanic]

We were also hit with this issue.
Is there anything one can do to proceed with the issue/PR?

[aaronhurt]

@valentin-krasontovitsch Sorry for the lag in response around this issue. If you could resolve the conflicts I'll get this merged. Otherwise I'll try and fix the simple conflict myself when I get time.

[valentin-krasontovitsch]

No worries, I’ll try to take a look this weekend, if you don’t hear from me, feel free to attempt a rebase or just patch it directly yourself - I can’t really say that I recall, but I assume it wasn’t a complicated change...

…

On Mon, 27 Jan 2020 at 21:53, Aaron Hurt ***@***.***> wrote: @valentin-krasontovitsch <https://github.com/valentin-krasontovitsch> Sorry for the lag in response around this issue. If you could resolve the conflicts I'll get this merged. Otherwise I'll try and fix the simple conflict myself when I get time. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#489?email_source=notifications&email_token=ADZR2BLCDWDPWGJVWFQ4IL3Q75CV5A5CNFSM4E4NDFS2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEKBAUDQ#issuecomment-578947598>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADZR2BI7YPSDLJLSOPXRC7DQ75CV5ANCNFSM4E4NDFSQ> .

[valentin-krasontovitsch]

okay, ran a rebase, and luckily it was only a superficial conflict - a test added in the same place where i added a test. resolved, and pushed - after the travis build checks out, you will hopefully be able to merge.

[valentin-krasontovitsch]

Works on go1.13, but a handful tests break on go master? Very strange.
I presume somebody more familiar with the code base might have a better chance of working out why the tests fail... I can try to take a look myself, too, though.... might just take some more time.

[aaronhurt]

That happens sometimes. There is some odd race condition where vault doesn't launch before the test suite runs. I need to spend some time debugging that but for now I'll just re-run that branch. Thank you again!

[valentin-krasontovitsch]

hmmm OK I see I've experienced such things myself, sure, thanks!

.... I'm not sure what vault is though, and I started looking at the first test that failed TestTableLookup_656 and tried it with master locally myself, just running the subpackage go test -count=1 ./route, and it sometimes fails, sometimes don't - could that be due to a vault thing? it seems to my naive understanding to just be a simple redirect test... or is the vault the place where the table information is stored, and it happens async?

oh and cool thanks for merging : )