normalize_uri fixes (double slashes and trailing slash)

[wchen-r7]

This pull request fixes mainly two issues:

1. One of the intended behaviors for normalize_uri() is to eliminate all double slashes, but all it does is it removes the first one. So this pull request addresses the correct behavior. I also added more comments to better explain what the function does.
2. Another problem is normalize_uri() removes the trailing slash, but sometimes that slash is actually needed. For example: if a module wants TARGETURI to be a directory, you should NOT strip the trailing slash. If you do, you may end up getting a 301 or 404 instead of 200 (expected output). This pull request only modifies those that need TARGETURI to be a directory.

Related to the following ticket:
http://dev.metasploit.com/redmine/issues/7727#change-34153

[jlee-r7]

target_uri is a URI object. You probably wanted target_uri.path

[wchen-r7]

Yup, fixed.

[todb-r7]

You're sure it wouldn't be better to just return the old behavior? End with at most one slash, if the TARGET_URI ends with a slash? I feel like the last change that forced dropping slashes was wrong, now.

[todb-r7]

It restores the old "//" typos but that's the module's problem, not really the normalizer's problem. No?

[wchen-r7]

Egypt and I talked about this problem. Basically, if you go back to the old behavior, you risk having double slashes. If you use normalize_uri(), a missing trailing slash when by default is needed can cause the module to malfunction, too. You are stuck between a rock, and a hard place.

I feel like this is the most ideal thing to do because many modules already check if there's a missing trailing slash after it's normalized. Although this kind of remedies our current problem, we'll have to rethink how a URI string should be handled. At one point we even talked about having a URI.join(), kinda like File.join(), for joining URIs.

[todb-r7]

This is all covered with some pretty exhaustive rspec tests, turns out:

(target_uri_fix) todb@aus-linux-1030:~/git/rapid7/metasploit-framework/spec/lib/msf/core/exploit/http$ rspec client_spec.rb
.......................

Finished in 0.02553 seconds
23 examples, 0 failures

Thanks @limhoff-r7 !