Upstream HTTP proxy issues.

[mclarence]

Hello,
So I have the following code found here: https://pastebin.com/hAwUuwS6
Most of the code is just copy and paste from the example posted on the readme. What it currently does is launch a Chrome web driver (from selenium) instance and points it to the proxy created by titanium web proxy. The cmd window then accepts a ip address and port number and after pressing enter it sets the upstream proxy with the inputs specified by the user. This program was created only for testing purposes.

The upstream proxy works, it is able to connect to the proxy just fine but ~95% of the proxies I tried resulted in an error when attempting to browse to a website. The proxies return a "Invalid URL" error. Such as the error shown in the image bellow.

This is not an issue with the proxy itself since if I connect to it directly (without the titanium proxy), I am able to browse websites just fine.

I'm not an expert in C#, I could have possibly missed something very obvious, so please bear with me.

Thanks.

[justcoding121]

I got busy recently with build issues, I will check this asap

[honfika]

@justcoding121: For some reason the StreamExtended package is not installing automatically when I add the latest beta build of Titanium with nuget package manager (BouncyCastle is installed correctly - at least with the normal .NET FW version)

So @mclarence : please install the folllwing package https://www.nuget.org/packages/StreamExtended/
and ensure that the StreamExtended.dll is in your bin folder when you run the application.

Btw: the error handling should be better if this package is missing. I'll do that, but I'm also quite busy now because of my work...

[mclarence]

@honfika I have installed StreamExtended and made sure that the DLL file is in the same location as the executable. Is this supposed to fix my issue? Is there an additional step after this?

[honfika]

I don't know, but I tried it, and seems to work.
I created a new Console application, copied the code from pastebin, installed Selenium, Titanium and StreamExtended, started the application. Started another local proxy. Entered the 127.0.0.1 and the port of the another proxy, and loaded the website which is in your screenshot successfully.

[mclarence]

@honfika

With the screenshot above. Both browsers are connected through the following proxy (found on a public proxy list) 52.169.217.168 with port 3128. The firefox browser on the left is connected directly to the proxy (52.169.217.168:3128) whilst the chrome browser on the right is connected to the proxy (52.169.217.168:3128) via the titanium web proxy (127.0.0.1:8000), so basically the titanium proxy is the "man in the middle". As you can see the chrome browser on the right returns an invalid url error whilst the firefox browser on the left is able to load the website sucsessfully.

Now this does not happen with all proxies, lets say around 3 out of 10 proxies that I found on public proxy lists were able to load any website via the titanium web proxy, whilst being connected directly to the proxy (w/o the titanium proxy) is able to load all websites regardless of what proxy I use, provided the proxy is working properly.

Looking at the error on the chrome browser, it seems that the titanium web proxy is not passing the hostname, eg leaving out the "example.com" in http://example.com/s, leaving only the path behind.

[honfika]

Thanks, I was able to reproduce the problem with "your" proxy.
The problem was that TWP sent the following request to the upstream proxy:

GET /s HTTP/1.1
Host: canihazip.com

While it accepts only:

GET http://canihazip.com/s HTTP/1.1
Host: canihazip.com

The proxy I tried accepted the first version, too.

It is fixed in the develop branch. So you can get the sourcecode from github and build it, or wait for @justcoding121 to merge it and create a nuget package.

[mclarence]

Thank you very much. Now it seems to be working perfectly.

[justcoding121]

@honfika I hope this won't break #261 just for a caution. It's about using TWP itself as an external proxy. I will look in to why StreamExtended is not getting referenced. Thx for the fix.

[honfika]

@justcoding121 This change is only for http, so it should not affect #261
btw: In an earlier PR I configured to use C#6 in this project (to avoid build errors in your system). But now as far as I know you have VS 2017, so maybe this setting should be changed.
In the csproj files:
<LangVersion>6</LangVersion>
could be changed to
<LangVersion>default</LangVersion>

Or in the UI: Project properties/Build/Advanced/Language version

[justcoding121]

So that will make the project not be opened from VS 2015 correct? Most developers I know don't have VS 2017 installed yet. So I would think waiting little bit more would be good, so that if someone have a bug and they want to debug the code it would be possible.

[justcoding121]

FYI for all.

I opened a ticket on AppVeyor which throws UnAuthorized error very often
https://ci.appveyor.com/project/justcoding121/titanium-web-proxy

So we will have to wait until they resolve it for next beta release with this fix.