How to use different ExternalProxy for each https request ?

[GF-Huang]

In my test, when the first https request had responsed, the second request doesn't call ProxyServer.GetExternalProxyFunc.

I had try the following code, but not work:

private async Task OnBeforeRequest(object sender, SessionEventArgs e) {
    e.CustomUpStreamHttpProxyUsed = myProxy;  
    e.CustomUpStreamHttpsProxyUsed = myProxy;  
}

[justcoding121]

We don't yet support each HTTP request from a single client TCP connection to use different external proxy. However we do support changing external stream proxy on a per connection basis. That's why you see the call is getting hit only once at the beginning of connection establishment to server and not for subsequent HTTP requests over same TCP connection from client. The reason we haven't given much interest to do the same have to do with performance.

You may be aware that typically a series of HTTP request/response sequence is done over a TCP connection from client to proxy and then from proxy to external proxy. If you keep changing the external proxy for each HTTP request from same client connection then the proxy would have to each time drop existing TCP connection to server and create new one. This will greatly affect performance, defeating the purpose of HTTP 1.1 keep-alive feature.

Ideally a single connection from client should be piped to a single connection to same up stream proxy for HTTP keep-alive functionality to accomplish its purpose.

Having said that a way to mitigate the overhead would be to use a TCP connection cache in proxy. I think one day when we implement the cache we would support this feature. Implementation of a connection cache is not trivial since we need to make sure that the connection is in valid TCP state in cache and matches exactly the same end point, external proxy and any other things that identify a unique connection.

https://github.com/justcoding121/Titanium-Web-Proxy/blob/develop/Titanium.Web.Proxy/RequestHandler.cs#L364

[justcoding121]

Meanwhile if any one is interested in doing a short term solution to this by dropping and creating server connection when external proxy changes this is up for grab. That would mean calling GetCustomHttp(s)ProxyFunc methods for each request.

https://github.com/justcoding121/Titanium-Web-Proxy/blob/develop/Titanium.Web.Proxy/RequestHandler.cs#L588

[GF-Huang]

Great, Thanks your guide. I try to modify this.
By the way, is it SessionEventArgs.CustomUpStreamHttpProxyUsed & SessionEventArgs.CustomUpStreamHttpsProxyUsed not use in OnBeforeRequest ?

[honfika]

No, they are set when the server connection is created.
But it contains only the the result of your GetCustomUpStreamProxyFunc method's call.

[miroslavp]

Meanwhile if any one is interested in doing a short term solution to this by dropping and creating server connection when external proxy changes this is up for grab. That would mean calling GetCustomHttp(s)ProxyFunc methods for each request.

https://github.com/justcoding121/Titanium-Web-Proxy/blob/develop/Titanium.Web.Proxy/RequestHandler.cs#L588

The link is broken :( (404)

[justcoding121]

See #512

[honfika]

I close this issue due to inactivity. Feel free to reopen if needed.

[POMATu]

all links leading to the actual piece of code dead once again

[honfika]

Ok, but if you are a developer, it should not be hard to find a class in the repository at the given date...
Here it is:

titanium-web-proxy/Titanium.Web.Proxy/RequestHandler.cs

Line 588 in 572866b

if (args.WebSession.Request.IsHttps)

And the latest version:

titanium-web-proxy/src/Titanium.Web.Proxy/Network/Tcp/TcpConnectionFactory.cs

Lines 111 to 135 in 1236dce

	internal async Task<string> GetConnectionCacheKey(ProxyServer server, SessionEventArgsBase session,
	SslApplicationProtocol applicationProtocol)
	{
	List<SslApplicationProtocol>? applicationProtocols = null;
	if (applicationProtocol != default)
	{
	applicationProtocols = new List<SslApplicationProtocol> { applicationProtocol };
	}
	var customUpStreamProxy = session.CustomUpStreamProxy;
	bool isHttps = session.IsHttps;
	if (customUpStreamProxy == null && server.GetCustomUpStreamProxyFunc != null)
	{
	customUpStreamProxy = await server.GetCustomUpStreamProxyFunc(session);
	}
	session.CustomUpStreamProxyUsed = customUpStreamProxy;
	var uri = session.HttpClient.Request.RequestUri;
	var upStreamEndPoint = session.HttpClient.UpStreamEndPoint ?? server.UpStreamEndPoint;
	var upStreamProxy = customUpStreamProxy ?? (isHttps ? server.UpStreamHttpsProxy : server.UpStreamHttpProxy);
	return GetConnectionCacheKey(uri.Host, uri.Port, isHttps, applicationProtocols, upStreamEndPoint,
	upStreamProxy);
	}