Feature Request - Add proxy settings on ChromiumWebBrowser instantiation

[Velka-DEV]

Describe the solution you'd like
Actually we have 2 possibility to set proxies if i'm not wrong. We have the built-in helper with Cef.Initialize() but it will set same proxy for all ChromiumWebBrowser(). And we have the RequestContext.SetPreference("proxy", dict, out error); at runtime.

I don't uderstand why we can't set a proxy on browser initialization, so we can isolate each browsers without have to call SetPreference() at runtime that can cause problems.

Checklist:

• [ X] I have reviewed the CEF API and have confirmed the feature I'm requesting is possible.
• [? ] The Feature I'm requesting is an improvement to Async Javascript Binding (No new feature are being added to the Sync Javascript binding)
• [. ] An open PR exists on the CEF Pull Requests and I'd like to propose this feature is added to CefSharp when/if it's merged.

[amaitland]

I have reviewed the CEF API and have confirmed the feature I'm requesting is possible.

Based on your review of the CEF API what is your proposal for implementing this feature?

Are planning on submitting a PR?

[Velka-DEV]

I know it's actually possible to start CEF with command lines arguments to set proxies for each chromium process. So it's not possible to wrap CEF launch with these arguments ? (I'm not experienced with CEF, if you know will never be possible with actual API, i will trust you)

Sources: https://bitbucket.org/chromiumembedded/cef/wiki/GeneralUsage.md#markdown-header-proxy-resolution

[amaitland]

I know it's actually possible to start CEF with command lines arguments to set proxies for each chromium process. So it's not possible to wrap CEF launch with these arguments ? (I'm not experienced with CEF, if you know will never be possible with actual API, i will trust you)

We already provide a wrapper around these command line args.

CefSharp/CefSharp.Core/Internals/CefSharpApp.h

Line 139 in 53f15f2

if (CefSharpSettings::Proxy != nullptr && !_cefSettings->CommandLineArgsDisabled)

CefSharp/CefSharp/CefSharpSettings.cs

Line 77 in 5e89f71

public static ProxyOptions Proxy { get; set; }

• Once set via command line the proxy cannot be changed.
• The settings are also global to the process.
• You cannot set a proxy with different settings within the same process.

There's nothing stopping you from launching multiple instances of your application with unique CachePath and doing this with the current API. The same command line args linked above will work with CefSharp.

I don't uderstand why we can't set a proxy on browser initialization, so we can isolate each browsers without have to call SetPreference() at runtime that can cause problems.

Setting different proxies within the same process can only be achieve via using a different RequestContext per browser and calling SetPreference, this is currently the only API CEF provides.

We could add some helper methods to simplify this, though exactly what that would look like isn't clear at this point in time.

• Anyone is welcome to submit a PR for review
• Alternatively if people would like to sponsor development (via GitHub sponsors) then I can look at simplifying setting a proxy per ChromiumWebBrowser instance (fundamentally this would use RequestContext SetPreference). I require financial support from the community to continue developing CefSharp see https://github.com/cefsharp/CefSharp#financial-support

[Velka-DEV]

Alternatively if people would like to sponsor development (via GitHub sponsors) then I can look at simplifying setting a proxy per ChromiumWebBrowser instance (fundamentally this would use RequestContext SetPreference). I require financial support from the community to continue developing CefSharp see https://github.com/cefsharp/CefSharp#financial-support

I am not rich but if it can help you ^^

[amaitland]

@Velka-DEV Thank you for the generious support 👍

Are you using WinForms/WPF/OffScreen?

Can provide an example of the current code you are using?

[Velka-DEV]

I'm actually using WinForms. There is a sample of my EmbedBrowser class.

public partial class MainEmbedBrowser : Form
    {
        private string ID;
        private ChromiumWebBrowser browser;
        private RequestContext requestContext;
        private ExtensionHandler extensionHandler;

        public MainEmbedBrowser(string url, string combo, string proxy = null, int maxPlays = 0)
        {
            InitializeComponent();

            this.ID = CreateMD5(DateTime.Now.Millisecond.ToString());
            this.extensionHandler = new ExtensionHandler();
            this.requestContext = new RequestContext(); // EDIT: Or add proxy into RequestContext() constructor

           // Extension - Here is an attemp to load unpacked CRX unpacked extension but throw error. I don't know if CEF can support it correctly
           //this.extensionHandler = new ExtensionHandler();
           //this.browser.RequestContext.LoadExtension("/vendors/static/fcfhplploccackoneaefokcmbjfbkenj", "/vendors/static/fcfhplploccackoneaefokcmbjfbkenj/manifest.json", extensionHandler);

            // I think that can be a good idea to add proxy here. 
            this.browser = new ChromiumWebBrowser("https://exemple.com/", this.requestContext, "socks5://0.0.0.0:12345" or BrowserProxy("socks5://0.0.0.0:12345"));


            browser.FrameLoadEnd += (sender, args) =>
            {
                if (args.Frame.IsMain)
                {
                    if (args.Url.ToLower().Contains("https://exemple.com/")) 
                    {
                        // Do something on the loaded page 
                    }
                }
            };

            this.Controls.Add(browser);
            this.browser.Dock = DockStyle.Fill;
            this.Show();
        }

        public static string CreateMD5(string input)
        {
            using (System.Security.Cryptography.MD5 md5 = System.Security.Cryptography.MD5.Create())
            {
                byte[] inputBytes = Encoding.ASCII.GetBytes(input);
                byte[] hashBytes = md5.ComputeHash(inputBytes);

                StringBuilder sb = new StringBuilder();
                for (int i = 0; i < hashBytes.Length; i++)
                {
                    sb.Append(hashBytes[i].ToString("X2"));
                }
                return sb.ToString();
            }
        }
    }
}

[amaitland]

this.requestContext = new RequestContext(); // EDIT: Or add proxy into RequestContext() constructor

As the proxy is set per RequestContext having a easy method to directly create a RequestContext with proxy settings is likely the option we'll go with.

I'll post something here shortly for review. I'll include some improvements in the next version

---

In the mean time to get you up and running, here is an example of using a IRequestContextHandler to set the proxy

public partial class MainEmbedBrowser : Form
{
        private string ID;
        private ChromiumWebBrowser browser;
        private RequestContext requestContext;
        private ExtensionHandler extensionHandler;

        public MainEmbedBrowser(string url, string combo, string proxy = null, int maxPlays = 0)
        {
            InitializeComponent();

            this.ID = CreateMD5(DateTime.Now.Millisecond.ToString());
            this.extensionHandler = new ExtensionHandler();
            this.requestContext = new RequestContext(new RequestContextHandler());

           // Extension - Here is an attemp to load unpacked CRX unpacked extension but throw error. I don't know if CEF can support it correctly
           //this.extensionHandler = new ExtensionHandler();
           //this.browser.RequestContext.LoadExtension("/vendors/static/fcfhplploccackoneaefokcmbjfbkenj", "/vendors/static/fcfhplploccackoneaefokcmbjfbkenj/manifest.json", extensionHandler);

            // I think that can be a good idea to add proxy here. 
            this.browser = new ChromiumWebBrowser("https://exemple.com/", this.requestContext);

            browser.FrameLoadEnd += (sender, args) =>
            {
                if (args.Frame.IsMain)
                {
                    if (args.Url.ToLower().Contains("https://exemple.com/")) 
                    {
                        // Do something on the loaded page 
                    }
                }
            };

            this.Controls.Add(browser);
            this.browser.Dock = DockStyle.Fill;
            this.Show();
        }

        public static string CreateMD5(string input)
        {
            using (System.Security.Cryptography.MD5 md5 = System.Security.Cryptography.MD5.Create())
            {
                byte[] inputBytes = Encoding.ASCII.GetBytes(input);
                byte[] hashBytes = md5.ComputeHash(inputBytes);

                StringBuilder sb = new StringBuilder();
                for (int i = 0; i < hashBytes.Length; i++)
                {
                    sb.Append(hashBytes[i].ToString("X2"));
                }
                return sb.ToString();
            }
        }
    }
}

public class RequestContextHandler : IRequestContextHandler
{
	IResourceRequestHandler IRequestContextHandler.GetResourceRequestHandler(IBrowser browser, IFrame frame, IRequest request, bool isNavigation, bool isDownload, string requestInitiator, ref bool disableDefaultHandling)
	{
		return null;
	}

	bool IRequestContextHandler.OnBeforePluginLoad(string mimeType, string url, bool isMainFrame, string topOriginUrl, WebPluginInfo pluginInfo, ref PluginPolicy pluginPolicy)
	{
		return false;
	}

	void IRequestContextHandler.OnRequestContextInitialized(IRequestContext requestContext)
	{
		//You can set preferences here on your newly initialized request context.
		//Note, there is called on the CEF UI Thread, so you can directly call SetPreference

		//You can set the proxy with code similar to the code below
		var v = new Dictionary<string, object>
		{
		    ["mode"] = "fixed_servers",
		    ["server"] = "socks5://foobar:66"
		};
		string errorMessage;
		bool success = requestContext.SetPreference("proxy", v, out errorMessage);
	}
}

[amaitland]

I've already added a number of extension methods to IRequestContext, to simplify setting a proxy given a specific RequestContext. This will need to be called on the CEF UI Thread`.

public static bool SetProxy(this IRequestContext requestContext, string scheme, string host, int? port, out string errorMessage)
public static bool SetProxy(this IRequestContext requestContext, string host, int? port, out string errorMessage)
public static bool SetProxy(this IRequestContext requestContext, string host, out string errorMessage)

I've created #3250 (still a work in progress) that contains some further enhancements

//SetPreferenceAsync can be called on any Thread and will ensure the SetPreference call happens on the CEF UI Thread.
public static Task<SetPreferenceResponse> SetPreferenceAsync(this IRequestContext requestContext, string name, object value)

I've added some additional helper classes around setting settings upon initialization of a new RequestContext. In this scenario
an example of setting the proxy would looks like below (using CefSharp.OffScreen unit test as base).

[Fact]
public async Task CanLoadGoogleUsingProxy()
{
	var requestContextHander = new RequestContextHandler()
		.SetProxyOnContextInitialized("localhost", 8080);

	var requestContext = new RequestContext(requestContextHander);
	using (var browser = new ChromiumWebBrowser("www.google.com", requestContext: requestContext))
	{
		await browser.LoadPageAsync();

		var mainFrame = browser.GetMainFrame();
		Assert.True(mainFrame.IsValid);
		Assert.Contains("www.google", mainFrame.Url);

		output.WriteLine("Url {0}", mainFrame.Url);
	}
}

Building upon this further and I think we can still go one step further to simplify things. Still not sure about the best implementation.

Something static would look like (not yet implemented)

//Share settings with Global RequestContext and use custom proxy
var requestContext = RequestContext.CreateNewWithProxy(Cef.GetGlobalRequestContext(), "http", "localhost", 8080);
//Custom CachePath with proxy settings
var requestContext = RequestContext.CreateNewWithProxy("cachePath", "http", "localhost", 8080)

Something fluid might look like (not yet implemented)

var requestContext = RequestContext.From(Cef.GetGlobalRequestContext())
					.UseProxy("http", "localhost", 8080)
					.SetPreference("webkit.webprefs.minimum_font_size", 24);
					
var requestContext = RequestContext.From(Cef.GetGlobalRequestContext())
					.SetPreference("proxy", Proxy.Create("http", "localhost", 8080))
					.SetPreference("webkit.webprefs.minimum_font_size", 24);
					
var requestContext = RequestContext.SetCachePath("c:\\temp\\cache")
					.UseProxy("http", "localhost", 8080)
					.SetPreference("webkit.webprefs.minimum_font_size", 24);

As yet I'm not quite sure how to implement the fluid approach. I'll have a think about it some more.

In the meantime, thoughts and feedback are welcome.

[amaitland]

I've implemented a fluent style API which looks like the following. It match

var requestContext = RequestContext
	.Configure()
	.WithProxyServer("127.0.0.1", 8080)
	.Create();

using (var browser = new ChromiumWebBrowser("http://cefsharp.github.io/", requestContext: requestContext))
{
	await browser.LoadPageAsync();

	var mainFrame = browser.GetMainFrame();
	Assert.True(mainFrame.IsValid);
	Assert.Contains("cefsharp.github.io", mainFrame.Url);

	output.WriteLine("Url {0}", mainFrame.Url);
}

I plan on merging #3250 in the next few days. Once that's done I'll package the next -pre release.

Thoughts and comments welcome.

[Velka-DEV]

Is a really good implementation, thanks for your work @amaitland. Just some questions, i see you have tested the feature with a local proxy so is 1000% faster than a real proxy server and the project is mostly async. So have you tested if page loading is correctly done ? And also how the proxy load process is handled ? Will wait for request done before going to next instruction or not ?

[amaitland]

local proxy so is 1000% faster than a real proxy server and the project is mostly async

The proxy is set when the RequestContext is initialized, all requests associated with the RequestContext will use the proxy. The proxy is set before any request is made, so the speed is irrelevant. Async makes no differences in this context.

So have you tested if page loading is correctly done ?

Yes. Changing the port and the test will fail as expected. You could add say var src = await browser.GetSourceAsync(); to see the web page source has been downloaded.

And also how the proxy load process is handled ? Will wait for request done before going to next instruction or not ?

Fundamentally we are just configuring Chromium to use the proxy, it will function exactly how Chromium does.

Creating a RequestContext with .WithProxyServer("127.0.0.1", 8080) is fundamentally the same as running Chrome like:

chrome.exe --proxy-server="127.0.0.1:8080"

[amaitland]

The 85.3.121-pre packages are now on Nuget.org, you can test this for yourself. The MinimalExample has also been updated for testing purposes see https://github.com/cefsharp/CefSharp.MinimalExample/tree/cefsharp/85

[Velka-DEV]

Will test it start of the next week. Thanks for your work

[amaitland]

Version 85.3.130 is on Nuget.org. Closing as this feature is now available.

Let me know if you see any problems.