Support enterprise environments with proxy servers

[davikor]

Hi, I am working at a company where all internet traffic is tunneled through a proxy server.

DuckDB supports this generally by setting a variable like SET http_proxy = 'proxy.mycompany.com:3128';. After this setting I can e.g. successfuly install the UI extension (without the http_proxy setting download will fail).

However, when I open the webinterface, it will fail with Could not fetch: '/' from 'https://ui.duckdb.org': Could not establish connection. This is because the internal http-server that serves to the webbrowser does not use this proxy.

I peaked at the source code and it seems that this could be solved by adding to

duckdb-ui/src/http_server.cpp

Line 247 in 763e521

httplib::Client client(remote_url);

something like client.set_proxy("proxy.mycompany.com", 3128); (not sure whether this is the only place that needs to be adapted).

If you are interested in having this fixed I will gladly provide a pr, unless you want to fix this by yourself.

Cheers
David

[Y--]

@davikor thanks for the report - I've started #152 to work on this.

I didn't have time to test it properly yet however. If you could build and test on your end it would be greatly appreciated :-)

Thanks!

[jraymakers]

Note that this is a duplicate of #23. However, I'll keep this open while it's being actively worked on.

[davikor]

@davikor thanks for the report - I've started #152 to work on this.

I didn't have time to test it properly yet however. If you could build and test on your end it would be greatly appreciated :-)

Thanks!

Thanks a lot! I just built your PR and did a quick test behind our company proxy and duckdb-ui seems to download the web-app now 🎉. The app however, still fails (see attached screenshot. Maybe you have an idea what this is, and if this is related to the proxy setting, or something else? Otherwise I can play around with it in more detail tomorrow.

[davikor]

Note that this is a duplicate of #23. However, I'll keep this open while it's being actively worked on.

sorry, I even read this issue and then read #62 and somehow considered them the same in my head and then kept on opening a new issue because my issue (proxy support) is clearly not the same as offline support 🫣

[Y--]

@davikor thanks for the report - I've started #152 to work on this.
I didn't have time to test it properly yet however. If you could build and test on your end it would be greatly appreciated :-)
Thanks!

Thanks a lot! I just built your PR and did a quick test behind our company proxy and duckdb-ui seems to download the web-app now 🎉. The app however, still fails (see attached screenshot. Maybe you have an idea what this is, and if this is related to the proxy setting, or something else? Otherwise I can play around with it in more detail tomorrow.

Ah yes, I'll check with @jraymakers to see how we can fix this one. Thanks a lot for confirming that it works!

[davikor]

it seems the error occurs, because I opened the web-ui in my browser via the http://127.0.0.1:<port> URL. When instead I open http://localhost:<port> the UI runs flawless. This is because the handlers for several web requests (e.g. here

duckdb-ui/src/http_server.cpp

Line 289 in 763e521

if (origin != local_url) {

) do check if the origin header value sent by the webbrowser is equal to the local_url variable which is hardcoded aforementioned localhost-URL (except for the port-part of course).

So regarding the proxy problem, the issue seems to be fixed with your PR and this issue can be closed from my side. The localhost problem is already discussed in #22 so no new issue is required

[Y--]

Ah that make sense! Thanks a lot for confirming :-)
We'll ship that PR then.

[jraymakers]

There's now a version on core_nightly with this fix.

If you're experiencing this issue, please try force install ui from core_nightly, set your http_proxy, and see if that works in your environment.

[jraymakers]

This fix is now in core.