Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for HTTP proxy in network commands #10401

Merged
merged 1 commit into from Sep 27, 2023

Conversation

fnuttens
Copy link
Contributor

Closes #8847

Description

If the HTTP_PROXY variable is found, use its value to setup ureq proxy. I haven't implemented NO_PROXY at the moment.

User-Facing Changes

No breaking change for the user, the network commands simply use an environment variable.

Tests + Formatting

The existing tests seem to run fine, although I can't think of a new test to add.

@amtoine
Copy link
Member

amtoine commented Sep 19, 2023

@fnuttens
thanks for trying to make Nushell better 🙏

i'm a total noob on the networking matter, is it possible to test such a new feature? 😋

@fnuttens
Copy link
Contributor Author

@amtoine
Sure thing! You just have to find an HTTP proxy (I use Bright Data's) and put its host and port (+ your credentials if applicable) inside the HTTP_PROXY environment variable:

> $env.HTTP_PROXY
http://user:passwd@host:port

After that you can use any of the http subcommands to make a request through the proxy:

> http get https://lumtest.com/myip.json
╭─────────┬─────────────────────────────────╮
│ ip      │ x.x.x.x                         │
│ country │ FR                              │
│         │ ╭──────────┬────────╮           │
│ asn     │ │ asnum    │ 3215   │           │
│         │ │ org_name │ Orange │           │
│         │ ╰──────────┴────────╯           │
│         │ ╭─────────────┬───────────────╮ │
│ geo     │ │ city        │ Paris         │ │
│         │ │ region      │ IDF           │ │
│         │ │ region_name │ Île-de-France │ │
│         │ │ postal_code │ 75019         │ │
│         │ │ latitude    │ 48.83         │ │
│         │ │ longitude   │ 2.41          │ │
│         │ │ tz          │ Europe/Paris  │ │
│         │ │ lum_city    │ paris         │ │
│         │ │ lum_region  │ idf           │ │
│         │ ╰─────────────┴───────────────╯ │
╰─────────┴─────────────────────────────────╯

.tls_connector(std::sync::Arc::new(tls));

if let Some(http_proxy) = retrieve_http_proxy_from_env() {
if let Ok(proxy) = ureq::Proxy::new(http_proxy) {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not particularly convinced of the if let approach here, as the user won't be notified if ureq is unable to parse the proxy. This could give them the sense that they're making HTTP calls through a proxy when they're not.

On the other hand, this error seems quite unlikely to happen; I couldn't find a value that wasn't Ok for ureq to parse, even white space.

Let me know what you think: is it best to silently discard the proxy if unable to parse the HTTP_PROXY value, or should we expect a successful parsing?


fn retrieve_http_proxy_from_env() -> Option<String> {
std::env::vars()
.find(|(key, _)| key == "HTTP_PROXY")
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi, thanks for your pr, I think it's good to check ALL_PROXY too.
In the case if HTTP_PROXY exists, use it, or else check ALL_PROXY

Copy link
Contributor Author

@fnuttens fnuttens Sep 21, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about HTTPS_PROXY?

After doing some research on the matter, it seems like tools like curl support HTTP proxy in the lowercase variant (http_proxy). Should we look for both variables?

Regarding ALL_PROXY, don't we risk an error if using say a FTP proxy to make HTTP calls?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@WindSoilder I put HTTP_PROXY and ALL_PROXY as fallbacks to http_proxy.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for late reply.
I think your changes is fine. Regards to your question:

Regarding ALL_PROXY, don't we risk an error if using say a FTP proxy to make HTTP calls?

Yeah it is, so that's why we have http_proxy.
And I don't think we need to adapt HTTPS_PROXY for now

@WindSoilder WindSoilder merged commit 8f4ea69 into nushell:main Sep 27, 2023
19 checks passed
@fnuttens fnuttens deleted the http_proxy branch September 27, 2023 07:56
hardfau1t pushed a commit to hardfau1t/nushell that referenced this pull request Dec 14, 2023
Closes nushell#8847

# Description

If the `HTTP_PROXY` variable is found, use its value to setup ureq
proxy. I haven't implemented `NO_PROXY` at the moment.

# User-Facing Changes

No breaking change for the user, the network commands simply use an
environment variable.

# Tests + Formatting

The existing tests seem to run fine, although I can't think of a new
test to add.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

The http commands in the Network category don't seem to respect system HTTP/S proxy settings
4 participants