The http commands in the Network category don't seem to respect system HTTP/S proxy settings

[yonkeltron]

Describe the bug

Sometimes I use nushell in environments where web traffic must be routed through a specific proxy in order to reach the internet. This is usually fine since most tools support proxy-specific configuration either through system-level configs or through environment variables. As such, my personal config has variables set such as HTTP_PROXY, HTTPS_PROXY, and NO_PROXY along with their lower-case equivalents. Unfortunately, it seems that http commands in the Network category do not respect any of these variables nor could I uncover how to make them aware of a proxy by some other means.

I did want to point out that I did see a half-filled out issue (now closed) which mentioned this matter, which would be #8743.

How to reproduce

From behind a proxy, even with environment variables set, the following fails:

> http get "https://google.com"
Error: nu::shell::network_failure

  × Network failure
   ╭─[entry #39:1:1]
 1 │ http get "https://google.com"
   ·          ──────────┬─────────
   ·                    ╰── Cannot make request to "https://google.com". Error is"...snip...,: status code 401 (redirected from https://google.com/)"

This happens every time but does not happen from machines which are not behind such web proxies.

Expected behavior

Given correct configuration, I would have expected the http get invocation to have succeeded and produced the same output as it does when not behind a proxy.

Screenshots

No response

Configuration

> version | transpose key value | to md --pretty
| key                | value                                                             |
| ------------------ | ----------------------------------------------------------------- |
| version            | 0.78.0                                                            |
| branch             |                                                                   |
| commit_hash        |                                                                   |
| build_os           | macos-x86_64                                                      |
| build_target       | x86_64-apple-darwin                                               |
| rust_version       | rustc 1.68.2 (9eb3afe9e 2023-03-27) (built from a source tarball) |
| cargo_version      | cargo 1.68.1 (6feb7c9cf 2023-03-26)                               |
| build_time         | 2023-04-04 18:47:00 +00:00                                        |
| build_rust_channel | release                                                           |
| features           | default, zip                                                      |
| installed_plugins  |                                                                   |

Additional context

Happy to provide more info wherever I can, and/or work to help test things. I'm a huge fan of the project and grateful for everyone's efforts.

Thanks,
+Jonathan

[hinricht]

Same here, with nu 0.84.0. curl i.e. works beeing connected to the proxy, while http doesn't.

[hinricht]

Is there a workaround in which I can use i.e. curl to pipe fetched data into nushell structured format in the mean time ?

[sholderbach]

Is there a workaround in which I can use i.e. curl to pipe fetched data into nushell structured format in the mean time ?

If you are for example dealing with a JSON API endpoint you can run curl ... | from json without too much trouble.

If we can inherit system proxy settings without too much trouble, we should probably look into fixing this but we certainly should not grow nushells networking commands to the configuration flexibility of curl. They already figured out the messy rough edges etc.

[hinricht]

Yeah, I didn't want to propose to get feature complete with curl :)
But beeing able to use http / all networking cmds honoring proxy env vars would be great, otherwise they are completely useless to me in my work environment.

[PyAntony]

I agree. I was trying to make http work but I cant pass proxy options in any way. Hope this gets resolved. curl command works by the way.