HTTP_PROXY support via global-agent is broken

[callumgare]

The docs indicate that SST can be configured to use a HTTP proxy (https://docs.sst.dev/advanced/connecting-via-proxy) however I believe this is no longer works.

I'm running SST in an AWS EC2 instance that does not have direct access to the internet. All external traffic must pass though a HTTP proxy. I have verified this behaviour works as expected since without supplying the proxy details via the env vars http_proxy and https_proxy then running curl http://example.com and curl https://example.com will result in a timeout (curl: (28) Failed to connect to example.com port 80 after 129952 ms: Connection timed out). However when these vars are specified the curl command runs fine.

The SST documentation states that it can be setup to use a http proxy by first installing the global-agent package (which I have done) and setting one of a list of env vars for http traffic and https traffic. I've tried numbers of combinations but am currently using GLOBAL_AGENT_HTTPS_PROXY and GLOBAL_AGENT_HTTP_PROXY set to the same values as the http_proxy and https_proxy env vars I was using to configure curl.

As far as I can tell this should be sufficient however that doesn't seem to work:

[Container] 2023/12/14 05:23:44.037154 Running command npx sst deploy --stage prod
Attention: SST now collects completely anonymous telemetry regarding usage. This is used to guide SST's roadmap.
You can learn more, including how to opt-out of this anonymous program, by heading over to:
https://docs.sst.dev/anonymous-telemetry

Error: connect ETIMEDOUT 54.83.2.133:443

Trace: Error: connect ETIMEDOUT 54.83.2.133:443
    at __node_internal_captureLargerStackTrace (node:internal/errors:496:5)
    at __node_internal_exceptionWithHostPort (node:internal/errors:671:12)
    at TCPConnectWrap.afterConnect [as oncomplete] (node:net:1555:16)
    at process.<anonymous> (file:///codebuild/output/src2748786936/src/github.com/infoxchange/sst-poc/node_modules/sst/cli/sst.js:58:17)
    at process.emit (node:events:529:35)
    at process.emit (node:domain:489:12)
    at process._fatalException (node:internal/process/execution:158:25)
Need help with this error? Post it in #help on the SST Discord https://sst.dev/discord

Thanks for reading!

Digging a little deeper I found the PR that added proxy support in the first case. It appears the actual code changes which called global-agent if the package was installed was lost in the transition between v1 and v2 (https://github.com/sst/sst/tree/v1.18.4/packages/core/src vs https://github.com/sst/sst/tree/v2.0.0/packages/sst/src).

Workaround

For anyone else running into this issue I found you can call global-agent youself by running sst with node -r 'global-agent/bootstrap' ./node_modules/.bin/sst. So instead of the command sst deploy --stage prod you'd use node -r 'global-agent/bootstrap' ./node_modules/.bin/sst deploy --stage prod.

[callumgare]

Also the documentation includes a small typo. The list of env var names that can be used for setting a https proxy includes the option HTTP_PROXY but this should be HTTPS_PROXY.

[jayair]

Oh interesting. I haven't dug into this but would it be possible to submit a PR for this?

[callumgare]

Sorry, wish I could but work is pretty hectic at the mo so I don't really have time given the workaround is sufficient in our use case. But thought I should report this in case anyone else is running into it.

[jayair]

Yeah appreciate it!

[dan-turner]

Thank you for raising this one @callumgare! Definitely saved me loads of time when I hit the same issue. Your workaround doesn't seem to work anymore as ./node_modules/.bin/sst is now a bash wrapper script.

My workaround was to add the following lines at the top of my sst.config.ts:

import { bootstrap } from "global-agent";
bootstrap();

I also had to explicitly use GLOBAL_AGENT_HTTP_PROXY and GLOBAL_AGENT_HTTPS_PROXY. HTTP_PROXY and HTTPS_PROXY (as used by most of other tools) did not work.