Failure to use NO_PROXY appropriately

[bdellegrazie]

Current behavior

In our setup, both environment variables: https_proxy, NO_PROXY and the .npmrc configuration of https_proxy / noproxy are configured.

The particular Cypress download URL is being redirected Nexus via CYPRESS_DOWNLOAD_MIRROR and should not go via the proxy.
All of this would work perfectly except:
During postinstall the cypress:lib/tasks/download.js task has the following function:

const getProxyForUrlWithNpmConfig = url => {
  return getProxyForUrl(url) || process.env.npm_config_https_proxy || process.env.npm_config_proxy || null;
};

Now getProxyForUrl(url) returns an empty string if it matches NO_PROXY or if there are no variables defined - these are not the same states and should be treated separately.
The || treats empty string as falsey and therefore picks up the process.env.npm_config_https_proxy but doesn't handle the corresponding process.env.npm_config_noproxy setting so it incorrectly selects the proxy.

Could you please adjust the function getProxyForUrlWithNpmConfig to correctly handle process.env.npm_config_noproxy and/or treat "nothing defined" differently from proxy explicitly disabled for this domain?

Debug logs

URLs changed!

Environment variables:

    http_proxy = 'http://outbound-proxy.example:3128/'
    https_proxy = 'http://outbound-proxy.example:3128/'
    no_proxy = 'nexus.example,*.example'
    NO_PROXY = 'nexus.example,.example,*.example'
    CYPRESS_DOWNLOAD_MIRROR = 'https://nexus.example/repository/cypress-dist/'

npmrc settings:

proxy=http://outbound-proxy.example:3128/
https_proxy=http://outbound-proxy.example:3128/
noproxy=*.example

Debug logs:

.../cypress@8.7.0/node_modules/cypress postinstall: [STARTED] Task without title.
.../cypress@8.7.0/node_modules/cypress postinstall: 2022-01-06T09:52:36.892Z cypress:cli Using CYPRESS_DOWNLOAD_MIRROR from environment variable
.../cypress@8.7.0/node_modules/cypress postinstall: 2022-01-06T09:52:36.892Z cypress:cli Using CYPRESS_DOWNLOAD_MIRROR from environment variable
.../cypress@8.7.0/node_modules/cypress postinstall: 2022-01-06T09:52:36.896Z cypress:cli needed Cypress version: 8.7.0
.../cypress@8.7.0/node_modules/cypress postinstall: 2022-01-06T09:52:36.896Z cypress:cli source url https://nexus.example/repository/cypress-dist/desktop/8.7.0?platform=linux&arch=x64
.../cypress@8.7.0/node_modules/cypress postinstall: 2022-01-06T09:52:36.896Z cypress:cli downloading cypress.zip to "/tmp/cypress-18030.zip"
.../cypress@8.7.0/node_modules/cypress postinstall: 2022-01-06T09:52:36.901Z cypress:cli Downloading package {
.../cypress@8.7.0/node_modules/cypress postinstall:   url: 'https://nexus.example/repository/cypress-dist/desktop/8.7.0?platform=linux&arch=x64',
.../cypress@8.7.0/node_modules/cypress postinstall:   proxy: 'http://outbound-proxy.example:3128/',
.../cypress@8.7.0/node_modules/cypress postinstall:   downloadDestination: '/tmp/cypress-18030.zip'
.../cypress@8.7.0/node_modules/cypress postinstall: }
.../cypress@8.7.0/node_modules/cypress postinstall: 2022-01-06T09:52:36.945Z cypress:cli Using CYPRESS_DOWNLOAD_MIRROR from environment variable
.../cypress@8.7.0/node_modules/cypress postinstall: 2022-01-06T09:52:36.945Z cypress:cli Using CYPRESS_DOWNLOAD_MIRROR from environment variable
.../cypress@8.7.0/node_modules/cypress postinstall: 2022-01-06T09:52:36.946Z cypress:cli URL: https://nexus.example/repository/cypress-dist/desktop/8.7.0?platform=linux&arch=x64
.../cypress@8.7.0/node_modules/cypress postinstall: Error: tunneling socket could not be established, statusCode=403

Cypress Version

8.7.0

Package Manager

pnpm

Operating system

Linux

Other

No response

[mscharp]

I am experiencing the same problem, though I think it's manifested differently... more like: #4303.. If this doesn't belong here, I'll be happy to create a new issue.

Cypress Version
12.5.1

Package Manager
npm / yarn

Operating system
MacOs - Monterey

I also think the way the no_proxy is processed (specifically checking for "<-loopback>) makes no sense. It's specifically checking for that loopback string and when it doesn't find it adds "localhost", "127.0.0.1", and "::1" to the no_proxy variable. You'll see in my logs that I already have localhost and 127.0.01 in there. This doesn't really have any affect... just odd behavior.

cypress:server:cypress starting cypress with argv [ '/Users/itsme/Library/Caches/Cypress/12.5.1/Cypress.app/Contents/MacOS/Cypress', '--no-sandbox', '--', '--run-project', 'apps/pim-app-e2e', '--config', '{"baseUrl":"http://localhost:5000"}', '--config-file', 'cypress.config.ts', '--env', '{"tsConfig":"/Users/itsme/dev/aps/uinta/dap/ui/apps/pim-app-e2e/tsconfig.json"}', '--output-path', '/var/folders/qk/91qw18xs50vcts3rptssvz90y946zh/T/tmp-40960-mPDVen31wZIQ', '--record', 'false', '--testing-type', 'e2e', '--cwd', '/Users/itsme/dev/aps/uinta/dap/ui', '--userNodePath', '/Users/itsme/.nvm/versions/node/v18.13.0/bin/node', '--userNodeVersion', '18.13.0' ] +0ms
  cypress:server:args argv array: [ '/Users/itsme/Library/Caches/Cypress/12.5.1/Cypress.app/Contents/MacOS/Cypress', '--no-sandbox', '--run-project', 'apps/pim-app-e2e', '--config', '{"baseUrl":"http://localhost:5000"}', '--config-file', 'cypress.config.ts', '--env', '{"tsConfig":"/Users/itsme/dev/aps/uinta/dap/ui/apps/pim-app-e2e/tsconfig.json"}', '--output-path', '/var/folders/qk/91qw18xs50vcts3rptssvz90y946zh/T/tmp-40960-mPDVen31wZIQ', '--record', 'false', '--testing-type', 'e2e', '--cwd', '/Users/itsme/dev/aps/uinta/dap/ui', '--userNodePath', '/Users/itsme/.nvm/versions/node/v18.13.0/bin/node', '--userNodeVersion', '18.13.0' ] +0ms
  cypress:server:args parsed argv options { options: { _: [ '/Users/itsme/Library/Caches/Cypress/12.5.1/Cypress.app/Contents/MacOS/Cypress' ], sandbox: false, 'run-project': 'apps/pim-app-e2e', runProject: 'apps/pim-app-e2e', config: '{"baseUrl":"http://localhost:5000"}', 'config-file': 'cypress.config.ts', configFile: 'cypress.config.ts', env: '{"tsConfig":"/Users/itsme/dev/aps/uinta/dap/ui/apps/pim-app-e2e/tsconfig.json"}', 'output-path': '/var/folders/qk/91qw18xs50vcts3rptssvz90y946zh/T/tmp-40960-mPDVen31wZIQ', outputPath: '/var/folders/qk/91qw18xs50vcts3rptssvz90y946zh/T/tmp-40960-mPDVen31wZIQ', record: 'false', 'testing-type': 'e2e', testingType: 'e2e', cwd: '/Users/itsme/dev/aps/uinta/dap/ui', userNodePath: '/Users/itsme/.nvm/versions/node/v18.13.0/bin/node', userNodeVersion: '18.13.0' } } +2ms
  cypress:server:args argv parsed: { _: [ '/Users/itsme/Library/Caches/Cypress/12.5.1/Cypress.app/Contents/MacOS/Cypress' ], sandbox: false, runProject: 'apps/pim-app-e2e', config: { baseUrl: 'http://localhost:5000' }, configFile: 'cypress.config.ts', env: { tsConfig: '/Users/itsme/dev/aps/uinta/dap/ui/apps/pim-app-e2e/tsconfig.json' }, outputPath: '/var/folders/qk/91qw18xs50vcts3rptssvz90y946zh/T/tmp-40960-mPDVen31wZIQ', record: false, testingType: 'e2e', cwd: '/Users/itsme/dev/aps/uinta/dap/ui', userNodePath: '/Users/itsme/.nvm/versions/node/v18.13.0/bin/node', userNodeVersion: '18.13.0', invokedFromCli: true } +5ms
  cypress:server:util:proxy found proxy environment variables { NO_PROXY: 'localhost,127.0.0.1', HTTP_PROXY: 'http://somerandomproxy.com:8000', HTTPS_PROXY: 'http://somerandomproxy.com:8000', no_proxy: 'localhost,127.0.0.1', http_proxy: 'http://somerandomproxy.com:8000', https_proxy: 'http://somerandomproxy.com:8000' } +0ms
  cypress:server:util:proxy overriding uppercase env var with lowercase { name: 'NO_PROXY' } +0ms
  cypress:server:util:proxy overriding uppercase env var with lowercase { name: 'HTTP_PROXY' } +0ms
  cypress:server:util:proxy overriding uppercase env var with lowercase { name: 'HTTPS_PROXY' } +0ms
  cypress:server:util:proxy <-loopback> not found, adding localhost to NO_PROXY +1ms
  cypress:server:util:proxy normalized proxy environment variables { NO_PROXY: 'localhost,127.0.0.1,127.0.0.1,::1,localhost', HTTP_PROXY: 'http://somerandomproxy.com:8000', HTTPS_PROXY: 'http://somerandomproxy.com:8000' } +0ms
  cypress:server:cypress starting in mode run with options { _: [ '/Users/itsme/Library/Caches/Cypress/12.5.1/Cypress.app/Contents/MacOS/Cypress' ], sandbox: false, runProject: 'apps/pim-app-e2e', config: { e2e: { baseUrl: 'http://localhost:5000' }, env: { tsConfig: '/Users/itsme/dev/aps/uinta/dap/ui/apps/pim-app-e2e/tsconfig.json' } }, configFile: 'cypress.config.ts', outputPath: '/var/folders/qk/91qw18xs50vcts3rptssvz90y946zh/T/tmp-40960-mPDVen31wZIQ', record: false, testingType: 'e2e', cwd: '/Users/itsme/dev/aps/uinta/dap/ui', userNodePath: '/Users/itsme/.nvm/versions/node/v18.13.0/bin/node', userNodeVersion: '18.13.0', invokedFromCli: true, proxyServer: 'http://somerandomproxy.com:8000', proxyBypassList: 'localhost,127.0.0.1,127.0.0.1,::1,localhost', projectRoot: '/Users/itsme/dev/aps/uinta/dap/ui/apps/pim-app-e2e' } +146ms
    cypress:server:server-base server open +0ms
  cypress:server:server-e2e createServer connecting to server +0ms
  cypress:server:server-base Server listening on  { address: '127.0.0.1', family: 'IPv4', port: 58985 } +25ms
  cypress:server:appdata path: /Users/itsme/Library/Application Support/Cypress/cy/production/proxy +218ms
  cypress:server:ensure-url checking that baseUrl is available {
  baseUrl: 'http://localhost:5000',
  delaysRemaining: [ 3000, 3000, 4000 ],
  retryIntervals: [ 3000, 3000, 4000 ]
} +0ms
Cypress could not verify that this server is running:

  > http://localhost:5000

We are verifying this server because it has been configured as your baseUrl.

Cypress automatically waits until your server is accessible before running tests.

We will try connecting to it 3 more times...
  cypress:server:ensure-url checking that baseUrl is available {
  baseUrl: 'http://localhost:5000',
  delaysRemaining: [ 3000, 4000 ],
  retryIntervals: [ 3000, 3000, 4000 ]
} +1m
We will try connecting to it 2 more times...
  cypress:server:ensure-url checking that baseUrl is available {
  baseUrl: 'http://localhost:5000',
  delaysRemaining: [ 4000 ],
  retryIntervals: [ 3000, 3000, 4000 ]
} +8s
We will try connecting to it 1 more time...

  cypress:server:ensure-url checking that baseUrl is available {
  baseUrl: 'http://localhost:5000',
  delaysRemaining: [],
  retryIntervals: [ 3000, 3000, 4000 ]
} +10s

As you can see, no_proxy is set correctly, but cypress can't access the server. Below is output from when I unset the proxy information and re-run it, which then successfully connects to the app.

cypress:server:cypress starting cypress with argv [ '/Users/itsme/Library/Caches/Cypress/12.5.1/Cypress.app/Contents/MacOS/Cypress', '--no-sandbox', '--', '--run-project', 'apps/pim-app-e2e', '--config', '{"baseUrl":"http://localhost:5000"}', '--config-file', 'cypress.config.ts', '--env', '{"tsConfig":"/Users/itsme/dev/aps/uinta/dap/ui/apps/pim-app-e2e/tsconfig.json"}', '--output-path', '/var/folders/qk/91qw18xs50vcts3rptssvz90y946zh/T/tmp-42546-ydNSlesOZKej', '--record', 'false', '--testing-type', 'e2e', '--cwd', '/Users/itsme/dev/aps/uinta/dap/ui', '--userNodePath', '/Users/itsme/.nvm/versions/node/v18.13.0/bin/node', '--userNodeVersion', '18.13.0' ] +0ms
  cypress:server:args argv array: [ '/Users/itsme/Library/Caches/Cypress/12.5.1/Cypress.app/Contents/MacOS/Cypress', '--no-sandbox', '--run-project', 'apps/pim-app-e2e', '--config', '{"baseUrl":"http://localhost:5000"}', '--config-file', 'cypress.config.ts', '--env', '{"tsConfig":"/Users/itsme/dev/aps/uinta/dap/ui/apps/pim-app-e2e/tsconfig.json"}', '--output-path', '/var/folders/qk/91qw18xs50vcts3rptssvz90y946zh/T/tmp-42546-ydNSlesOZKej', '--record', 'false', '--testing-type', 'e2e', '--cwd', '/Users/itsme/dev/aps/uinta/dap/ui', '--userNodePath', '/Users/itsme/.nvm/versions/node/v18.13.0/bin/node', '--userNodeVersion', '18.13.0' ] +0ms
  cypress:server:args parsed argv options { options: { _: [ '/Users/itsme/Library/Caches/Cypress/12.5.1/Cypress.app/Contents/MacOS/Cypress' ], sandbox: false, 'run-project': 'apps/pim-app-e2e', runProject: 'apps/pim-app-e2e', config: '{"baseUrl":"http://localhost:5000"}', 'config-file': 'cypress.config.ts', configFile: 'cypress.config.ts', env: '{"tsConfig":"/Users/itsme/dev/aps/uinta/dap/ui/apps/pim-app-e2e/tsconfig.json"}', 'output-path': '/var/folders/qk/91qw18xs50vcts3rptssvz90y946zh/T/tmp-42546-ydNSlesOZKej', outputPath: '/var/folders/qk/91qw18xs50vcts3rptssvz90y946zh/T/tmp-42546-ydNSlesOZKej', record: 'false', 'testing-type': 'e2e', testingType: 'e2e', cwd: '/Users/itsme/dev/aps/uinta/dap/ui', userNodePath: '/Users/itsme/.nvm/versions/node/v18.13.0/bin/node', userNodeVersion: '18.13.0' } } +2ms
  cypress:server:args argv parsed: { _: [ '/Users/itsme/Library/Caches/Cypress/12.5.1/Cypress.app/Contents/MacOS/Cypress' ], sandbox: false, runProject: 'apps/pim-app-e2e', config: { baseUrl: 'http://localhost:5000' }, configFile: 'cypress.config.ts', env: { tsConfig: '/Users/itsme/dev/aps/uinta/dap/ui/apps/pim-app-e2e/tsconfig.json' }, outputPath: '/var/folders/qk/91qw18xs50vcts3rptssvz90y946zh/T/tmp-42546-ydNSlesOZKej', record: false, testingType: 'e2e', cwd: '/Users/itsme/dev/aps/uinta/dap/ui', userNodePath: '/Users/itsme/.nvm/versions/node/v18.13.0/bin/node', userNodeVersion: '18.13.0', invokedFromCli: true } +6ms
  cypress:server:util:proxy found proxy environment variables { NO_PROXY: 'localhost,127.0.0.1', no_proxy: 'localhost,127.0.0.1' } +0ms
  cypress:server:util:proxy overriding uppercase env var with lowercase { name: 'NO_PROXY' } +0ms
  cypress:server:cypress starting in mode run with options { _: [ '/Users/itsme/Library/Caches/Cypress/12.5.1/Cypress.app/Contents/MacOS/Cypress' ], sandbox: false, runProject: 'apps/pim-app-e2e', config: { e2e: { baseUrl: 'http://localhost:5000' }, env: { tsConfig: '/Users/itsme/dev/aps/uinta/dap/ui/apps/pim-app-e2e/tsconfig.json' } }, configFile: 'cypress.config.ts', outputPath: '/var/folders/qk/91qw18xs50vcts3rptssvz90y946zh/T/tmp-42546-ydNSlesOZKej', record: false, testingType: 'e2e', cwd: '/Users/itsme/dev/aps/uinta/dap/ui', userNodePath: '/Users/itsme/.nvm/versions/node/v18.13.0/bin/node', userNodeVersion: '18.13.0', invokedFromCli: true, projectRoot: '/Users/itsme/dev/aps/uinta/dap/ui/apps/pim-app-e2e' } +177ms
  cypress:server:server-base server open +0ms
  cypress:server:server-e2e createServer connecting to server +0ms
  cypress:server:server-base Server listening on  { address: '127.0.0.1', family: 'IPv4', port: 59724 } +37ms
  cypress:server:appdata path: /Users/itsme/Library/Application Support/Cypress/cy/production/proxy +371ms
  cypress:server:ensure-url checking that baseUrl is available {
  baseUrl: 'http://localhost:5000',
  delaysRemaining: [ 3000, 3000, 4000 ],
  retryIntervals: [ 3000, 3000, 4000 ]
} +0ms
cypress:server:open_project open project url http://localhost:5000/__/#/specs/runner?file=src/integration/variables.e2e.ts +2ms

Note: I've only added what I thought to be the relevant logs for both the failing and succeeding case.

[cypress-app-bot]

This issue has not had any activity in 180 days. Cypress evolves quickly and the reported behavior should be tested on the latest version of Cypress to verify the behavior is still occurring. It will be closed in 14 days if no updates are provided.

[bdellegrazie]

The code referenced in the bug description still exists as previously described and the problem still exists. This is not a stale issue.

[cypress-app-bot]

This issue has not had any activity in 180 days. Cypress evolves quickly and the reported behavior should be tested on the latest version of Cypress to verify the behavior is still occurring. It will be closed in 14 days if no updates are provided.

[agreedSkiing]

Still an issue that needs to be addressed

[ricksbrown]

From what I can see this PR needs to be merged first: https://github.com/Rob--W/proxy-from-env/pull/19
Then, Cypress needs to pick up the new version of that dep and it will hopefully automagically work.

I question whether this codebase should be in the "proxy" business at all. It may find a more appropriate home in the dependency @cypress/request.

WORKAROUND

For anybody else with this problem, our solution was to update our build scripts to prime the cache dir (download and unzip) BEFORE we run Cypress. When Cypress runs it finds everything it needs in the cache and doesn't attempt the download.