Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improved proxy support #3531

Merged
merged 143 commits into from Apr 1, 2019

Conversation

9 participants
@flotwig
Copy link
Member

commented Feb 19, 2019

Fixes #1469
Fixes #1312

  • fix baseUrl check - fixes #1541
  • proxy cypress download - addresses issues mentioned in #787 comments, fixes #733
  • fix https requests to non-intercepted domains - fixes #672, fixes #939, fixes #1039, fixes #1459, fixes #1608, fixes #2295
    • do this better with https-proxy-agent
  • fix major source of proxy slowness - fixes #3192
  • test websockets
  • fix api.cypress.io not being reached - fixes #1470
  • fix not being able to login w/ github
  • fix Connection: close headers not using Connection: keep-alive on http requests - fixes #3857
  • load Windows registry settings to HTTP_PROXY env var - fixes #3855
  • try to find a way to override ProxyMode gonna skip this for now - https://www.chromium.org/administrators/policy-list-3#ProxyMode - #1253
    • Chrome extension - doesn't work, registry overrides it
    • NPAPI extension - jk, this has been completely removed from Chrome >42
    • PPAPI plugin - can only modify proxy for a Chrome App, and both chrome apps and PPAPI are being deprecated
    • modifying arbitrary profile settings
    • magic DLL injection
  • autoload OS X system proxy settings?
  • brainstorm better api for proxy stuff
  • add visual indicator in gui for proxy settings - fixes #3856
  • add tests for server cli args
  • add tests for cli
  • add "learn more" to proxy settings, tooltip on proxy bypass list
  • write desktop-gui tests
  • re-check proxy in server now that cli may have failed
  • get windows dependency hell sorted out
  • try to make it go fast
    • socket.setNoDelay where needed
    • understand when dns is cached
    • use chrome har capturer to capture hars for simple proxy, cypress proxy, fancy proxy, simple proxy + delay in handshake
    • do different OS's have different socket times? nodejs versions?
    • test in node if capping the # of sockets limits it, test node through a proxy too with capped sockets
    • change node socket chunk size?
  • don't proxy localhost
  • https://on.cypress.io/proxy-configuration link setup
  • write the docs for the on-link - cypress-io/cypress-documentation#1486
  • events (close, connect, drain, end, error, lookup, time timeout,
  • who controls the keepalive / timeout settings? do we keep a persistent socket connection between us and the browser? what about the origin? are connections ever disconnected?
    • TCP keepalive can be initiated by either party, it's just TCP ACKs used to keep a connection alive and make sure the other party is still responsive
    • HTTP keepalive??
  • do the keepalive probes keep the socket open even if no data is written within the timeout?

@flotwig flotwig marked this pull request as ready for review Feb 20, 2019

@flotwig flotwig force-pushed the issue-1469-proxies branch from fe12a1f to 065f51c Feb 20, 2019

@jennifer-shehane jennifer-shehane referenced this pull request Feb 20, 2019

Closed

Support proxies and improve proxy performance #1469

10 of 10 tasks complete
@flotwig

This comment has been minimized.

Copy link
Member Author

commented Feb 22, 2019

Added some visual indicators to desktop-gui. Kinda feel like these proxy settings should be part of an app-wide "preferences" window, like how it is in Postman. They're a little out of place in Project Settings.

Visuals (outdated)

On Linux with some proxy environment vars set:

image

image

On Windows with some proxy registry settings:

image

With no proxy settings:

image

@flotwig flotwig force-pushed the issue-1469-proxies branch from 5da02e9 to 4ffc140 Feb 26, 2019

flotwig added some commits Feb 26, 2019

@JesterXL

This comment has been minimized.

Copy link

commented Feb 27, 2019

These are awesome, @flotwig!

Liu233w added a commit to Liu233w/acm-statistics that referenced this pull request Mar 31, 2019

fix(e2e): 暂时在 ci 中将 cypress.json 的baseUrl换成 localhost
因为cypress对于 proxy 的支持还不大够,等到
cypress-io/cypress#3531 合并之后应该就可以了

Liu233w added a commit to Liu233w/acm-statistics that referenced this pull request Mar 31, 2019

test(e2e): 暂时移除全局的 cypress baseUrl
因为cypress对于 proxy 的支持还不大够,等到 cypress-io/cypress#3531 合并之后应该就可以了

@brian-mann brian-mann merged commit c1a345d into develop Apr 1, 2019

22 checks passed

ci/circleci: Linux lint Your tests passed on CircleCI!
Details
ci/circleci: Mac build Your tests passed on CircleCI!
Details
ci/circleci: Mac lint Your tests passed on CircleCI!
Details
ci/circleci: build Your tests passed on CircleCI!
Details
ci/circleci: desktop-gui-integration-tests-2x Your tests passed on CircleCI!
Details
ci/circleci: driver-integration-tests-3x Your tests passed on CircleCI!
Details
ci/circleci: lint-typescript Your tests passed on CircleCI!
Details
ci/circleci: run-launcher Your tests passed on CircleCI!
Details
ci/circleci: server-e2e-tests-1 Your tests passed on CircleCI!
Details
ci/circleci: server-e2e-tests-2 Your tests passed on CircleCI!
Details
ci/circleci: server-e2e-tests-3 Your tests passed on CircleCI!
Details
ci/circleci: server-e2e-tests-4 Your tests passed on CircleCI!
Details
ci/circleci: server-e2e-tests-5 Your tests passed on CircleCI!
Details
ci/circleci: server-e2e-tests-6 Your tests passed on CircleCI!
Details
ci/circleci: server-e2e-tests-7 Your tests passed on CircleCI!
Details
ci/circleci: server-e2e-tests-8 Your tests passed on CircleCI!
Details
ci/circleci: server-integration-tests Your tests passed on CircleCI!
Details
ci/circleci: server-performance-tests Your tests passed on CircleCI!
Details
ci/circleci: server-unit-tests Your tests passed on CircleCI!
Details
ci/circleci: unit-tests Your tests passed on CircleCI!
Details
continuous-integration/appveyor/pr AppVeyor build succeeded
Details
license/cla Contributor License Agreement is signed.
Details
@RicardoVaranda

This comment has been minimized.

Copy link

commented Apr 1, 2019

@brian-mann @flotwig Great work on this PR, I'm Just wondering if you had a timeline of when this will be released.

The changes made are critical to users like me sitting behind a corporate firewall. I'm currently facing a lot of issues running Cypress behind a corporate firewall so I'm hoping this will help address them.

Once again great work guys and thank you for this awesome tool 👍

@jennifer-shehane

This comment has been minimized.

Copy link
Member

commented Apr 1, 2019

@RicardoVaranda We're working on releasing this as soon as possible. We've already written up most of the changelog.

probot-auto-merge bot added a commit to Liu233w/acm-statistics that referenced this pull request Apr 1, 2019

test(e2e): 暂时移除全局的 cypress baseUrl
因为cypress对于 proxy 的支持还不大够,等到 cypress-io/cypress#3531 合并之后应该就可以了
@markobalogh

This comment has been minimized.

Copy link

commented Apr 1, 2019

echoing @RicardoVaranda great work on this @flotwig and I'm excited to be able to use Cypress at my day job finally :-)

@leepowellcouk

This comment has been minimized.

Copy link

commented Apr 1, 2019

I concur, I've been wanting to use Cypress in our company for a while, now it might finally happen. Great work.

@sesteva

This comment has been minimized.

Copy link

commented Apr 3, 2019

Yes!!! Please release! :)))

@jennifer-shehane

This comment has been minimized.

Copy link
Member

commented Apr 4, 2019

We're working on releasing. We noticed a regression and also some unexpected size increase before officially releasing. Just trying to prevent a release + insta-patch release. 😄

laurinenas added a commit to laurinenas/cypress that referenced this pull request Apr 28, 2019

Improved proxy support (cypress-io#3531)
* https-proxy: unused file

* server: wrap all https requests that use a proxy

* server: use request lib in ensureUrl if proxy is in use. this makes runs tab work behind a proxy

* electron: pass --proxy-server to app itself, so the embedded github login page works

* cli: first attempt at env vars from windows registry

* cli: api cleanup

* cli: lint

* cli: fix crash on no proxy, add tests

* add desktop-gui watch to terminals.json

* cli: pass along --proxy-source

* electron: pass --proxy-bypass-list too

* server: whitelist proxy* args

* cli: better wording

* desktop-gui: display proxy settings

* extension: force proxy [wip]

* extension: finally, i am victorious over coffeescript

* extension: add -loopback to bypasslist

* extension: revert changes

Revert "extension: force proxy [wip]"

This reverts commit 3ab6ba4.

* desktop-gui: skip proxysettings if there aren't any

* https-proxy, server: proxy directConnections using https-proxy-agent

* https-agent: pool httpsAgents

* https-proxy: work when they're not on a proxy

* https-proxy: ci - use agent 1.0

* https-proxy: tests

* desktop-gui: hide proxy settings when not using proxy

* https-proxy: pass req through to https-proxy-agent callback

* cli: use get-windows-proxy

* desktop-gui: always show proxy settings

* server: use get-windows-proxy

* electron, server: supply electron proxy config when window launched

* server: fix

* https-proxy: cleanup

* server: clean up ensureUrl

* https-proxy: cleanup

* cli: fix

* cli: fix destructuring

* server: enable ForeverAgent to pool HTTPS/HTTP connections

cypress-io#3192

* server: updating snapshot

* https-proxy: don't crash, do error if proxy unreachable

* https-proxy:

* get-windows-proxy@1.0.0

* https-proxy: use proxy-from-env to decide on a proxy for a url

* server: fallback to HTTP_PROXY globally if HTTPS_PROXY not set

* server: proxy args test

* cli: add proxy tests

* cli: add test that loadSystemProxySettings is called during download

* cli, server: account for the fact that CI has some proxy vars set

* https-proxy: ""

* cli, https-proxy, server: ""

* desktop-gui: update settings gui

* desktop-gui: cypress tests for proxy settings

* server: strict undefined check

* cli, server: move get-windows-proxy to scope, optionalDeps

* server, cli: use new and improved get-windows-proxy

* cli, server: 1.5.0

* server: re-check for proxy since cli may have failed to load the lib

* server, cli: 1.5.1

* server: NO_PROXY=localhost by default, clean up

* https-proxy: disable Nagle's on proxy sockets

\cypress-io#3192

* https-proxy: use setNoDelay on upstream, cache https agent

* https-proxy: test basic auth

* https-proxy: add todo: remove this

* server: add custom HTTP(s) Agent implementation w keepalive, tunneling

* server: typescript for agent

* add ts to zunder

* server: more ts

* ts: add missing Agent type declaration

* server: create CombinedAgent

* server: use agent in more places

* ts: more declarations

* server: make script work even if debug port not supplied

* server: begin some testing

* server, ts: agent, tests

* server: test

* server: agent works with websockets now

* server: update snapshot

* server: work out some more bugs with websockets

* server: more websockets

* server: add net_profiler

* https-proxy: fix dangling socket on direct connection

* server: fix potential 'headers have already been sent'

* https-proxy: nab another dangler

* server: update test to expect agent

* https-proxy: fix failing test

* desktop-gui: change on-link

* server: add tests for empty response case

* server: tests

* server: send keep-alive with requests

* server: make net profiler hook on socket.connect

* server: only hook profiler once

* server: update tests, add keep-alive test

* server: only regen headers if needed

* server: move http_overrides into CombinedAgent, make it proxy-proof

for cypress-io#112

* server: update snapshot

* server: undo

* server: avoid circular dependency

* https-proxy, server: use our Agent instead of https-proxy-agent

* server: add dependency back

* cli: actually use proxy for download

* server, launcher, ts: typescript

* Revert "server, launcher, ts: typescript"

This reverts commit d3f8b8b.

* Revert "Revert "server, launcher, ts: typescript""

This reverts commit 818dfdf.

* ts, server: respond to PR

* server, ts: types

* ts: really fix types

* https-proxy, server: export CA from https-proxy

* agent, server, https-proxy: move agent to own package

* agent => networking, move connect into networking

* fix tests

* fix test

* networking: respond to PR changes, add more unit tests

* rename ctx

* networking, ts: add more tests

* server: add ensureUrl tests

* https-proxy: remove https-proxy-agent

* server: use CombinedAgent for API

* server: updates

* add proxy performance tests

* add perf tests to workflow

* circle

* run perf tests with --no-sandbox

* networking, ts: ch-ch-ch-ch-changes

* server, networking: pr changes

* run networking tests in circle

* server: fix performance test

* https-proxy: test that sockets are being closed

* https-proxy: write, not emit

* networking: fix test

* networking: bubble err in connect

* networking: style

* networking: clean p connect error handling

* networking => network

* server: make perf tests really work

* server: really report

* server: use args from browser

* server: use AI to determine max run time

* server: load electron only when needed


Co-authored-by: Brian Mann <brian@cypress.io>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.