Pre-install mypy deps in pre-commit.ci
#666
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Codecov Report
@@ Coverage Diff @@
## develop #666 +/- ##
========================================
Coverage 74.01% 74.01%
========================================
Files 84 84
Lines 3264 3264
========================================
Hits 2416 2416
Misses 848 848
Continue to review full report at Codecov.
|
|
#667 fixes the other failure that appears in |
abhinavsingh
added a commit
that referenced
this pull request
Nov 28, 2021
* Add proxy auth test coverage (#496) * Cover all proxy auth scenarios * Add test_proxy_auth_works_with_mixed_case_basic_string * Update tox from 3.21.3 to 3.21.4 (#497) * Update autopep8 from 1.5.4 to 1.5.5 (#499) * Update pylint from 2.6.0 to 2.6.2 (#501) * Fix mypy errors (#504) * Update tox from 3.21.4 to 3.22.0 (#502) Co-authored-by: Abhinav Singh <mailsforabhinav@gmail.com> * Update mypy from 0.790 to 0.812 (#503) Co-authored-by: Abhinav Singh <mailsforabhinav@gmail.com> * Update pylint from 2.6.2 to 2.7.1 (#506) * Update coverage from 5.4 to 5.5 (#508) * Update pylint from 2.7.1 to 2.7.2 (#509) * Update tox from 3.22.0 to 3.23.0 (#510) * Update twine from 3.3.0 to 3.4.1 (#517) * Update flake8 from 3.8.4 to 3.9.0 (#514) * Update autopep8 from 1.5.5 to 1.5.6 (#516) Co-authored-by: Abhinav Singh <mailsforabhinav@gmail.com> * Update py-spy from 0.3.3 to 0.3.5 (#522) * DeepSource: Code quality issues (#523) * Update pylint from 2.7.2 to 2.7.3 (#524) * Bump y18n from 3.2.1 to 3.2.2 in /dashboard (#526) Bumps [y18n](https://github.com/yargs/y18n) from 3.2.1 to 3.2.2. - [Release notes](https://github.com/yargs/y18n/releases) - [Changelog](https://github.com/yargs/y18n/blob/master/CHANGELOG.md) - [Commits](https://github.com/yargs/y18n/commits) Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Update pylint from 2.7.3 to 2.7.4 (#527) * Move wheel package to testing (#531) * Update pytest from 6.2.2 to 6.2.3 (#532) * Update flake8 from 3.9.0 to 3.9.1 (#538) * Update rope from 0.18.0 to 0.19.0 (#539) * Update pylint from 2.7.4 to 2.8.0 (#540) * Update pylint from 2.8.0 to 2.8.1 (#541) * Update pylint from 2.8.1 to 2.8.2 (#542) * Update autopep8 from 1.5.6 to 1.5.7 (#543) * Update typing-extensions from 3.7.4.3 to 3.10.0.0 (#544) * Update pytest from 6.2.3 to 6.2.4 (#545) * Update tox from 3.23.0 to 3.23.1 (#546) * Update py-spy from 0.3.5 to 0.3.6 (#547) * Update flake8 from 3.9.1 to 3.9.2 (#549) * Bump hosted-git-info from 2.8.5 to 2.8.9 in /dashboard (#548) Bumps [hosted-git-info](https://github.com/npm/hosted-git-info) from 2.8.5 to 2.8.9. - [Release notes](https://github.com/npm/hosted-git-info/releases) - [Changelog](https://github.com/npm/hosted-git-info/blob/v2.8.9/CHANGELOG.md) - [Commits](npm/hosted-git-info@v2.8.5...v2.8.9) Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Abhinav Singh <mailsforabhinav@gmail.com> * Bump lodash from 4.17.19 to 4.17.21 in /dashboard (#550) Bumps [lodash](https://github.com/lodash/lodash) from 4.17.19 to 4.17.21. - [Release notes](https://github.com/lodash/lodash/releases) - [Commits](lodash/lodash@4.17.19...4.17.21) Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Abhinav Singh <mailsforabhinav@gmail.com> * Update pytest-cov from 2.11.1 to 2.12.0 (#552) * Update py-spy from 0.3.6 to 0.3.7 (#555) * Bump ws from 7.4.0 to 7.4.6 in /dashboard (#556) Bumps [ws](https://github.com/websockets/ws) from 7.4.0 to 7.4.6. - [Release notes](https://github.com/websockets/ws/releases) - [Commits](websockets/ws@7.4.0...7.4.6) Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Update pylint from 2.8.2 to 2.8.3 (#558) * Update pytest-cov from 2.12.0 to 2.12.1 (#561) * Bump glob-parent from 5.1.1 to 5.1.2 in /dashboard (#564) Bumps [glob-parent](https://github.com/gulpjs/glob-parent) from 5.1.1 to 5.1.2. - [Release notes](https://github.com/gulpjs/glob-parent/releases) - [Changelog](https://github.com/gulpjs/glob-parent/blob/main/CHANGELOG.md) - [Commits](gulpjs/glob-parent@v5.1.1...v5.1.2) --- updated-dependencies: - dependency-name: glob-parent dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Update pylint from 2.8.3 to 2.9.3 (#573) * Update tox from 3.23.1 to 3.24.0 (#575) * Update twine from 3.4.1 to 3.4.2 (#576) * Update pylint from 2.9.3 to 2.9.5 (#577) Co-authored-by: Abhinav Singh <mailsforabhinav@gmail.com> * Update wheel from 0.36.2 to 0.37.0 (#585) * Update codecov from 2.1.11 to 2.1.12 (#582) Co-authored-by: Abhinav Singh <mailsforabhinav@gmail.com> * Update typing-extensions from 3.10.0.0 to 3.10.0.2 (#599) * Update pytest from 6.2.4 to 6.2.5 (#598) Co-authored-by: Abhinav Singh <mailsforabhinav@gmail.com> * Update tox from 3.24.0 to 3.24.3 (#592) Co-authored-by: Abhinav Singh <mailsforabhinav@gmail.com> * Update pylint from 2.9.5 to 2.10.2 (#591) Co-authored-by: Abhinav Singh <mailsforabhinav@gmail.com> * Bump path-parse from 1.0.6 to 1.0.7 in /dashboard (#586) Bumps [path-parse](https://github.com/jbgutierrez/path-parse) from 1.0.6 to 1.0.7. - [Release notes](https://github.com/jbgutierrez/path-parse/releases) - [Commits](https://github.com/jbgutierrez/path-parse/commits/v1.0.7) --- updated-dependencies: - dependency-name: path-parse dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Abhinav Singh <mailsforabhinav@gmail.com> * Update py-spy from 0.3.7 to 0.3.9 (#604) * Update rope from 0.19.0 to 0.20.1 (#611) * Update pylint from 2.10.2 to 2.11.1 (#609) Co-authored-by: Abhinav Singh <mailsforabhinav@gmail.com> * Update tox from 3.24.3 to 3.24.4 (#607) Co-authored-by: Abhinav Singh <mailsforabhinav@gmail.com> * add paramiko types for mypy compliance (#613) * Update py-spy from 0.3.9 to 0.3.10 (#616) * Update coverage from 5.5 to 6.0 (#618) * Update pytest-cov from 2.12.1 to 3.0.0 (#619) * Update types-paramiko from 2.7.0 to 2.7.1 (#620) * Update coverage from 6.0 to 6.0.1 (#623) * Update paramiko from 2.7.2 to 2.8.0 (#624) * Update coverage from 6.0.1 to 6.0.2 (#628) * Update types-paramiko from 2.7.1 to 2.7.2 (#629) * Update types-paramiko from 2.7.2 to 2.7.3 (#630) * Update rope from 0.20.1 to 0.21.0 (#631) * Update flake8 from 3.9.2 to 4.0.1 (#627) Co-authored-by: Abhinav Singh <mailsforabhinav@gmail.com> * Add support for 3.10 (#637) * Add support for 3.10 * Upgrade to v2 actions * v3.4.0 (#638) * Build docker from 3.10-alpine * Bump version to 3.4.0 * Add instructions for how to run dashboard * Order of menu * Override dashboard png path until submitted * Add some doc string for top-level Proxy class. Also some TODOs and warnings regarding PID file overwrite * Allow HttpProxyBasePlugin implementations to register custom descriptors for read/write events * Remove hardcoded adblock regex into json config. Update upstream filter to block facebook, not google * ProxyPoolPlugin and ReverseProxyPlugin must now be updated to use get/read/write descriptor APIs * Add get/read/write descriptor API for HttpWebServerBasePlugin too * Surface actual listening port via flags.port * Update autopep8 from 1.5.7 to 1.6.0 (#632) Co-authored-by: Abhinav Singh <mailsforabhinav@gmail.com> * Fix the `typing_extensions` runtime dependency version (#641) * Only use `typing_extensions` below Python 3.8 * Rely on unconstrained `typing-extensions` version * Update coverage from 6.0.2 to 6.1 (#640) Co-authored-by: Abhinav Singh <mailsforabhinav@gmail.com> * Fix path to dashboard.png (#643) * [ImgBot] Optimize images (#644) *Total -- 1,253.27kb -> 774.20kb (38.23%) /Dashboard.png -- 1,072.21kb -> 617.66kb (42.39%) /ProxyPy.png -- 56.83kb -> 38.15kb (32.86%) /shortlink.gif -- 124.23kb -> 118.39kb (4.7%) Signed-off-by: ImgBotApp <ImgBotHelp@gmail.com> Co-authored-by: ImgBotApp <ImgBotHelp@gmail.com> * Update coverage from 6.1 to 6.1.1 (#646) * Async proxy pool, Event manager, Custom access log, Expose loop to plugins (#645) * Async proxy pool * Async proxy pool * Late upstream initialization and exception guards * Close upstream proxy connection on client connection close * Refactor into EventManager * Fix tests accounting in the event manager * Ensure each process initializes logger * pragma no cover * Teardown connection when proxy pool upstream proxy closes * Add ability to customize access log format and add additional context to it * Maintain total size for response bytes in access logs * Fix tests broken due to new plugin methods missing mock * Update pubsub_eventing to use EventManager to avoid entire bootstrapping step * Add tox envs for building dists via PEP 517 (#647) * Invoke self-install via PEP517 in the CI * Add tox envs for building dists via PEP 517 * Add linting dists via tox to GHA * Upgrade `master` to `develop` in `setup.py` * Simplify `python_requires` in `setup.py` * Convert dynamic `setup.py` into static `setup.cfg` * Revert "Upgrade `master` to `develop` in `setup.py`" (#650) This reverts commit 06f6f56. * Add a no-op check job to GHA for branch protection (#652) This patch adds an "empty" job that depends on all the important ones making it possible to use just this `check` job in the branch protection settings. This reduces the maintenance burden by preventing the need to update these settings on any changes to the job declarations. * Add a config for YAMLLint (#653) Co-authored-by: Abhinav Singh <mailsforabhinav@gmail.com> * Add a config for flake8 (#654) Co-authored-by: Abhinav Singh <mailsforabhinav@gmail.com> * Correct spelling mistakes caught by `codespell` (#656) Co-authored-by: Abhinav Singh <mailsforabhinav@gmail.com> * Add a config for pylint (#655) * Add an initial auto-generated PyLint config * Align pylint line length settings in with flake8 * Colorize the pylint report output * Disable all currently violated PyLint rules Co-authored-by: Abhinav Singh <mailsforabhinav@gmail.com> * Organize the linting setup around tox+pre-commit (#657) * Add a no-op check job to GHA for branch protection This patch adds an "empty" job that depends on all the important ones making it possible to use just this `check` job in the branch protection settings. This reduces the maintenance burden by preventing the need to update these settings on any changes to the job declarations. * Add a config for YAMLLint * Add a config for flake8 * Add an initial auto-generated PyLint config * Align pylint line length settings in with flake8 * Colorize the pylint report output * Correct spelling mistakes caught by `codespell` * Disable all currently violated PyLint rules * Start managing the linters setup with pre-commit Co-authored-by: Abhinav Singh <mailsforabhinav@gmail.com> * Enable yamllint for all previously ignored files (#658) * Add autocancellation of the stale PR GHA jobs (#663) Co-authored-by: Abhinav Singh <mailsforabhinav@gmail.com> * Update twine from 3.4.2 to 3.5.0 (#665) Co-authored-by: Abhinav Singh <mailsforabhinav@gmail.com> * Enable the `add-trailing-comma` pre-commit fixer (#661) Co-authored-by: Abhinav Singh <mailsforabhinav@gmail.com> * Migrate the pytest invocation to `tox` (#662) * Fix the `commands` setting in the tox config * Wire up `pytest` invocations via tox * Pre-install mypy deps in `pre-commit.ci` (#666) Co-authored-by: Abhinav Singh <mailsforabhinav@gmail.com> * Fix the YTT201 flake8 violation (#667) This approach makes sure that if there's ever Python 4, it wouldn't fall back to the Python 2 behavior silently. Co-authored-by: Abhinav Singh <mailsforabhinav@gmail.com> * Make names in the GHA lib workflow short (#669) Co-authored-by: Abhinav Singh <mailsforabhinav@gmail.com> * Update outdated sections of the codebase (#670) * Remove autopep8, is redundant now after recent CI changes * Add pyenv .python-version to .gitignore * Update year * Add lib-pytest target so that pytest can run in isolation * Add git-push hook which will also run the lint. By default now git-pre-commit hook will only run pytest. * Update outdated sections of README * Update requirement to match setup.cfg install_requires * Deprecate proxy.start and TestCase.PROXY_PORT Proxy port during test is now available as self.PROXY.pool.flags.port. Also now TestCase utilize ephemeral port strategy instead of calling get_available_port utility method. * Rename to git-pre-push * Ideally public repo dont require CODECOV_TOKEN but codecov integration is broken since introduction of codecov-action@v2 (instead of codecov binary invocation) * Issue is possibly with codecov@v2 action, fallback to codecov. See https://github.com/abhinavsingh/proxy.py/runs/4110423084\?check_suite_focus\=true and codecov/uploader#223 * Revert back to v2 * Make pytest emit XML coverage (#673) This patch should make GHA report coverage to codecov properly. * CustomDnsResolver plugin, CloudflareDnsResolver plugin, Allow plugins to configure network interface (#671) * Add CustomDnsResolver plugin. Addresses #535 and #664 * Add cloudflare DNS resolver plugin * Lint fixes * Add tests for missing core modules (#674) * Rename is_py3 to is_py2 for more logical guard * Add stubs for missing tests, add few more tests for core modules * Lint fixes * Line too long fix * Remove unnecessary KeyboardInterrupt * Consistent workflow names * Update homebrew formulae. Doesnt seems to work now * test_enable_dashboard and test_enable_events * test_enable_dashboard and test_enable_events * Fix problem where empty plugin string was passed as plugin module * test_enable_devtools and remove redundant guards for None and "" which was there due to a bug * Use core loop for reverse proxy async IO operations (#675) * Make reverse proxy plugin use proxy.py core loop for async io operations * Address lint errors * Deprecate on_websocket_close and replace with on_client_connection_close * Lint fixes * Retry on SSLWantReadError and SSLWantWriteError * Collect coverage for `tests/` (#677) This is necessary to make sure all the tests actually get executed somewhere and allows finding dead code in this area. * Allow pylint to utilize all available CPU cores (#676) This is a nice addition to the linter's responsiveness. Co-authored-by: Abhinav Singh <mailsforabhinav@gmail.com> * Add a config for Coverage.py (#679) * Add a config for pytest (#680) * Ignore a `DeprecationWarning` in `pytest` config (#683) This was added in Python 3.10 and needs to be dealt with properly. * Adopt BaseTcpServerHandler within HttpProtocolHandler (#681) * Rename .server to .upstream * Lint fixes * Mark internal methods with _ prefix * Fix broken test * lint changes * Wah, double client :D * Avoid selector initialization for threadless mode * remove unused imports * Now HttpProtocolHandler implements BaseTcpServerHandler * Consistent return and guard againt upstream.closed * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * AcceptorPool as context manager * Group multiprocessing imports together * Use com.jaxl bundle identifier as proxy.py will eventually move under jaxl org * revisit devtools integration :) * Emit all necessary events for devtools integration * Lint fixes Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Inline the dist description in pkg metadata (#684) This is necessary because setuptools is about to deprecate supporting LF in one-line metadata fields. Co-authored-by: Abhinav Singh <mailsforabhinav@gmail.com> * Add a pytest-based test for catching import loops (#678) Co-authored-by: Abhinav Singh <mailsforabhinav@gmail.com> * Test built artifacts in the CI instead of Git src (#682) This approach helps make sure that what is tested in the project is the same as what the end-users get installed on their machines. It will catch failures to include important failes into the packages that get uploaded to PyPI. Co-authored-by: Abhinav Singh <mailsforabhinav@gmail.com> * Fix python 3.10 @ ubuntu pytest ignore (#685) * Use suggested fix in #683 to remove pytest ignore * Handle should not flush now which can block, instead let base tcp handler do its magic * test speed up, doc string, logging enhancements * Move macOS to the end of workflows * Fix mypy warnings * must_flush_before_shutdown until entire client buffer has been flushed (#686) * Readme Updates (#687) * FilterByUpstreamHostPlugin now uses facebook as example * Advertise that cloudflare plugin require httpx library * Advertise Dashboard as WIP * More clarity around how plugins can be loaded * Advertise how to get direct access to Chrome DevTools websocket endpoint * Move plugin developer and contributor guide below the fold * Update README.md Co-authored-by: Sviatoslav Sydorenko <wk@sydorenko.org.ua> * bash to console and consistent prompt sign * Remove pytest commented out ignore clause * Add dummy CustomNetworkInterface plugin documentation Co-authored-by: Sviatoslav Sydorenko <wk@sydorenko.org.ua> * Response bodies can be empty or missing + HttpParser refactoring (#688) * Responses can have None body, remove assertions, update modify chunk plugin to not modify chunks for responses with no content * Address mypy warning after removing assertion * Reusable get_body_or_chunks * Order methods by public/private, mark private ones with _ prefix * HttpParser.url deprecation notice (renamed to _url). Add zero-copy todo * [ModifyChunkResponsePlugin] Only bail out of body is not expected (#690) * DEFAULT_CA_FILE is now certifi/cacert.pem (#691) * Add FAQ: OSError when wrapping client for TLS Interception * Silence exception log for several valid "cert verification failed" by client during tls interception * Lint checks * Move exception handling within wrap_server/wrap_client methods * Lint fixes * Use certifi/cacert.pem as default --ca-file flag value * Address tests after DEFAULT_CA_FILE change * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Expose pre-commit tool hooks execution times (#692) * Work (#693) * Refactor work acceptor and executor * Lint fixes * Fix expression-not-assigned pylint error * Pool (#694) * Refactor pool * mypy fixes * Fix import (relative) * Add WebScraper example skeleton & ConnectionPool skeleton * Add ConnectionPool class * Integrate ConnectionPool with proxy server (experimental) * Lint fixes * Remove unused imports. TODO: Put pool behind a flag. Default to false for now * Make ConnectionPool multiprocess safe. Later we want to make it safe but without using locks * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Remove unused imports * Return created flag from acquire * Guard connection pool behind --enable-conn-pool flag * Flag belongs within connection pool class * spelling * self.upstream = None only for pool config Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Top-level notion of work not client (#695) * Top-level notion of work not client * Update ssl echo server example * Rename `Proxy.initialize` as `FlagParser.initialize` (#696) * Move Proxy.initialize within FlagParser.initialize. Also move other staticmethods from within proxy class into utils * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * remove unused imports * Fix `import-outside-toplevel` error * add `make lib-flake8` and `make lib-mypy` targets Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Add a `--unix-socket-path` flag (#697) * Add a `--unix-socket-path` flag. When available `--hostname` and `--port` flags are ignored. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * `print` statement is allowed only in `flags.py` and `version-check.py`. All other places must use a `logger` instance * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Add guard for `AF_UNIX` on Windows * Comment out assertion on Windows for now Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Trigger workflows only when necessary file changes are detected (#699) * Trigger workflows only when necessary file changes are detected * Address yamllint * Trigger workflows even when workflow file itself is changed * Also include `examples` directory, may be one day we will have tests for them * `brew` and `docker` workflows must also only execute for python file changes. This will skip them for pure README.md changes * Execute `lib` workflows for package artifact changes * Disable static web server test on GHA environment (flaky on Ubuntu) (#700) * Add `lib-dep` makefile target (#701) * Put core flags where they belong (#702) * Move flags to where they belong * Move `get_default_plugins` within FlagParser as it depends upon args TODO: We need plugin dependency system * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Update types-paramiko from 2.7.3 to 2.7.4 (#704) * Fix `--enable-dashboard` flags (#707) * Expose within __all__ * Enable `--numprocesses=auto` for `pytest.ini` * make lib-lint * Also consider `--plugins` flag when bootstrapping plugins * Add `from .dashboard import ProxyDashboard` in top-level `__init__.py` to make `ProxyDashboard` flags auto discoverable * Move `--enable-dashboard` to top-level * Move logging utility within `Logger` class * Consider comma separated --plugin and --plugins during discover_plugins * Refactor plugin related utilities in Plugins module * mypy and lint * Fix unused import * Safe to use tempdir on Github actions to avoid race conditions??? * pki (generically disk based file) based tests are flaky on macOS under parallel execution * Move pid file write/remove within `AcceptorPool` (#708) * Move pid file write/remove within AcceptorPool * Remove unused * `EventManager` is also a context manager (#709) * `EventManager` is also a context manager * unused * Rename `EventManager.event_queue` to `EventManager.queue` * `--threadless` default for `Python 3.8+` on `mac` and `linux` (#710) * Explicit `multiprocessing.Manager.shutdown` Multiprocessing manager is used within eventing core. From doc, it appears to start a BaseManager which starts a server???? Seriously???? Anyways, using multiprocessing manager is a PITA and mistake, as it doesn't even give us performance we expect. Our proxy server can handle more requests than what multiprocess manager can exchange between processes. * `--threadless is now ON by default for `Python 3.8+` on `mac` and `linux` environments * Clarity around why multiprocessing.Manager must be deprecated * Add `--threaded` flag which can be used to fallback for environments where `--threadless` is now default * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * never used * Update README * Use `threaded=True` in tests which were written for threaded model * Fix issue where sharing manager between global event queue and subscriber can lead to TypeError Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * `OP_NO_TLSv1_1` by default for upstream connection negotiations (#712) * print mode via acceptor pool * `OP_NO_TLSv1_1` by default for upstream connection negotiations. Fixes #639 * Proper fix for flaky static web server test. Diff in payload was due to a different compression algorithm being used. * mypy fixes * Add more context in intro (#713) * Add more context in intro * Take another pass * Add threadless in log section * Add an explicit config for darglint (#717) * Add `--num-acceptors` flag + Allow `work_klass` via `Proxy` context manager kwargs (#714) * Allow overriding work_klass via Proxy context manager kwargs * Decouple acceptor and executor pools * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Add `--num_acceptors` flag and better load balancing * Remove unused * Lint errors * Another arg not kwarg * Move start work staticmethods within ExecutorPool * mypy fixes * Update README with `--num-acceptors` flag * Rename `Proxy.pool` to `Proxy.acceptors` * Add SetupShutdownContextManager abstraction * Match --num-acceptors logic with PR description * Rename executor utility methods and add docstring * Remove work_klass from constructors and pass it via flags * Update docstring for pools as they no longer accept a work_klass argument * Turn work_klass into a flag. main() no longer accepts input_args (only kwargs opts). Similarly, Proxy doesnt accept any input_args now (only kwargs opts) * Expose default work klass in README * Expose `HttpProtocolHandler` and `HttpProtocolHandlerPlugin` within `proxy.http` module * Start to fix tests * Fix tests * mypy and flake8 * Trailing comma * Remove unused var * Unused arg * uff Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Update badges to match new GHA workflows (#718) * Add `Listener`, Web server close on header, use `Pipe` instead of `Manager` in eventing core (#720) * Abstract out a Listener class * unused * Use connection instead of manager queue * For web close connection of client requested via headers * Remove eventing WIP module * Sub and Unsub ack * Fix tests * mypy and flake8 * comma * Move callback within EventSubscriber constructor * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Skip test_unix_path_listener on Windows * Spelling fix Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Test cov (#721) * nocover for abstract classes * Add event manager test * Assert call args * Sponsored by `Jaxl Innovations Private Limited` * Allow `--plugins` flag to be used multiple times (#725) * deprecate server_file_or_404 * Optionally compress static content. Currently only if content length higher than 300 * trailing comma * Allow `--plugins` flag to be used multiple times Following are valid invocation: 1) `--plugins A` 2) `--plugins A,B` 3) `--plugins A --plugins B` 4) `--plugins A,B --plugins C` * mypy * Flake8 * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * correct type * Add `HttpParser.is_https_tunnel()` utility method * mypy * lint checks * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Add `--proxy-pool` flag (#727) * Add `--proxy-pool` flag * lint checks * Custom Url Parser (#730) * Custom Url parser for our needs * lint fix * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix proxy_pool plugin as scheme can be None if not present in the Url * Address the ambiguous ipv6:port scenario along with valid cases * lint checks * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * docstring * Abstract into `http.parser` module * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix #398 HTTP/1.0 related issue * lint checks Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Add more info in log context (#732) * Provide more info in log context, ideally we could just pass client/upstream/request/response objects but for now passing dict is ok * lint checks * Allow `access_log` format override by web plugins (#733) * Return DEFAULT_404_RESPONSE by default from static server when file doesnt exist * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix web server with proxy test Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Add `--auth-plugin` flag to override default basic auth plugin (#734) * Allow override of `--auth-plugin`. This can be used in scenarios where a hardcoded `--basic-auth` credentials cannot be used across all your users. May-be you want to connect to a database before authorizing the request. For all such scenario, disable in-build default auth plugin by providing `--auth-plugin` flag. * Allow usage of `--auth-plugin` flag without having to provide `--basic-auth` flag. When `--basic-auth` flag is used, default auth plugin will load. When `--basic-auth` and `--auth-plugin` flag is used, custom auth plugin will load * Address test_main broken after auth_plugin flag * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update README * Fix long line Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Switch versioning to be SCM-based (#715) Co-authored-by: Abhinav Singh <mailsforabhinav@gmail.com> * `--enable-proxy-protocol` : HAProxy Protocol v1 (#735) * Introduce `--haproxy-protocol` flag * Complete proxy protocol v1 implementation, enable using `--enable-proxy-protocol` flag * link checks * Advertise support for haproxy protocol in readme * Add make target `lib-scm-version` * `make lib-version` is now `make lib-check` * Dont enforce -dev part of version within README * Add provision to update readme flags using check * Wrap help text within console * Add closing ticks * Remove verbose logging and update homebrew formulae (may be fixed?) * Reuse a pre-existing `version_tuple` from SCM * No abstract method for proxy plugin (#738) * Remove abstractmethod for proxy plugin base class, remove unused methods from bundled plugins * Move httpStatusCodes, httpMethods and Url within top-level proxy.http package * Implement publishing via GHA (#716) * Enable release-testpypi (#741) * Fix dist version in CI/CD on pushes to `develop` (#743) * Fix homebrew formulae URL to install from git and not zip because of scm changes (#744) * Add an example of loading external plugins * Move GHA e2e integration job to pytest (#746) Co-authored-by: Abhinav Singh <126065+abhinavsingh@users.noreply.github.com> * Make Git archives `setuptools-scm` compatible (#737) Co-authored-by: Abhinav Singh <126065+abhinavsingh@users.noreply.github.com> * Add initial Sphinx docs * Handle invalid/malformed data from clients in HttpParser (#740) * add validation in _process_line in parser.py add validation in _process_line in proxy/http/parser/parser.py * quick fail when parsing request quick fail when parsing request add test case for parsing invalid http request * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * remove unnecessary checks and empty line remove unnecessary checks and empty line * minor fix minor fix * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * solve exception expression conflict solve exception expression conflict * use NotImplementedError temporary measure use change HttpProtocolException to NotImplementedError for a temporary measure * change exception type in test change exception type in test * remove unnecessary import remove unnecessary import Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Abhinav Singh <mailsforabhinav@gmail.com> Co-authored-by: Abhinav Singh <126065+abhinavsingh@users.noreply.github.com> * Add all IANA assigned HTTP method names (#751) * Ignore `docs/_build` * Remove `v` prefix from VERSION. Also added a `lib-doc` target * Raise a `ValueError` instead of `NotImplementedError` * Add all registered http method verbs * Generate `_scm_version.py` on every `make` invocation. Fix `v` prefix bug. * `+proxy` for emails * Make explicit that this script writes to file * `PROXY_AGENT_HEADER_VALUE` still needs the `v` :) * Fix Makefile to use tox for docs (#752) * Fix Makefile to use tox for docs * Resolve `open` based upon operating system Co-authored-by: Abhinav Singh <126065+abhinavsingh@users.noreply.github.com> * Include GitHub's CoC and security in Sphinx docs (#755) Co-authored-by: Abhinav Singh <126065+abhinavsingh@users.noreply.github.com> * Add community contribution guidelines (#757) * [IntegrationTest] Use `127.0.0.1` as target address and a random port (#756) * Use `127.0.0.1` as target address and a random port * Fix spellcheck-docs * Transparent `ProxyPy.png` (#759) * Add `TcpUpstreamConnectionHandler` class (#760) * Add `TcpUpstreamConnectionHandler` which can be used as standalone or as mixin * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * `TcpUpstreamConnectionHandler` is now an abstract class * Fix mypy * `nitpick_ignore` the `proxy.core.base.tcp_upstream.TcpUpstreamConnectionHandler` class * Add mypy exception for now * Fix flake * Fix docstring Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Add `.vscode` settings (#761) * Added `DEFAULT_SELECTOR_SELECT_TIMEOUT` (#762) * Enhancements (#763) * Update `make lib-profile` * Optimize `utils.find_http_line` * Pass work to executors within their own multiprocessing lock * Fix tests * Add `(_py_class_role, Url)` to fix rtfd :D * Run push workflow only for push to master and develop branch (#764) * Run push workflow only for push to master and develop branch * remove comments from `settings.json` * Single workflow (#765) * To optimize pull all workflows within check guard * give space between version and to * Move all other workflow under `other-checks` guard * Dont run brew/dashboard/docker until lib checks have passed * Additional check bridging slows it down * Name diff via emoji * More emojis * 🐳 * Per GitHub it will try to maximize, lets fallback to GHA behavior * False position (not indented) * Remove tagline from readme, its in hero image now * Remove badge for deprecated workflows * [ImgBot] Optimize images (#766) /ProxyPy.png -- 985.28kb -> 884.17kb (10.26%) Signed-off-by: ImgBotApp <ImgBotHelp@gmail.com> Co-authored-by: ImgBotApp <ImgBotHelp@gmail.com> * Acceptors performance (#767) * Use threads for delegation. Now `run_once` lock before `accept` not `select` * Add support to use master proxy within proxy pool plugin. When used, proxy pool plugin will be a no-op for the master node * Fix acceptor tests now that mask is being used * Use `cached_property` for web server routes * Use `select(timeout=1)` otherwise acceptor wont join if total blocking * mypy, flake, doc spell fixes * R0205: Class `cached_property` inherits from object, can be safely removed from bases in python3 (useless-object-inheritance) * Include contributing guidelines in Sphinx docs (#771) * Fix tox invocation in the doc make target (#772) Co-authored-by: Abhinav Singh <126065+abhinavsingh@users.noreply.github.com> * Declare project URLs in Python package metadata (#778) * Declare project URLs in Python package metadata * Add link to GHA discussions Co-authored-by: Sviatoslav Sydorenko <webknjaz@redhat.com> Co-authored-by: Abhinav Singh <126065+abhinavsingh@users.noreply.github.com> * Async `get_events`, `handle_event`, `handle_readables`, `handle_writables` (#769) * Asynchronous `handle_event` and `LocalExecutor` thread * Bail out on first task completion * mypy * Add `helper/benchmark.sh` and fix threaded which must now use asyncio (reduced performance of threaded) * Print open file diff from `benchmark.sh` * Add `--local-executor` flag, disabled by default for now until tests are updated * Async `handle_readables` and `handle_writables` for `HttpProtocolHandlerPlugin` interface (doesnt impact proxy/web plugins for now) * Async `get_events` * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Address tests after async changes * mypy and flake8 * spelldoc * `check.py` and trailing comma * Rename to `_assertions.py` * Add missing `pytest-mock` and `pytest-asyncio` deps * Add `pytest-mock` to `pylint` deps * Correct use of `parameterize` and add `PT007` to flake8 ignores * Fix mypy hints broken for `< Python3.9` * Remove usage of `asynccontextmanager` which is not available for all Python versions that `proxy.py` supports * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix for pre-python-3.9 versions * `AsyncTask` apis `set_name` and `get_name` are not available on all supported versions * Install setuptools via `lib-dep` until we recommend editable install * Deprecate support for `Python 3.6` * Use recommendation suggested here https://github.com/abhinavsingh/proxy.py/pull/769\#discussion_r753840929 * Address recommendation here https://github.com/abhinavsingh/proxy.py/pull/769\#discussion_r753841906 * Make `Threadless` agnostic of `multiprocessing.Process` * Acceptors must dispatch to local executor in non-blocking fashion * No daemon for executor processes and fix shutdown logic * Only return fds from `_selected_events` not all events data * Refactor logic * Prefix private methods with `_` * `work_queue` and not `client_queue` * Turn `Threadless` into an abstract executor. Introduce `RemoteExecutor` * Make `LocalExecutor` agnostic of `threading.Thread` * `LocalExecutor` now implements `Threadless` * `get_events` and `get_descriptors` now must return int and not sock. `Threadless` now avoids repeated register/unregister and instead make use of `selectors.modify` * Fix `main` tests * Apply suggestions from code review Co-authored-by: Sviatoslav Sydorenko <wk@sydorenko.org.ua> * Apply code review recommendations manually * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Revert back `Any` and use `addr or None` * Address `flake8` * Update tests to use `fileno` * Fix doc build * Fix doc spell, use tear down and not teardown * Doc updates * Add back support for `Python 3.6` * Acceptors dont need loop initialization * On Python 3.6 `asyncio.new_event_loop()` is necessary * Make doc happy * `--threaded` needs a new event loop for 3.7 too * Always use `asyncio.new_event_loop()` for threaded mode Added e2e integration tests (subprocess & curl) for all modes. * Lint fixes Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Sviatoslav Sydorenko <wk@sydorenko.org.ua> * Prune unnecessary directories from `sdist` package (#779) * Prune unnecessary directories from `sdist` package * Remove `LICENSE` & `README.md` from `MANIFEST.in` Co-authored-by: Sviatoslav Sydorenko <wk@sydorenko.org.ua> * Explicitly ignore top-level images and include `.github` folder Co-authored-by: Sviatoslav Sydorenko <wk@sydorenko.org.ua> * Use proper email for the Git user in GHA (#781) * Use proper email for the Git user in GHA * Use an action to set the GHA user in Git * Publish a GitHub Release after tagging (#782) Co-authored-by: Abhinav Singh <126065+abhinavsingh@users.noreply.github.com> * Create `dependabot.yml` (#783) * Create dependabot.yml * Lint fixes * Yamllint * Reviewers fix * Optimize (#780) * Optimize `find_http_line` which is in critical path * Update benchmark results * Keep the loop hot, TCP no delay, cleanup inactive check periodically * Check for shutdown signal with tick * Use non-reentrant `NonBlockingQueue` implementation instead of `queue.Queue` * Fix listener test * lint and doc * Convert `recv` errors as warnings and not exceptions (#787) * Update codecov config to include separate tests and lib sub-projects (#788) * Update codecov config to include separate tests and lib sub-projects * Yaml lint * Bump actions/cache from 2.1.5 to 2.1.7 (#784) Bumps [actions/cache](https://github.com/actions/cache) from 2.1.5 to 2.1.7. - [Release notes](https://github.com/actions/cache/releases) - [Commits](actions/cache@v2.1.5...v2.1.7) --- updated-dependencies: - dependency-name: actions/cache dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Abhinav Singh <126065+abhinavsingh@users.noreply.github.com> * pip prod(deps): bump rope from 0.21.0 to 0.22.0 (#785) Bumps [rope](https://github.com/python-rope/rope) from 0.21.0 to 0.22.0. - [Release notes](https://github.com/python-rope/rope/releases) - [Changelog](https://github.com/python-rope/rope/blob/master/CHANGELOG.md) - [Commits](python-rope/rope@0.21.0...0.22.0) --- updated-dependencies: - dependency-name: rope dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Abhinav Singh <126065+abhinavsingh@users.noreply.github.com> * npm: bump chrome-devtools-frontend in /dashboard (#786) Bumps [chrome-devtools-frontend](https://github.com/ChromeDevTools/devtools-frontend) from 1.0.827632 to 1.0.944903. - [Release notes](https://github.com/ChromeDevTools/devtools-frontend/releases) - [Changelog](https://github.com/ChromeDevTools/devtools-frontend/blob/main/docs/release_management.md) - [Commits](https://github.com/ChromeDevTools/devtools-frontend/commits) --- updated-dependencies: - dependency-name: chrome-devtools-frontend dependency-type: direct:development update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Abhinav Singh <126065+abhinavsingh@users.noreply.github.com> * pip prod(deps): bump types-paramiko from 2.7.4 to 2.8.1 (#799) Bumps [types-paramiko](https://github.com/python/typeshed) from 2.7.4 to 2.8.1. - [Release notes](https://github.com/python/typeshed/releases) - [Commits](https://github.com/python/typeshed/commits) --- updated-dependencies: - dependency-name: types-paramiko dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * npm: bump eslint-plugin-import from 2.22.1 to 2.25.3 in /dashboard (#798) Bumps [eslint-plugin-import](https://github.com/import-js/eslint-plugin-import) from 2.22.1 to 2.25.3. - [Release notes](https://github.com/import-js/eslint-plugin-import/releases) - [Changelog](https://github.com/import-js/eslint-plugin-import/blob/main/CHANGELOG.md) - [Commits](import-js/eslint-plugin-import@v2.22.1...v2.25.3) --- updated-dependencies: - dependency-name: eslint-plugin-import dependency-type: direct:development update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Build docker container for all linux target architectures`386`, `amd64`, `arm/v6`, `arm/v7`, `arm64/v8`, `ppc64le`, `s390x` (#797) * Fixes #768 * buildx build * We need `setuptools_scm` to run `Makefile` * Build `docker` container using `wheel` * Download package distribution * Adjust supported platforms * Alpine only supports v6 * . * May-be fix `lsb_release` * Enable `multiarch` env for docker buildx * Deprecate `DOCKER_IMAGE_TAG` which removes `write-scm-version.py` dependency. Also pass targetplatform as an argument to `container-buildx` make target * Remove `setuptools_scm` dep for docker workflow step * woof * Match all target platforms to match `python-alpine` docker * yamllint * Use `yamllint disable rule:line-length` for `Dockerfile` * Tag the container using `dist-version` * Replace + with . for dev version tag * Add `PROXYPY_CONTAINER_VERSION` step * Fix publishing a GitHub Release from GHA (#811) * pip prod(deps): bump coverage from 6.1.1 to 6.1.2 (#813) Bumps [coverage](https://github.com/nedbat/coveragepy) from 6.1.1 to 6.1.2. - [Release notes](https://github.com/nedbat/coveragepy/releases) - [Changelog](https://github.com/nedbat/coveragepy/blob/master/CHANGES.rst) - [Commits](nedbat/coveragepy@6.1.1...6.1.2) --- updated-dependencies: - dependency-name: coverage dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Fix matching requested version with Git tag in GHA (#815) * Fix the tagging condition in GHA build job (#816) * Publish to TestPyPI from the release request jobs (#819) * Handle `KBI` (#821) Co-authored-by: pyup.io bot <github-bot@pyup.io> Co-authored-by: Aksh Gupta <akshgpt7@gmail.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua> Co-authored-by: imgbot[bot] <31301654+imgbot[bot]@users.noreply.github.com> Co-authored-by: ImgBotApp <ImgBotHelp@gmail.com> Co-authored-by: Sviatoslav Sydorenko <wk@sydorenko.org.ua> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: JerryKwan <Jinzhong.Guan@gmail.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
pre-commit.ciweb-service runs checks in a container with networking disabled. This is whymypy --install-typesdoesn't work there and the additional dependencies need to be declared explicitly. This patch does just that.