Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Plugins] Allow custom command line flags per plugin #301

Closed
trianta2 opened this issue Mar 5, 2020 · 3 comments · Fixed by #452
Closed

[Plugins] Allow custom command line flags per plugin #301

trianta2 opened this issue Mar 5, 2020 · 3 comments · Fixed by #452
Assignees

Comments

@trianta2
Copy link

trianta2 commented Mar 5, 2020

The cache directory seems to be hard-coded as a random temporary directory:

uid=self.uid, cache_dir=tempfile.gettempdir())

Not sure why. Consider exposing this as a command line parameter, with default value of None, which results in a temporary directory.

@abhinavsingh
Copy link
Owner

Hi @trianta2 Thanks for reporting this. Frankly, cache plugin (and other plugins) only exists for demonstration purposes, though with time they are maturing enough for production usage out of box.

Before command line flag for cache_responses plugin, proxy.py core will need to expose Flags class in a way that allows plugins to define custom command line parameters.

I have repurposed this issue to address core enhancement before adding a flag for cache_responses plugin. Let me know if a cleaner or alternative & faster way exists for doing the same.

Cheers!!!

@abhinavsingh abhinavsingh changed the title Make the cache directory configurable Allow plugins to define custom command line flags Mar 7, 2020
@trianta2
Copy link
Author

trianta2 commented Mar 7, 2020

Hi @abhinavsingh I forked your repo and implemented this quickly (i.e. added a --cache-dir argument to global flags, and accessed this CLI argument via the flags object passed to CacheResponsesPlugin as a positional argument). However I noticed that a new cached file was being made each request, instead of serving back cached responses as I had hoped. Indeed, the more general problem is exposing plugin parameters.

@abhinavsingh
Copy link
Owner

abhinavsingh commented Mar 8, 2020 via email

@abhinavsingh abhinavsingh changed the title Allow plugins to define custom command line flags [Plugins] Allow custom command line flags per plugin Mar 24, 2020
abhinavsingh added a commit that referenced this issue Oct 3, 2020
* Allow plugins to add custom command line flags.  Addresses #301

* Reduce dependency over Flags class.  This will be deprecated so that adhoc flags can be added without any additional manual configuration

* Fix: Argument 1 to "mock_default_args" of "TestMain" has incompatible type "Namespace"; expected "Mock"

* Reduce Flags class to just the initializer.

* Store list of action dest in FlagParser
abhinavsingh added a commit that referenced this issue Oct 14, 2020
* Go flagless to allow custom user defined flags. Fixes #301

* Add --cache-dir flag for cache plugin (when used with on-disk store)

* Enable discovery of flags from external plugins, example those that reside outside of proxy.py package and loaded on demand.  This also allows external flags to surface in --help section

* Define --filtered-client-ips flag for FilterByClientIpPlugin
abhinavsingh added a commit that referenced this issue Dec 20, 2020
* Update mypy from 0.780 to 0.781 (#379)

* Add FilterByClientIpPlugin example (#381)

* Update mypy from 0.781 to 0.782 (#382)

* Update twine from 3.1.1 to 3.2.0 (#384)

* Update tox from 3.15.2 to 3.16.0 (#385)

* Update tox from 3.16.0 to 3.16.1 (#386)

* Document FilterByClientIpPlugin & ModifyChunkResponsePlugin (#387)

* Refactor plugin base classes for plugin specific flags (#388)

* Update to latest code signing recommendations

* Move HttpProtocolHandlerPlugin into separate file

* Dont add subject attributes if not provided by upstream. Also handle subprocess.TimeoutExpired raised during certificate generation.  Instead of retries, we simply close the connection on timeout

* Remove plugin specific flag initialization methods for now

* Update coverage from 5.1 to 5.2 (#390)

* Core acceptor pool doc, cleanup and standalone example (#393)

* Better document acceptor module and add a TCP Echo Server example

* autopep8 formating

* Rename ThreadlessWork --> Work class

* Make initialize, is_inactive and shutdown as optional interface methods.

Also introduce Readables & Writables custom types.

* Move websocket code into its own module

* Add websocket client example

* Cleanup websocket client

* Decouple SSL wrap logic into connection classes (#394)

* Move wrap functionality within respective connection classes. Also decouple websocket client handshake method

* Add a TCP echo client example that works with TCP echo server example

* Add SSL echo server & client example (#395)

* Move wrap_socket for SSL server within utils.

Also complete proxy.common.pki gen_csr and sign_csr actions. Used by Makefile sign-https-certificates.

* Add SSL echo server and client example

* Add examples documentation

* Add core pubsub eventing example and add menubar item skeleton (#396)

* Initialize menu bar items with click handler and open a popover for preferences

* Add Core PubSub eventing example

* Remove hardcoded request ids

* Move codecov.yml to top level directory (#400)

* Add cross ref for how to generate SSL certs. (#401)

* Add plugin "FilterByURLRegexPlugin" (#397)

* Initial draft of filter_by_url_regex.py

* Add FilterByURLRegexPlugin

* Fix dictionary key & add logging

* Add proper logging

* Add better logging

* Add logging

* move code to handle_client_request

* development logging

* development

* development

* development

* dev

* dev

* dev

* dev

* dev

* dev

* dev

* dev

* dev

* dev

* dev

* Fix blocked log

* Add to FILTER_LIST, some tidy up

* Update FILTER_LIST

* dev

* remove scheme from url

* Add to FILTER_LIST

* Add to FILTER_LIST

* Update FILTER_LIST

* commenting

* Update FILTER_LIST

* After autopep8

* Fix Anomalous backslash in string (pep8)

* Address code quality checks - flake8 F401 & W605

* Address flake8 errors

* Attempt to fix flake8 errors

* Fix linting issues

* Address flake8 W292

* Attempt to create tests

* Add FilterByURLRegexPlugin

* Rename test

* Work on tests

* Work on tests

* Work on tests

Co-authored-by: Abhinav Singh <mailsforabhinav@gmail.com>

* Update tox from 3.16.1 to 3.17.0 (#402)

* Update codecov from 2.1.7 to 2.1.8 (#404)

* Update tox from 3.17.0 to 3.17.1 (#403)

Co-authored-by: Abhinav Singh <mailsforabhinav@gmail.com>

* Bump lodash from 4.17.15 to 4.17.19 in /dashboard (#405)

Bumps [lodash](https://github.com/lodash/lodash) from 4.17.15 to 4.17.19.
- [Release notes](https://github.com/lodash/lodash/releases)
- [Commits](lodash/lodash@4.17.15...4.17.19)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Update tox from 3.17.1 to 3.18.0 (#406)

* Update coverage from 5.2 to 5.2.1 (#407)

* Update tox from 3.18.0 to 3.18.1 (#408)

* Fix docker build by using correct pip flags (#417)

* Update tox from 3.18.1 to 3.19.0 (#416)

Co-authored-by: Abhinav Singh <mailsforabhinav@gmail.com>

* Update autopep8 from 1.5.3 to 1.5.4 (#412)

Co-authored-by: Abhinav Singh <mailsforabhinav@gmail.com>

* Update pytest from 5.4.3 to 6.0.1 (#410)

Co-authored-by: Abhinav Singh <mailsforabhinav@gmail.com>

* npm upgrade (#418)

* Remove test for 'HttpWebServerRouteHandler'

This does not exist (as fas as I can see) and it bother the linter (Mypy),
when I tell it `klass` is a `type` instance.

* Pass a list plugin class objects or bytes to proxy when used in embeded mode.

No automated tests for the feature yet.

* Tests for Flags.load_plugins method.

* Ensure plugins are loaded only once.

Also changed module name for plugins passed by type.

* Update wheel from 0.34.2 to 0.35.0 (#421)

* Allow to use types when embeding Proxy (#420)

* Remove test for 'HttpWebServerRouteHandler'

This does not exist (as fas as I can see) and it bother the linter (Mypy),
when I tell it `klass` is a `type` instance.

* Pass a list plugin class objects or bytes to proxy when used in embeded mode.

No automated tests for the feature yet.

* Tests for Flags.load_plugins method.

* Ensure plugins are loaded only once.

Also changed module name for plugins passed by type.

Co-authored-by: Abhinav Singh <mailsforabhinav@gmail.com>

* Documentation for plugin loading in embedded mode (#422)

* Update pytest-cov from 2.10.0 to 2.10.1 (#423)

* Update wheel from 0.35.0 to 0.35.1 (#424)

* Update typing-extensions from 3.7.4.2 to 3.7.4.3 (#428)

* Update codecov from 2.1.8 to 2.1.9 (#427)

Co-authored-by: Abhinav Singh <mailsforabhinav@gmail.com>

* Update pylint from 2.5.3 to 2.6.0 (#426)

Co-authored-by: Abhinav Singh <mailsforabhinav@gmail.com>

* Update paramiko from 2.7.1 to 2.7.2 (#429)

* Update pytest from 6.0.1 to 6.1.0 (#436)

* Update coverage from 5.2.1 to 5.3 (#433)

Co-authored-by: Abhinav Singh <mailsforabhinav@gmail.com>

* Update tox from 3.19.0 to 3.20.0 (#430)

Co-authored-by: Abhinav Singh <mailsforabhinav@gmail.com>

* Update flake8 from 3.8.3 to 3.8.4 (#439)

* Allow plugins to add custom command line flags (#438)

* Allow plugins to add custom command line flags.  Addresses #301

* Reduce dependency over Flags class.  This will be deprecated so that adhoc flags can be added without any additional manual configuration

* Fix: Argument 1 to "mock_default_args" of "TestMain" has incompatible type "Namespace"; expected "Mock"

* Reduce Flags class to just the initializer.

* Store list of action dest in FlagParser

* Update pytest from 6.1.0 to 6.1.1 (#440)

* More examples (#444)

* Refactor into BaseServerHandler and BaseEchoServerHandler classes

* Add connect tunnel example

* Update rope from 0.17.0 to 0.18.0 (#445)

* Update tox from 3.20.0 to 3.20.1 (#446)

* Update codecov from 2.1.9 to 2.1.10 (#447)

* Update mypy (#449)

* Fix path to devtools websocket endpoint, broken after refactoring (#450)

* Relax proxy auth requirement to allow mixed case for the auth type e.g. "basic", "Basic", "BaSiC" are all allowed (#451)

* Go flagless to allow custom user defined flags. (#452)

* Go flagless to allow custom user defined flags. Fixes #301

* Add --cache-dir flag for cache plugin (when used with on-disk store)

* Enable discovery of flags from external plugins, example those that reside outside of proxy.py package and loaded on demand.  This also allows external flags to surface in --help section

* Define --filtered-client-ips flag for FilterByClientIpPlugin

* Separate basic auth plugin outside of core server (#453)

* Separate basic auth plugin outside of core

* Put basic auth plugin at top

* Create codeql-analysis.yml (#454)

* Create SECURITY.md (#455)

* Refactor (#456)

* Update pytest from 6.1.1 to 6.1.2 (#457)

* npm update (#460)

* Refactor base server interfaces into core modules (#461)

* Ensure pending buffers are flushed before shutting down in base_server.py

Handle unsupported scheme cases within connect_tunnel.py

* Move base implementations within core module

* Update ssl_echo_server

* Update wheel from 0.35.1 to 0.36.0 (#462)

* Update wheel from 0.36.0 to 0.36.1 (#463)

* Update pytest from 6.1.2 to 6.2.0 (#465)

* Update wheel from 0.36.1 to 0.36.2 (#466)

* Update pytest from 6.2.0 to 6.2.1 (#467)

* Update codecov from 2.1.10 to 2.1.11 (#469)

* Add version check for README.md (#471)

Co-authored-by: pyup.io bot <github-bot@pyup.io>
Co-authored-by: Mike <mike.nye@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Pascal COMBES <pascom@orange.fr>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants