Skip to content

Commit

Permalink
Big improvements to saving requests back to requester file, added iss…
Browse files Browse the repository at this point in the history
…ue template
  • Loading branch information
kylebebak committed Oct 7, 2017
1 parent 0584c64 commit 0c17463
Show file tree
Hide file tree
Showing 7 changed files with 63 additions and 37 deletions.
71 changes: 40 additions & 31 deletions docs/_content/contributing.md → CONTRIBUTING.md
@@ -1,61 +1,44 @@
# Contributing
Please do! Possible improvements:
Please do! Some possible improvements:

- Snippets and examples for more auth schemes
+ Requester can import __any Python package__
+ Remember, Requester can import any Python package
* OAuth 2, using `requests-oauthlib`
* Other common auth protocols, using whichever package is best
+ `RequesterAuthOptionsCommand` for snippets
+ Examples in documentation
+ Snippets are defined in `RequesterAuthOptionsCommand`
+ Examples of how to use snippets should be [added to documentation](https://github.com/kylebebak/Requester/blob/master/docs/_content/body.md)
- More export/import formats
+ [HTTP](https://tools.ietf.org/html/rfc7230)
+ Look at `commands/import_export.py`
- Randomized requests with benchmark runs (e.g. for fuzz testing)
+ Execute all requests constructred from an iterable?
+ Execute all requests constructed from an iterable?
- Improve architecture and test coverage
+ More pure functions, fewer coupling points to Sublime Text API
+ Test coverage for syntax files
- Tell your friends about Requester!


## How Does It Work?
The core API is defined in modules under the `core` directory. The main class is `RequestCommandMixin`.

This mixin is the motor for parsing an env, executing requests in parallel in the context of this env, invoking activity indicator methods, and invoking response handling methods. These methods can be overridden to control the behavior of classes that inherit from this mixin.

The mixin uses the `ResponseThreadPool` class, which wraps a thread pool to execute requests in parallel. Default concurrency is determined by the mixin's `MAX_WORKERS` class property. The thread pool is inspected at regular intervals to remove completed responses and handle them, and show activity for pending requests.

Command classes that use this mixin can override `handle_response` and/or `handle_responses`. This way they can handle responses one at a time as they are completed, or as a group when they're all finished. Each response object contains a `req` namedtuple with a bunch of useful properties, the `res` (a __requests.Response__ object), and an `err` string. Responses are sorted by request parsing order.

## Issues and Bugs
If you open an issue, follow the guidelines in the issue template to make the issue easier to reproduce.

### Parser
Command classes __must__ also override `get_requests`, which must return a list of request strings, for example strings parsed from the current view. To simplify this, `core` has a `parsers` module. The important parser is `parse_requests`. It takes a string, such as a selection from a view, and returns a list of all calls to `requests` in the string.

## Pull Requests
If you're going to submit a PR, read on for instructions on how to set up Requester for development.

### Use of Eval and Exec
`exec` is used to build an environment dict from a string, in basically the same way `import` populates a variables dict from a module. The `exec`ed string is simply the concatenation of the env block and the env file.

`eval` is used in `prepare_request`, in conjunction with `parse_args`, to parse the args passed to `requests.<method>`. This way Requester can modify, remove, or add to these args before actually calling `requests.<method>`. Much of Requester's most powerful behavior derives from this ability.


### Writing a New Command Class
If you want to write a new command class for Requester, check out how `RequesterCommand` works; it simply uses the mixin and overrides `get_requests` and `handle_response`.

If you want a better understanding of the details, dive into the `core` directory. This is where the heavy lifting is done.


## Installing the `pre-commit` Git hook
### Installing the `pre-commit` Git hook
Run `cd .git/hooks && ln -s -f ../../docs/_hooks/* ./` from the root of the repo. This creates a symlink from the `.git/hooks` directory to Requester's `pre-commit` hook. This hook runs code linting and style checking with `flake8`, and builds the documentation from the sources in `docs/_content`.


## Python Linting and Code Style
### Python Linting and Code Style
Uses __flake8__. First, install __flake8__ with `pip3 install flake8`.

As long as you installed Requester's `pre-commit` hook (see above), you will be unable to commit anything that doesn't pass __flake8__ validation.

All classes and methods should have docstrings, limited to 82 characters per line. Except for `run` and `__init__`. Feel free to add comments for anything that's not obvious.


## Tests
### Tests
Tests are divided into tests of the `core` package, which depend on a mocked `sublime` and run on Travis, and integration tests run within Sublime Text.

Many tests for Requester are asynchronous, because they depend on responses coming back before examining response tabs. For this reason, tests are executed against a local server powered by __httpbin__. You can install __httpbin__ by running `pip install httpbin`. You can then run it with `gunicorn httpbin:app` or `gunicorn --access-logfile /dev/stdout httpbin:app`.
Expand All @@ -67,5 +50,31 @@ To run the integration tests, install __UnitTesting__ via Package Control. Read
Run integration tests before committing changes. Look for __UnitTesting__ in the command palette, hit enter, and type `Requester`. Or, if you've created a project for Requester, run __UnitTesting: Test Current Project__.


## Docs
### Docs
`README.md` is generated by Requester's `pre-commit` hook. If you want to improve the docs, __don't edit README.md__. The correct file is `docs/_content/body.md`.


## How Does Requester Work?
The core API is defined in modules under the `core` directory. The main class is `RequestCommandMixin`.

This mixin is the motor for parsing an env, executing requests in parallel in the context of this env, invoking activity indicator methods, and invoking response handling methods. These methods can be overridden to control the behavior of classes that inherit from this mixin.

The mixin uses the `ResponseThreadPool` class, which wraps a thread pool to execute requests in parallel. Default concurrency is determined by the mixin's `MAX_WORKERS` class property. The thread pool is inspected at regular intervals to remove completed responses and handle them, and show activity for pending requests.

Command classes that use this mixin can override `handle_response` and/or `handle_responses`. This way they can handle responses one at a time as they are completed, or as a group when they're all finished. Each response object contains a `req` namedtuple with a bunch of useful properties, the `res` (a __requests.Response__ object), and an `err` string. Responses are sorted by request parsing order.


### Parser
Command classes __must__ also override `get_requests`, which must return a list of request strings, for example strings parsed from the current view. To simplify this, `core` has a `parsers` module. The important parser is `parse_requests`. It takes a string, such as a selection from a view, and returns a list of all calls to `requests` in the string.


### Use of Eval and Exec
`exec` is used to build an environment dict from a string, in basically the same way `import` populates a variables dict from a module. The `exec`ed string is simply the concatenation of the env block and the env file.

`eval` is used in `prepare_request`, in conjunction with `parse_args`, to parse the args passed to `requests.<method>`. This way Requester can modify, remove, or add to these args before actually calling `requests.<method>`. Much of Requester's most powerful behavior derives from this ability.


### Writing a New Command Class
If you want to write a new command class for Requester, check out how `RequesterCommand` works; it simply uses the mixin and overrides `get_requests` and `handle_response`.

If you want a better understanding of the details, dive into the `core` directory. This is where the heavy lifting is done.
5 changes: 5 additions & 0 deletions ISSUE_TEMPLATE.md
@@ -0,0 +1,5 @@
Requester prints debug information to the Sublime Text console. Go to __View > Show Console__ and include any info printed there.

If your issue is related to a request failing or not returning the expected response, include the request string and env vars.

And please include your OS/version!
4 changes: 2 additions & 2 deletions README.md
Expand Up @@ -169,7 +169,7 @@ Imagine you're debugging a request in a response tab, replaying and modifying th

In a response tab, go to the command palette and look for __Requester: Save Request Back To Requester File__, or look in the response tab for the keyboard shortcut to __save request__.

If the requester file still exists and your original request hasn't been changed since you first sent it, Requester will overwrite the it with the modified one from your response tab.
If the requester file exists and you haven't changed your original request since you sent it, Requester opens the requester file, scrolls to the original request, and overwrites it with the modified one from your response tab. You can repeatedly save back a request as you edit it in the response tab. ✨✨


### Environment Variables
Expand Down Expand Up @@ -739,7 +739,7 @@ Requester's modifiable settings, and their default values. You can override any


## Contributing and Tests
[See here](https://github.com/kylebebak/Requester/blob/master/docs/_content/contributing.md).
[See here](https://github.com/kylebebak/Requester/blob/master/CONTRIBUTING.md).


## Inspired By
Expand Down
7 changes: 6 additions & 1 deletion commands/request.py
Expand Up @@ -548,7 +548,8 @@ def run(self):
sublime.error_message('Save Error: requester file\n"{}"\nno longer exists'.format(file))
return

requester_view = view.window().open_file(file)
is_open = bool(self.window.find_open_file(file))
requester_view = self.window.open_file(file)

def save_request():
"""Wait on another thread for view to load on main thread, then save.
Expand All @@ -572,6 +573,10 @@ def save_request():
{'text': request, 'start_index': start_index, 'end_index': start_index + len(request)})
requester_view.sel().clear()
requester_view.sel().add(sublime.Region(start_index))
if not is_open: # hacky trick to make sure scroll works
time.sleep(.2)
else:
time.sleep(.1)
requester_view.show_at_center(start_index)
set_save_info_on_view(view, request)

Expand Down
4 changes: 2 additions & 2 deletions docs/_content/body.md
Expand Up @@ -82,7 +82,7 @@ Imagine you're debugging a request in a response tab, replaying and modifying th

In a response tab, go to the command palette and look for __Requester: Save Request Back To Requester File__, or look in the response tab for the keyboard shortcut to __save request__.

If the requester file still exists and your original request hasn't been changed since you first sent it, Requester will overwrite the it with the modified one from your response tab.
If the requester file exists and you haven't changed your original request since you sent it, Requester opens the requester file, scrolls to the original request, and overwrites it with the modified one from your response tab. You can repeatedly save back a request as you edit it in the response tab. ✨✨


### Environment Variables
Expand Down Expand Up @@ -652,7 +652,7 @@ Requester's modifiable settings, and their default values. You can override any


## Contributing and Tests
[See here](https://github.com/kylebebak/Requester/blob/master/docs/_content/contributing.md).
[See here](https://github.com/kylebebak/Requester/blob/master/CONTRIBUTING.md).


## Inspired By
Expand Down
3 changes: 2 additions & 1 deletion messages.json
Expand Up @@ -25,5 +25,6 @@
"2.16.0": "messages/2.16.0.txt",
"2.17.0": "messages/2.17.0.txt",
"2.18.0": "messages/2.18.0.txt",
"2.19.0": "messages/2.19.0.txt"
"2.19.0": "messages/2.19.0.txt",
"2.19.1": "messages/2.19.1.txt"
}
6 changes: 6 additions & 0 deletions messages/2.19.1.txt
@@ -0,0 +1,6 @@
# New Features/Bug Fix
Not exactly new features or a bug fixes, but saving requests back to the requester file was significantly improved and simplified.

- You can save back requests even if your requester file has been modified in the meantime. Save will only fail if the original request itself has been modified.
- You can now save back requests that you load from your history file.
- Scrolling to the original request now works reliably even if the requester file isn't currently open when you save back your request.

0 comments on commit 0c17463

Please sign in to comment.