Move 3rd party to extras (#841)

* moved keystone, capstone, unicorn and ropper to gef-extras

* moved test files to extras

* moved docs

* [docs] removed references to 3rd party packages

* [actions] mlc -> lychee

* updated PR & contrib templates and fixed actions command for lychee

* moved requirements.txt to tests/ so people don't think there's a need to install any package

* no need for readthedocs anymore

* replaced references to rtfd to gh-pages

.github/CONTRIBUTING.md

@@ -2,11 +2,9 @@
 
 ### License ###
 
-`gef` is placed under [license MIT](https://github.com/hugsy/gef/LICENSE) which
-provides Open-Source access to the code and its use.
+`gef` is placed under [license MIT](https://github.com/hugsy/gef/blob/master/LICENSE) which provides Open-Source access to the code and its use.
 
-By contributing to `gef` code through the _Pull Requests_ mechanism, you accept
-to release the code written by you under the said license.
+By contributing to `gef` code through the _Pull Requests_ mechanism, you accept to release the code written by you under the said license.
 
 
 ### Submitting a Patch ###
@@ -17,7 +15,7 @@ to release the code written by you under the said license.
     * Respect the coding style (same indentation format, explicit names as possible)
     * Comment your code
     * If you add a new feature/GDB command, also write the adequate
-      documentation (in [`docs/`](https://github.com/hugsy/gef/docs))
+      documentation (in [`docs/`](docs/))
 1. Submit a pull request, **make sure it is made against the `dev` branch (not `master`, which only contains tested and stable features)**
 1. The contributors will review your patch. If it is approved, the change will
    be merged via the GitHub, and you will be seen as contributors. If it needs

.github/ISSUE_TEMPLATE/bug_report.md

@@ -11,7 +11,7 @@ brackets if done/agreed):-->
 
 * [ ] Did you use the latest version of GEF from `dev` branch?
 * [ ] Is your bug specific to GEF (not GDB)? - Try to reproduce it running `gdb -nx`
-* [ ] Did you read the [documentation](https://gef.readthedocs.org/en/latest/) first?
+* [ ] Did you search through the [documentation](https://github.com/hugsy/gef/) first?
 * [ ] Did you check [issues](https://github.com/hugsy/gef/issues) (including
   the closed ones) - and the [PR](https://github.com/hugsy/gef/pulls)?
 

.github/PULL_REQUEST_TEMPLATE.md

@@ -1,31 +1,34 @@
-## <TITLE OF YOUR PATCH HERE> ##
 
-### Description/Motivation/Screenshots ###
+## Description/Motivation/Screenshots
+
 <!-- Describe technically what your patch does. -->
 <!-- Why is this change required? What problem does it solve? -->
 <!-- Why is this patch will make a better world? -->
 <!-- How does this look? Add a screenshot if you can -->
 
-### How Has This Been Tested? ###
+
+## How Has This Been Tested?
 
 | Architecture |          Yes/No          | Comments                                  |
 | ------------ | :----------------------: | ----------------------------------------- |
-| x86-32       | :heavy_multiplication_x: | Replace with :heavy_check_mark: if tested |
-| x86-64       | :heavy_multiplication_x: |                                           |
-| ARM          | :heavy_multiplication_x: |                                           |
-| AARCH64      | :heavy_multiplication_x: |                                           |
-| MIPS         | :heavy_multiplication_x: |                                           |
-| POWERPC      | :heavy_multiplication_x: |                                           |
-| SPARC        | :heavy_multiplication_x: |                                           |
-| RISC-V       | :heavy_multiplication_x: |                                           |
-| `make test`  | :heavy_multiplication_x: |                                           |
-
-### Checklist ###
+| x86-32       | :x:                      | Replace with :white_check_mark: if tested |
+| x86-64       | :x:                      |                                           |
+| ARM          | :x:                      |                                           |
+| AARCH64      | :x:                      |                                           |
+| MIPS         | :x:                      |                                           |
+| POWERPC      | :x:                      |                                           |
+| SPARC        | :x:                      |                                           |
+| RISC-V       | :x:                      |                                           |
+| `make test`  | :x:                      |                                           |
+
+
+
+## Checklist
 
 <!-- N.B.: Your patch won't be reviewed unless fulfilling the following base requirements: -->
 <!--- Put an `x` in all the boxes that are complete, or that don't apply -->
-- [] My PR was done against the `dev` branch, not `master`.
-- [] My code follows the code style of this project.
-- [] My change includes a change to the documentation, if required.
-- [] If my change adds new code, [adequate tests](docs/testing.md) have been added.
-- [] I have read and agree to the **CONTRIBUTING** document.
+- [ ] My PR was done against the `dev` branch, not `master`.
+- [ ] My code follows the code style of this project.
+- [ ] My change includes a change to the documentation, if required.
+- [ ] If my change adds new code, [adequate tests](docs/testing.md) have been added.
+- [ ] I have read and agree to the **CONTRIBUTING** document.

.github/workflows/generate-docs.yml

@@ -0,0 +1,20 @@
+name: Generate Documentation on GithubPages
+
+on:
+  push:
+    branches:
+      - dev
+      - main
+      - master
+
+jobs:
+  deploy:
+    runs-on: ubuntu-latest
+    if: github.event.repository.fork == false
+    steps:
+      - uses: actions/checkout@v2
+      - uses: actions/setup-python@v2
+        with:
+          python-version: 3.9
+      - run: pip install mkdocs-material
+      - run: mkdocs gh-deploy --force

.github/workflows/run-tests.yml

@@ -61,10 +61,10 @@ jobs:
           ${{ runner.os }}-${{ env.cache-name }}-
           ${{ runner.os }}-
 
-    - name: Install Python modules (Capstone, Keystone, Unicorn, Ropper)
+    - name: Install Python Requirements
       run: |
         mkdir -p ${{ steps.pip-cache.outputs.dir }}
-        python3 -m pip install --user --upgrade -r ./requirements.txt
+        python3 -m pip install --user --upgrade -r tests/requirements.txt
 
     - name: Setup GEF
       run: |

.gitignore

@@ -12,3 +12,4 @@ debug.log
 .pytest_cache
 .coverage
 htmlcov
+.benchmarks

README.md

@@ -55,10 +55,10 @@ _Note_: For maintenance simplicity, the unified communities on IRC/Gitter/Slack/
 A few of `GEF` features include:
 
   * **One** single GDB script
-  * Entirely **OS Agnostic**, **NO** dependencies: `GEF` is battery-included and [is installable instantly](https://gef.readthedocs.io/en/master/#setup)
+  * Entirely **OS Agnostic**, **NO** dependencies: `GEF` is battery-included and [is installable instantly](https://hugsy.github.io/gef/master/#setup)
   * **Fast** limiting the number of dependencies and optimizing code to make the commands as fast as possible
-  * Provides [a great variety of commands](https://gef.readthedocs.io/en/master/commands/) to drastically change your experience in GDB.
-  * [**Easily** extensible](https://gef.readthedocs.io/en/master/api/) to create other commands by providing more comprehensible layout to GDB Python API.
+  * Provides [a great variety of commands](https://hugsy.github.io/gef/master/commands/) to drastically change your experience in GDB.
+  * [**Easily** extensible](https://hugsy.github.io/gef/master/api/) to create other commands by providing more comprehensible layout to GDB Python API.
   * Full Python3 support ([Python2 support was dropped](https://github.com/hugsy/gef/releases/tag/2020.03) - see [`gef-legacy`](https://github.com/hugsy/gef-legacy)).
   * Built around an architecture abstraction layer, so all commands work in any GDB-supported architecture such as x86-32/64, ARMv5/6/7, AARCH64, SPARC, MIPS, PowerPC, etc.
   * Suited for real-life apps debugging, exploit development, just as much as CTF
@@ -70,24 +70,24 @@ Or [try it online](https://demo.gef.blah.cat) (user:`gef`/password:`gef-demo`)
 
 ## Documentation ##
 
-Unlike other GDB plugins, GEF has an extensive and up-to-date [documentation](https://gef.readthedocs.io/). Users are recommended to refer to it as it may help them in their attempts to use GEF. In particular, new users should navigate through it (see the [FAQ](https://gef.readthedocs.io/en/master/faq/) for common installation problems), and the problem persists, try to reach out for help on the Discord channel or submit an issue.
+Unlike other GDB plugins, GEF has an extensive and up-to-date [documentation](https://hugsy.github.io/gef/). Users are recommended to refer to it as it may help them in their attempts to use GEF. In particular, new users should navigate through it (see the [FAQ](https://hugsy.github.io/gef/master/faq/) for common installation problems), and the problem persists, try to reach out for help on the Discord channel or submit an issue.
 
 
 ## Current status ##
 
 
 | Documentation |License | Compatibility |
 |:---:|:---:|:---|
-| [![ReadTheDocs](https://readthedocs.org/projects/gef/badge/?version=master)](https://gef.readthedocs.org/en/master/) | [![MIT](https://img.shields.io/packagist/l/doctrine/orm.svg?maxAge=2592000?style=plastic)](https://github.com/hugsy/gef/blob/master/LICENSE) | [![Python 3](https://img.shields.io/badge/Python-3-green.svg)](https://github.com/hugsy/gef/) |
+| [![Docs](https://img.shields.io/badge/Documentation-blue.svg)](https://hugsy.github.io/gef/) | [![MIT](https://img.shields.io/packagist/l/doctrine/orm.svg?maxAge=2592000?style=plastic)](https://github.com/hugsy/gef/blob/master/LICENSE) | [![Python 3](https://img.shields.io/badge/Python-3-green.svg)](https://github.com/hugsy/gef/) |
 
 
 ## Contribute ##
 
-To get involved, refer to the [Contribution documentation](https://gef.readthedocs.io/en/master/#contribution) and the [guidelines](https://github.com/hugsy/gef/blob/dev/.github/CONTRIBUTING.md) to start.
+To get involved, refer to the [Contribution documentation](https://hugsy.github.io/gef/master/#contribution) and the [guidelines](https://github.com/hugsy/gef/blob/dev/.github/CONTRIBUTING.md) to start.
 
 ## Sponsors ##
 
-Another way to contribute to keeping the project alive is by sponsoring it! Check out [the sponsoring documentation](https://gef.readthedocs.io/en/master/#sponsors) for details so you can be part of the list of those [awesome sponsors](https://github.com/sponsors/hugsy).
+Another way to contribute to keeping the project alive is by sponsoring it! Check out [the sponsoring documentation](https://hugsy.github.io/gef/master/#sponsors) for details so you can be part of the list of those [awesome sponsors](https://github.com/sponsors/hugsy).
 
 
 ## Happy Hacking   ##

docs/commands/gef.md

@@ -20,20 +20,13 @@ bytearray                 -- BytearrayCommand: Generate a bytearray to be compar
 
 ### GEF Missing Command
 
-Displays the GEF commands which couldn't be loaded, along with the reason for
-the issue.
+GEF is fully battery-included. However in some rare cases, it is possible that not all commands be loaded. If that's the case the command `gef missing` will detail which command failed to load, along with a (likely) reason. Read the documentation for a solution, or reach out on the Discord.
 
 ```
 gef➤  gef missing
-[*] Command `set-permission` is missing, reason  →  Missing `keystone-engine` package, install with: `pip install keystone-engine`.
-[*] Command `assemble` is missing, reason  →  Missing `keystone-engine` package for Python, install with: `pip install keystone-engine`.
-
-[...snip...]
-
+[*] Command `XXXX` is missing, reason  →  YYYYY.
 ```
 
-As it says in the above output, the issues should be resolved by installing the
-missing package(s) using pip.
 
 ### GEF Config Command