Plugins (#5)

* Initial plugin implementation.

* Code tuning.

* Doc linting and directory management.

* README linting.

* Renamed argument_parsers to parsers

* Documentation linting.

.gitignore

@@ -157,4 +157,6 @@ cython_debug/
 
 # Custom exclusions
 data/
+src/plugins/
+scratch/
 .init/

LICENSE

@@ -2,20 +2,21 @@ MIT License
 
 Copyright 2024 Peter Nardi
 
-Permission is hereby granted, free of charge, to any person obtaining a copy of
-this software and associated documentation files (the "Software"), to deal in
-the Software without restriction, including without limitation the rights to
-use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies
-of the Software, and to permit persons to whom the Software is furnished to do
-so, subject to the following conditions:
+Permission is hereby granted, free of charge, to any person obtaining a
+copy of this software and associated documentation files (the
+"Software"), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
 
-The above copyright notice and this permission notice shall be included in all
-copies or substantial portions of the Software.
+The above copyright notice and this permission notice shall be included
+in all copies or substantial portions of the Software.
 
-THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
-IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
-FITNESS FOR A PARTICULAR PURPOSE AND NON-INFRINGEMENT. IN NO EVENT SHALL THE
-AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
-LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
-OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
-SOFTWARE.
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS
+OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NON-INFRINGEMENT.
+IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
+CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
+TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
+SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

Makefile

@@ -10,6 +10,14 @@ ifeq (,$(wildcard .init/setup))
 	@if [ ! -d "./data" ]; then \
 		mkdir -p data/geolite; \
 	fi
+	@if [ ! -d "./src/plugins/parsers" ]; then \
+		mkdir -p src/plugins/parsers; \
+		touch src/plugins/parsers/__init__.py; \
+	fi
+	@if [ ! -d "./src/plugins/code" ]; then \
+		mkdir -p src/plugins/code; \
+		touch src/plugins/code/__init__.py; \
+	fi
 	mkdir .init
 	touch .init/setup
 	poetry install --only=main
@@ -55,8 +63,8 @@ reset: clean ## remove venv, artifacts, and init directory
 # --------------------------------------------
 
 .PHONY: clean
-clean: ## cleanup python build artifacts
-	@echo Cleaning python build artifacts
+clean: ## cleanup python runtime artifacts
+	@echo Cleaning python runtime artifacts
 	@find . -type d -name __pycache__ -exec rm -rf {} \; -prune
 
 # --------------------------------------------

README.md

@@ -1,4 +1,4 @@
-# banip
+# <a id="top"></a> banip
 
 <img
 src="https://drive.google.com/uc?export=view&id=1H04KVAA3ohH_dLXIrC0bXuJXDn3VutKc"
@@ -26,7 +26,16 @@ from all countries except those that I've whitelisted. I also want the
 ability to create a customized IP list to block any bad actors from
 those whitelisted countries. This tool accomplishes that.
 
-## Requirements
+## Contents
+
+* [Requirements](#requirements)
+* [Setup](#setup)
+* [Running](#running)
+* [Updating](#updating)
+* [Plugins](#plugins)
+* [Uninstalling](#uninstall)
+
+## <a id="requirements"></a> Requirements
 
 ### Operating System
 
@@ -38,7 +47,7 @@ Machine, or [Windows Subsystem for Linux (WSL)][def7] is required.
 You'll need a copy of the [MaxMind][def8] GeoLite2 database for
 country-level geotagging of IP addresses. If you have a premium or
 corporate MaxMind account, you're all set. If not, the free GeoLite2
-account will work just fine ([signup here][def5]). Once you login, on
+account will work just fine ([signup here][def5]). Once you login, using
 the menu on the top right select:
 
 ```text
@@ -75,7 +84,8 @@ own use.
 
 ### Global list of blacklisted IPs
 
-Download the list as follows:
+banip uses the [ipsum][def9] threat intelligence blacklist. You can
+direct download it using:
 
 ```shell
 curl -sL https://raw.githubusercontent.com/stamparm/ipsum/master/ipsum.txt > ipsum.txt
@@ -86,7 +96,9 @@ curl -sL https://raw.githubusercontent.com/stamparm/ipsum/master/ipsum.txt > ips
 You'll need the [make][def6] utility installed (*it probably
 already is*).
 
-## Setup
+[top](#top)
+
+## <a id="setup"></a> Setup
 
 ### Unpack GeoLite2 data
 
@@ -128,7 +140,7 @@ cp <wherever you put it>/ipsum.txt ./data/ipsum.txt
 #### Target countries
 
 ```shell
-cp sample-targets.txt ./data/targets.txt
+cp ./samples/targets.txt ./data/targets.txt
 ```
 
 Modify `./data/targets.txt` to select your desired target countries. The
@@ -137,7 +149,7 @@ comments in the file will guide you.
 #### Custom whitelist (optional)
 
 ```shell
-cp sample-custom_whitelist.txt ./data/custom_whitelist.txt
+cp ./samples/custom_whitelist.txt ./data/custom_whitelist.txt
 ```
 
 There may be IP addresses that banip will flag as malicious, but you
@@ -151,7 +163,7 @@ blank one when you run it.
 #### Custom blacklist (optional)
 
 ```shell
-cp sample-custom_blacklist.txt ./data/custom_blacklist.txt
+cp ./samples/custom_blacklist.txt ./data/custom_blacklist.txt
 ```
 
 The source database of banned IPs isn't perfect. You may determine that
@@ -194,23 +206,80 @@ data
 └── targets.txt (required)
 ```
 
-## Running
+[top](#top)
+
+## <a id="running"></a> Running
+
+After copying/tweaking all the required files, start by activating the
+python virtual environment:
+
+```shell
+source .venv/bin/activate
+```
 
-After copying/tweaking all the required files, start with this command
-to learn how to build your custom blacklist:
+Now run this command to learn how to build your custom blacklist:
 
 ```shell
-poetry run banip -h
+banip -h
 ```
 
-## Updating
+[top](#top)
+
+## <a id="updating"></a> Updating
 
 MaxMind updates the GeoLite2 Country database on Tuesdays and Fridays,
 and the list of blacklisted IPs (`ipsum.txt`) is updated daily. Pull
 updated copies of both and put them in `banip/data/geolite` (for the
 GeoLite2 data) and `banip/data` (for the `ipsum.txt` file). Run `banip`
 again to generate an updated blacklist.
 
+[top](#top)
+
+## <a id="plugins"></a> Plugins
+
+banip generates some data that you may want to use for other purposes.
+For example, everytime you build a new blacklist banip also creates and
+saves a textfile of all worldwide subnets, each tagged with a two-letter
+country code. The file is saved in:
+
+```'text
+./banip/data/haproxy_geo_ip.txt
+```
+
+Next time you run banip, open that file and take a look at it. Since you
+may have a very specific usecase for that data, you can write a plugin
+for banip which will make use of the build products for your purposes.
+
+A banip plugin consists of two required files:
+
+1. Code that generates an argument parser for your new command.
+2. Code that implements the functionality of your new command.
+
+All your plugins go into the `./src/plugins` directory in the
+appropriate subdirectory (either `parsers` or `code`). Your
+plugins are not under version control, so they will only reside on your
+machine.
+
+Look at the comments in these two files for instructions on how to
+create your own plugins:
+
+```text
+./samples/plugins/foo.py
+./samples/plugins/foo_args.py
+```
+
+[top](#top)
+
+## <a id="uninstall"></a> Uninstalling banip
+
+If you want out, just do this:
+
+```shell
+rm -rf ~/.banip
+```
+
+[top](#top)
+
 [def]: https://aws.amazon.com/what-is/cidr/#:~:text=CIDR%20notation%20represents%20an%20IP,as%20192.168.1.0%2F22.
 [def2]: https://python-poetry.org/
 [def3]: https://docs.github.com/en/get-started/getting-started-with-git/ignoring-files
@@ -219,3 +288,4 @@ again to generate an updated blacklist.
 [def4]: https://dev.maxmind.com/geoip/updating-databases#directly-downloading-databases
 [def5]: https://dev.maxmind.com/geoip/geolite2-free-geolocation-data
 [def8]: https://www.maxmind.com/en/home
+[def9]: https://github.com/stamparm/ipsum

pyproject.toml

@@ -1,6 +1,6 @@
 [tool.poetry]
 name = "banip"
-version = "0.1.0"
+version = "1.0.0"
 description = "Create a custom list of band ip for specific countries"
 authors = ["Peter Nardi <pete@nardi.com>"]
 license = "MIT"

samples/plugins/foo.py

@@ -0,0 +1,40 @@
+"""Rules for writing custom commands.
+
+1. Start with this sample command file and modify it.
+2. Make sure to give your file the same name as your command (e.g.
+   foo.py for the "foo" command).
+3. The entry point for your command will be "task_runner(args)", which
+   is defined below. Do not change this function signature. Most of your
+   code will go there, but you may add additional functions/modules as
+   needed. The only rule here is that you must use this pre-defined
+   entry point.
+4. "args" will be a argparse.Namespace variable that contains all the
+   inputs captured on the command line, which are defined by foo_args.py
+5. Your code should return None.
+5. Each newly defined function must have an associated argument parser,
+   defined in {function name}_args.py. See "foo_args.py" in
+   banip/samples/plugins as an example.
+6. Make sure to put this file in banip/src/plugins/code
+"""
+
+import argparse
+
+
+def task_runner(args: argparse.Namespace) -> None:
+    """Do something wonderful.
+
+    This is foo, so by definition it's wonderful!
+
+    Parameters
+    ----------
+    args : argparse.Namespace
+        Arguments passed on the command line.
+    """
+    total = args.first + args.second
+    print(f"The sum of {args.first} and {args.second} is {total}.")
+    print("Cool! Right?")
+    return
+
+
+if __name__ == "__main__":
+    pass

samples/plugins/foo_args.py

@@ -0,0 +1,54 @@
+"""Rules for writing custom argument parsers.
+
+1. Start with this sample argument parser and modify it.
+2. Make the filename for this parser the same as the filename of your
+   command, but with the '_args' suffix (e.g. foo_args.py for the
+   command defined by foo.py).
+3. Do not change the function signature of the "load_command_args"
+   function below.
+4. The variable COMMAND_NAME below is required. It must contain the name
+   of your command.
+5. The sp.add_parser() function call is required. You can add additional
+   arguments to the function call, but the "name", "help", and
+   "description" arguments are required. Modify the "msg" variable to
+   tailor your help and description messages.
+6. Add as many parser.add_argument() calls as you need.
+7. You may add additional functions/modules as needed. For example,
+   adding something like "from argparse import FileType" if one of your
+   command line arguments is a file type.
+8. Your code should return None.
+9. Make sure to put this file in banip/src/plugins/parsers
+"""
+
+from argparse import _SubParsersAction
+
+COMMAND_NAME = "foo"
+
+
+def load_command_args(sp: _SubParsersAction) -> None:
+    """Assemble the argument parser."""
+    msg = """This command takes two intergers on the command line, adds
+    them together, then prints the result. Isn't that wonderful!"""
+    parser = sp.add_parser(
+        name=COMMAND_NAME,
+        help=msg,
+        description=msg,
+    )
+
+    # Add an argument to the parser
+    msg = """The first variable to be added."""
+    parser.add_argument(
+        "first",
+        type=int,
+        help=msg,
+    )
+
+    # Add another argument to the parser
+    msg = """This is the second."""
+    parser.add_argument(
+        "second",
+        type=int,
+        help=msg,
+    )
+
+    return