This tool will create a customized list of IP addresses that are cross-referenced between two sources:
- A global (worldwide) list of identified blacklisted IPs.
- A list of the IP subnets associated with each country.
The result is a customized blacklist of IP addresses based on countries that you select.
You could, but where's the fun in that?
You may want to create a list of bad actors for specific countries. The global list contains several hundred thousand entries, and you may need more targeted list for testing or deployment in production.
For example, I've configured my HAProxy server to drop IP connections from all countries except those that I've whitelisted. I also want the ability to create a customized IP list to block any bad actors from those whitelisted countries. This tool accomplishes that.
banip runs in Unix-like OSes. Either macOS, a Linux PC, Linux Virtual Machine, or Windows Subsystem for Linux (WSL) is required.
You'll need a copy of the MaxMind GeoLite2 database for country-level geotagging of IP addresses. If you have a premium or corporate MaxMind account, you're all set. If not, the free GeoLite2 account will work just fine (sign up here). Once you login, using the menu on the top right select:
My Account > My Account
From there, click on Download Files on the bottom left. The file you
want to download is:
GeoLite2 Country: CSV format
banip requires uv for dependency management. uv is well behaved and extremely fast. If you're a Python developer you should check it out. Visit the uv site and install it using your preferred method, with the instructions for your operating system.
If you want to fork and develop this repo, I've included a file called
global-gitignore.txt which is a copy of the .gitignore I placed in
my home directory and configured globally for all my development
projects. The global-gitignore.txt file reflects my development setup
(for example using tools like vscode), but yours may be different. Just
cherrypick any necessary elements from global-gitignore.txt for your
own use.
Details on gitignore files are available on GitHub.
banip uses the ipsum threat intelligence blacklist. You can direct download it using:
curl -sL https://raw.githubusercontent.com/stamparm/ipsum/master/ipsum.txt > ipsum.txtYou'll need the make utility installed (it probably already is).
Unpack the GeoLite2-Country zip archive and save the files to a location you can easily get to.
Note: if you're looking for a quick way to download the MaxMind data
using curl and a direct download permalink, SEE HERE.
Clone this repository. We'll assume you clone it to your home directory
(~):
git clone https://github.com/geozeke/banip.gitChange to ~/banip and run this command:
make setupcp <wherever you put them>/* ./data/geolite/cp <wherever you put it>/ipsum.txt ./data/ipsum.txtcp ./samples/targets.txt ./data/targets.txtModify ./data/targets.txt to select your desired target countries. The
comments in the file will guide you.
cp ./samples/custom_whitelist.txt ./data/custom_whitelist.txtThere may be IP addresses that banip will flag as malicious, but you still want to whitelist them (for example to use for testing). This file will contain specific IP addresses, one per line, that you want to allow.
This file is optional. If you choose not to use it, banip will create a blank one when you run it.
cp ./samples/custom_blacklist.txt ./data/custom_blacklist.txtThe source database of banned IPs isn't perfect. You may determine that
there's an IP address you want to ban this is not found in ipsum.txt.
Also, the ipsum.txt file only contains IP addresses, and you may want
to ban an entire subnet. The custom blacklist allows you to capture
specific IP addresses or subnets (in CIDR format), one per line,
that you want to block. Some of your custom blacklist IPs may be found
when you run the banip, so this file (custom_blacklist.txt) will be
overwritten to remove the duplicates. The contents of the de-duplicated
file will then be appended to the list generated when you run the
program.
This file is optional. If you choose not to use it, banip will create a blank one when you run it.
Note: If you're concerned about keeping your original list of custom blacklisted IPs, save a copy of it somewhere outside the repository.
When you're done, the ~/banip/data directory should look like this:
data
├── custom_blacklist.txt (optional)
├── custom_whitelist.txt (optional)
├── geolite (required)
│ ├── COPYRIGHT.txt
│ ├── GeoLite2-Country-Blocks-IPv4.csv
│ ├── GeoLite2-Country-Blocks-IPv6.csv
│ ├── GeoLite2-Country-Locations-de.csv
│ ├── GeoLite2-Country-Locations-en.csv
│ ├── GeoLite2-Country-Locations-es.csv
│ ├── GeoLite2-Country-Locations-fr.csv
│ ├── GeoLite2-Country-Locations-ja.csv
│ ├── GeoLite2-Country-Locations-pt-BR.csv
│ ├── GeoLite2-Country-Locations-ru.csv
│ ├── GeoLite2-Country-Locations-zh-CN.csv
│ └── LICENSE.txt
├── ipsum.txt (required)
└── targets.txt (required)
After copying/tweaking all the required files, start by activating the python virtual environment:
source .venv/bin/activateNow run this command to learn how to build your custom blacklist:
banip -hMaxMind updates the GeoLite2 Country database on Tuesdays and Fridays,
and the list of blacklisted IPs (ipsum.txt) is updated daily. Pull
updated copies of both and put them in banip/data/geolite (for the
GeoLite2 data) and banip/data (for the ipsum.txt file). Run banip
again to generate an updated blacklist.
banip generates some data that you may want to use for other purposes. For example, everytime you build a new blacklist banip also creates and saves a textfile of all worldwide subnets, each tagged with a two-letter country code. The file is saved in:
./banip/data/haproxy_geo_ip.txt
Next time you run banip, open that file and take a look at it. Since you may have a very specific usecase for that data, you can write a plugin for banip which will make use of the build products for your purposes.
A banip plugin consists of two required files:
- Code that generates an argument parser for your new command.
- Code that implements the functionality of your new command.
All your plugins go into the ./src/plugins directory in the
appropriate subdirectory (either parsers or code). Your
plugins are not under version control, so they will only reside on your
machine.
Look at the comments in these two files for instructions on how to create your own plugins:
./samples/plugins/foo.py
./samples/plugins/foo_args.py
If you want out, just do this:
rm -rf ~/.banip