Crawler to map out all cryptocurrency nodes based on the Bitcoin protocol
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Type Name Latest commit message Commit time
Failed to load latest commit information.

Open Nodes

Open Nodes is a crawler that attempts to map out all nodes of crypto currencies based on the bitcoin protocol.

A flask web server is included to display the data.


# Install python virtual environment
apt install python3-pip python3-venv
python3 -m venv venv
source venv/bin/activate

# Install packages (Python 3)
# psycopg2-binary is required for postgres support
# uwsgi is required for nginx/apache deployment
pip install -r requirements.txt

# setup geoip database
cd geoip && ./ && cd ..

# run crawler
python --seed --crawl --dump

# run development flask server

The --seed parameter is only needed for the first run or when adding a new network. It will hit all the DNS seeds specified in the config file, as well as all individual seeder nodes (if applicable)

The --crawl parameter iterates through all known nodes and stores them in the specified database

The --dump parameter writes all data to disk in json, csv, and txt format for ingestion by the webserver

IPv6 Nodes will only be reachable if you have IPv6 Routing available. To set up IPv6 routing on an AWS deployment see here

Onion Nodes will only be reachable if you have a Tor server running (apt install tor)


The crawler is best run via cron jobs, --dump instances should be scheduled separately from --crawl jobs.

flock should be used to prevent multiple instances from running concurrently

For production use, the default database (sqlite) should not be used, as the file lock timeout will prevent simultaneous crawling, dumping, and reporting/api calls.

NGINX deployment and flask.ini are included for nginx deployment. You will need to pip install uwsgi and then set up a system service (see

nano /etc/systemd/system/opennodes.service

and add

Description=OpenNodes uWSGI instance

User= {{ USERNAME }}
WorkingDirectory=/home/{{ PROJECT ROOT }}
Environment="PATH=/home/{{ PATH TO PYTHON/VENV BIN DIR }}"
ExecStart=/home/{{ PATH TO PYTHON/VENV BIN DIR }}/uwsgi --ini flask.ini


Now you need to add nginx configuration to handle proxy requests

nano /etc/nginx/sites-available/opennodes

and add

server {
    listen 80;
    server_name {{ SERVER DOMAIN OR IP }};
    location / {
        include uwsgi_params;
        uwsgi_pass unix:///home/{{ PROJECT ROOT }}/opennodes.sock;

Where the domain/ip is likely

Then add a symbolic link, remove the default configuration, and restart nginx

ln -s /etc/nginx/sites-available/opennodes /etc/nginx/sites-enabled
rm /etc/nginx/sites-enabled/default
systemctl restart nginx