Skip to content
master
Switch branches/tags
Code
This branch is 55 commits ahead, 16 commits behind ayeowch:master.
Contribute

Latest commit

 

Git stats

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time
 
 
 
 
 
 
 
 
 
 
 
 

Ravennodes

This project is based on Bitnodes.

Ravennodes is currently being developed to estimate the size of the Ravencoin network by finding all the reachable nodes in the network. These are the nodes that accept incoming connections. Why you should run a full node is explained here on the Bitcoin wiki. The current methodology involves sending getaddr messages recursively to find all the reachable nodes in the network, starting from a set of seed nodes. It is worth mentioning that this method of estimating network size does not list all full nodes because not all nodes have an open port that can be probed using Ravennodes. These nodes are either behind firewalls or they are configured to not listen for connections.

The goal is to run the service as efficient as possible. The ravennodes crawler, web_updater, and website currently all run on an a1 medium AWS server (1 vCPu, 2GB RAM).

TOC

Ravennodes Crawler

Main Changes

  • Changed parameters to support Ravencoin
  • Turned off Tor network support.

Dependencies

  • Python 2.7 virtual environment
  • Redis

Python Packages

Package Version
dpkt 1.9.1
flake8 3.5.0
geoip2 2.9.0
scipy 0.13.0
gevent 1.3.4
ipaddress 1.0.22
PySocks 1.6.8
pytest 3.7.3
redis 2.10.6
requests 2.20.0

Steps on setting up a machine to run Ravennodes

Ubuntu 18.04:

Install redis

sudo apt update && sudo apt upgrade
cd ~/
sudo apt install redis

# I prefer running redis manually
# Optional, remove the service: sudo service redis-server disable

Install Ravennodes and set up dependencies

#Move redis conf file
sudo cp [LOCATION]/crawler/depends/redis/redis.conf /etc/redis/
#Install Python requirements
cd [LOCATION]/crawler
pip install -r requirements.txt
#Update GeoIP
bash geoip/update.sh

NOTE: For GeoIP, you now need an account at maxmind.com and acquire a license key. Add this key to ~/.bashrc:

export MAXMIND_LICENSE_KEY=YOUR-KEY-HERE

Update open file limits to prevent Ravennodes crashing on IO errors

# Edit the following file:
sudo nano /etc/security/limits.conf
	#Add the following:
	* soft nofile 1000000
	* hard nofile 1000000
 	root soft nofile 1000000
	root hard nofile 1000000
	
# If you run an Ubuntu GUI, Edit the following files too:
sudo nano /etc/systemd/user.conf
	#Change the following setting:
	DefaultLimitNOFILE=1000000
sudo nano /etc/systemd/system.conf 
	#Change the following setting:
	DefaultLimitNOFILE=1000000

Start redis service

#Login as admin
sudo -i
#Update /proc/sys/net/core/somaxconn
sysctl -w net.core.somaxconn=511
#Disable THP
echo never > /sys/kernel/mm/transparent_hugepage/enabled
# Set overcommit_memory to 1
sysctl vm.overcommit_memory=1
#Start Redis Server
redis-server /etc/redis/redis.conf
#Alternatively, run redis in the background with redis-server /etc/redis/redis.conf --daemonize yes

#Make sure the file redis.sock is made in /tmp/

To start the Ravennodes crawler

Open a new console, activate the python 2.7 virtual environment and start the crawler:

cd [LOCATION]/Ravennodes
./start.sh

Data output will be available in ~/Ravennodes/data/export/5241564e

Process logs will be available in ~/Ravennodes/log

Ravennodes Web-updater

  • Data change watcher adaptation from Michael Cho looking for crawler updates
  • Data updater generating ravencoin nodes tables and world maps for the website

Dependencies

  • Python 3.7 virtual environment

Python Packages

Package Version
pandas 0.24.1
watchdog 0.9.0
numpy 1.16.1
plotly 3.6.1

To start the Ravennodes Web-updater

Open a new console, activate the python 3.7 virtual environment and start the updater:

cd [LOCATION]/web_updater
./start.sh

Ravennodes website

  • Website built using flask

Dependencies

  • Python 3.7 virtual environment

Python Packages

Package Version
Flask 1.0.2
gunicorn 19.9.0
Jinja2 2.10
pandas 0.24.1
Werkzeug 0.14.1

Website hosting

Website is hosted according to this guide

Call for improvements

Everyone is welcome to propose changes. Currently, the whole system is set up using python for self-educational purposes. Using an SQL type database, for example, will drastically improve performance.


-Jeroz

https://github.com/jeroz1/Ravennodes https://github.com/RavenCommunity/Ravennodes

About

Ravennodes estimates the size of the Ravencoin network by finding all the reachable nodes.

Resources

Releases

No releases published

Packages

No packages published