Skip to content
Go to file

Latest commit


Git stats


Failed to load latest commit information.
Latest commit message
Commit time


This is the documentation for

It is an elasticsearch node that contains a recent dump of the SKS keyserver pool database (I maintain a keyserver in the pool). This pool is what is GPG uses by default to fetch public keys when using gpg --recv-key. The purpose of this elasticsearch project is to let people do data analysis on the keys in the pool.

NOTE: Do not rely on elasticsearch's _id for looking up keys! I will be refreshing the database with a new dump regularly, which will create completely new elasticsearch ids for the keys (it's easier to blow away and recreate than try and find/update based on fingerprint).

Current dump from: Sun Apr 5 05:02:58 UTC 2015

Document format

The keys loaded into the index are based on the output from my script that converts pgp public keys to json.

Quick links

Instructions for making your own

The server this runs on isn't super powerful, so if you want to run some heavy queries, you may want to setup a local copy on your system. Here's how.

Step 1: Setup elasticsearch on your local machine.

Step 2: Download

mkdir ~/opengpg-python
cd ~/openpgp-python
wget >

Step 3: Download the latest SKS keyserver dump (this will take a while, ~7GB).

mkdir ~/dump
cd ~/dump
wget -c -r -p -e robots=off --timestamping --level=1 --cut-dirs=3 \

Step 4: Parse keyserver dump to json gzip files (split every 1000 lines) (this will take several hours, ~16GB).

ls -1 ~/dump/*.pgp | \
xargs -I % sh -c "python ~/openpgp-python/ --merge-public-keys '%' | \
split -l 1000 -d --filter 'gzip -9 > $FILE.gz' - '%.json.'"

Step 5: Bulk index each gzip file into elasticsearch (this will take several hours, ~100GB).

ls -1 ~/dump/*.json.*.gz | \
xargs -I % sh -c "zcat '%' | \
sed '0~1 s/^/{ \"index\" : { \"_index\" : \"keyserver1\", \"_type\" : \"key\" } }\n/' | \
curl -X POST --data-binary @- http://localhost:9200/_bulk | \
{ cat -; echo ''; } >> ~/results.log"


No releases published


No packages published
You can’t perform that action at this time.