Databases
This document decribes the following:
- The various databases used for each component in the Storj network.
- How to view the contents of the various databases.
- Steps on how to modify existing databases.
Each component of the Storj network (i.e. satellites, storage nodes, uplinks) has different persistent data stores.
Satellites and Storage Nodes both have a single Database interface, referred to as the master DB. The master DB specifies which databases are needed, it doesn't specify how they are implemented. This means different satellites can use different backends.
Satellites have two databases behind the master DB interface.
The three databases are:
- Satellite.DB
- PointerDB
Storage Nodes have three databases behind the master DB interface.
The databases include:
- pieces
- infodb
- psdb
See below for more details about each database. Note: Kademlia has been deprecated.
Satellites store the most data of all the components. Satellites have three databases behind one interface. The interface is called the master DB. The master database specifies which databases are needed, it doesn't specify how they are implemented. This means different satellites can end up with different backends.
Satellites have two databases:
- Satellite.DB.
- PointerDB.
The Satellite.DB is a SQL relational database. Currently supported are SQLite3 and PostgreSQL.
There are a number of different tables in the Satellite.DB including the following:
- Users
- Accounting
- Nodes and Overlay Cache Nodes
- Projects and ApiKeys
- and more
References:
PointerDB is a key/value store. Currently supported are BoltDB and Postgresql k/v store. PointerDB stores information about pieces and which storage nodes they are stored on.
Keys in PointerDB are a path name (Storj.Path object) in the format "projectName/segment/bucketName".
Values in PointerDB are Pointer objects. Pointer objects store metadata about where pieces are stored on which nodes.
Reference:
- Source code for Pointer objects.
- BoltDB documentation.
Storage Nodes have three databases behind a single master DB interface.
The databases include:
- pieces
- infodb
- psdb
pieces is an interface to disk blob storage. pieces saves the actual pieces.
A SQL database that contains six tables. The tables contain information on:
- OrderArchive and UnsentOrders: stores sent and unsent orders to track what still needs to be sent to the Satellite.
- PieceInfo: stores piece meta info, tracking what pieces are currently stored.
- Cert: stores uplink and satellite certificates.
- Bandwidth: stores bandwidth usage info, tracking how much bandwidth has been transferred.
- UserSerials: stores serial numbers that have already been used.
Is a SQL database that stores bandwidth agreements.
Uplinks do not have any persistent storage.
To view the contents of a sqlite3 database file, you can use the command sqlite3. This comes installed by default on OSX. See documentation on for details SQLite CLI.
$ sqlite3 path/to/db/file
To view the contents of a BoltDB database file, there are a number of BoltDB browser tools. See bolt browser or bolter as options.
$ boltbrowser path/to/boltdb/file
To view the contents of a PostgreSQL database, use the psql command line program. See psql docs for details.
$ psql -h hostname -U user