git for structured data
Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
chez
doc
guile
.dir-locals.el
.gitignore
README.md

README.md

neon -- a functional database

This is a work in progress, it's not useable!

What?

The goal of this project is to build a functional database. That is an immutable database.

If you are asking yourself how would be useful a database that can not change? Then you mis-understand what immutability is and what immutable databases are. Immutable in this context doesn't mean that the database doesn't change, it means that the data is never overwritten ie. that old data is always reachable. In other words, in an immutable database, data has an history.

Why?

The main use case is to be the git of data. That is, use neon to store structured or semi-structured data that can be bigger than RAM. And enable the same cooperative practices that you see in software development around data. The goal is to foster reproducible science.

In particular, neon aims to be able to store conceptnet (10G) and wikidata (100G) and provide the tools to cooperate around those projects.

A secondary goal is to use neon to build small web applications.

How?

neon is implemented in GNU Guile on top of wiredtiger storage engine.

When?

I aim for a stable release at the end of 2018.

Who?

Say hello.

Documentation

neon is a persistent set storing quads ie. 4-tuples made of:

(graph, subject, predicate, object)

It's a set, it means each quad is unique given a version of the database. When you interact with the database, you always work in a defined branch, a version. A branch is described by a direct-acyclic-graph (DAG) of transactions. When you do a read-only query, the transaction is fixed. When you want to do the equivalent of a commit in git, you create a new read-and-write transaction that has a given transaction as parent and its own transaction identifier. And if you do a merge, it's similar but you have two parents. One difference with git, is that neon history is made of transactions instead of commits. neon's transactions map one-to-one with backend storage transactions, so there is no ambiguity.