A key value store with some smarts
Switch branches/tags
Nothing to show
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Failed to load latest commit information.


Lucien Key-Value Store

Lucien is a proof of concept key value store.

I wanted to try out the idea of a coprocessor (i.e. stored procedures) to a key value model.

Many times, interacting with a kv-store (i.e. NoSQL) has a lot of latency. If you wanted to grab a post with all it's comments, a lot of requests could occur to display a view. If the database was smart enough to get all the data you need to display the page in one request, there would be a lot less latency.

The idea of having the database doing the work is often seen as a bad idea for scalability. This may be true and something to consider.

This implementation is terrible, and should not be used in production, it is only a proof of concept to see what this model could look like.


A bucket is a namespace of data, a key/value pair only belongs to one bucket. A keys are unique in a bucket but not necessarily duplicated across buckets.

Keys are always strings, values are always JSON objects.

client.put("someBucket", "some_key", "{'a':'value'}")

client.get("someBucket", "some_key")

Backing Store

The backing store at this time is Google's LevelDB which I've used in a few projects with success. The backing store is plug-able and there is nothing about this design that couldn't be done with another backing store.


This is the cool part, you can write a script that you can run later to collect data and send results.

Write more here...


There is a sample python client available, the example below shows some usage.

It uses thrift so many languages can be used.


Here's an example of what Lucien can currently do:

# Simple put
client.put("hey","there", "{'hey':'there'}")

# Simple get
print client.get("hey","there")

# Basic Script
script = '''result = {"success": True, "answer" : 42}'''
v = {"engine":"python", "script":script}

# Check that it's in the database
print client.get("_scripts","the_ultimate_method")

# Check run the method with no arguments
print client.run("the_ultimate_method", [])

# Let's do a more advanced query
post = {
    "title" : "Some entry",
    "content": "It's really cool",
    "comments" : ["comment_1","comment_2"]

comment_1 = {
    "comment": "This is an insightful comment!"

comment_2 = {
    "comment": "This is a boring comment!"

bob = {"name":"Bob The Guy", "email":"bob@gmail.com"}

dave = {"name":"David King", "email":"dave@gmail.com"}

script = """
posts = buckets.getBucket("posts")
comments = buckets.getBucket("comments")
users = buckets.getBucket("users")

post_1 = dict(posts.get("post_1"))

result = post_1

comment_list = list()
for comment_key in post_1['comments']:
    comment = dict(comments.get(comment_key))
    user_key = comment['user_key']
    user = dict(users.get(user_key))


result['comments'] = comment_list
v = {"engine":"python", "script":script}

# Check run the method with the post as an argument
print client.run("render_post", ["post_1"])

Built On

Lucien is built on:

It is Apache Licensed.


If you have any questions or comments, contact me at stephen.holiday@gmail.com