Modules

Scott Lott edited this page Mar 24, 2017 · 146 revisions

Modules

Complete databases

couchup

A CouchDB implementation on top of LevelUP.

dat

Dat is a built on top of LevelDB and lets you build streaming data pipelines that can be shared and replicated by others.

firedup

A node.js implementation of firebase based on leveldb/levelup

len

Len is a resource booking database using LevelDB for storage. Useful for calendar and gantt chart apps and for questions like 'can a customer book this resource starting X and ending Y'.

lem

Lem is a telemetry storage database using LevelDB. Keys are indexed by timestamp and you can read values in-between 2 points in time.

LevelGraph

A Graph database built on top of LevelUP with pattern-matching and join support.

LinvoDB

Persistent database on top of LevelUp for Node.js/NW.js with MongoDB-style queries, Mongoose-like models and a map/reduce system.

PlumbDB

HTTP request handlers for building web services on top of LevelDB. (Not in active development)

PouchDB

PouchDB is a JavaScript library that allows you to store and query data offline, and then sync with CouchDB when online. Uses LevelUP when run in Node, and can use any *DOWN adapter in the browser, in Node, or as its own server.

pushdb

A programmable database with document storage and unique indexing capabilities.

search-index

A persistent full text search engine for the browser and Node.js

tacodb

A responsive, Node.js-style database ideal for realtime data. Highly modular and adaptable, allowing extension with the level* module ecosystem.

timestreamdb

A full-featured timeseries database on top of LevelDB. Includes a library for streaming statistical operations on timeseries data including joins, aggregates, filters, and map-like operations.

NanoSQL

A small (10Kb) RDBMS abstraction on top of LevelUP that also runs in the browser with undo/redo support, immutable objects and IndexedDB persistence.

ORMs

Dulcimer

Dulcimer allows you define JSON models and then helps you manage indexes, children, foreign keys, and much more.

skeyma

skeyma turns a ES6-like template string (like ${forumId}/${postId}) into parse/serialize stream transforms that convert objects like {forumId, postId, text} into KV objects like {key, text} and back.

Storage back-ends

asyncstorage-down

An implementation of the LevelDOWN API (passes the Abstract LevelDOWN tests suite) that utilizes AsyncStorage from React Native. Can be used as an alternative backend to LevelUP.

azureleveldown

An implementation of the LevelDOWN API (passes the Abstract LevelDOWN tests suite) using Windows Azure Table Storage.

cachedown

An implementation of the LevelDOWN API that wraps another leveldown instance with an lru-cache for faster gets/puts

dynamo-down

An implementation of the LevelDOWN API using AWS's DynamoDB as the backend.

FruitDOWN

Yet another IndexedDB LevelUP implementation, but optimized to run in every IndexedDB version, including Apple's.

indexedup

Another implementation of the LevelUP API sitting in front of IndexedDB in browsers.

jsondown

A drop-in replacement for LevelDOWN that writes to a JSON file on disk. It also retains the contents of the entire JSON file in memory, so it's only really useful for debugging purposes and/or very small data stores that need just a pinch of persistence.

level.js

An implementation of the LevelDOWN API (passes the Abstract LevelDOWN tests suite) sitting in front of IndexedDB in browsers. Can be used as an alternative backend to LevelUP.

LevelDOWN

A pure C++ LevelDB binding, exposing raw LevelDB operations. The default storage engine installed when you npm install level. Also the default engine that LevelUP looks for when you don't provide a 'db' option.

leveldown-basho

A LevelDOWN-compatible Node.js binding for Basho's LevelDB fork (which is used in Riak).

LevelUP

Designed to allow swappable storage back-ends using the 'db' option.

localstorage-down

An implementation of the LevelDOWN API (passes the Abstract LevelDOWN tests suite) that utilizes localStorage. Supports Android, iOS, and Blackberry via PhoneGap. Can be used as an alternative backend to LevelUP.

MedeaDOWN

An implementation of the LevelDOWN API (passes the Abstract LevelDOWN tests suite) using medea as the backend.

MemDOWN

An implementation of the LevelDOWN API (passes the Abstract LevelDOWN tests suite) using a pure-memory backend. Very fast, but limited storage. Can be used as an alternative backend to LevelUP.

MongoDOWN

A drop-in replacement for LevelDOWN that runs on MongoDB.

MysqlDOWN

An implementation of the LevelDOWN API (passes the Abstract LevelDOWN tests suite) using a mysql backend.

RedisDOWN

An implementation of the LevelDOWN API (passes the Abstract LevelDOWN tests suite) using a redis backend.

RiakDOWN

An implementation of the LevelDOWN API (passes the Abstract LevelDOWN tests suite) using a riakpbc backend.

rocksdb

A Node.js RocksDB binding, LevelDOWN-compatible.

sheet-down

The LevelDOWN API over Google Sheets.

SQLdown

An implementation of the LevelDOWN API (passes the Abstract LevelDOWN tests suite) that can use sqlite3, postgres, mysql, or websql as a backend, uses knex.

Plugins

Plugins extend LevelUP in some way, providing additional functionality beyond that provided by the core.

byteup

Adds the awesome bytewise sortable javascript type encoding as a native encoding for LevelUP.

bytewise

A binary serialization which sorts bytewise for arbitrarily complex data structures, respecting typewise structured sorting efficiently.

changeset

Library to diff JSON objects into atomic put and delete operations, and apply change sets to objects. Useful with Levelup/LevelDB object synchronization.

jsonquery

Query your levelup instances with node.js streams with the mongodb query API.

myJSONStream()
.pipe(jsonquery({ val: { $and: [ { $gt: 970 }, { $gt: 950 } ] } })
.on('data', console.log);

jsonquery-engine

A full MongoDB query language implementation with INDEXES for querying your levelup/leveldb database.

// uses indexes
db.ensureIndex('tags');
db.ensureIndex('num');
db.query({ $and: [ { tags: 'tag1' }, { num: { $lt: 100 } } ] }); // results stream

Level TTL

Add a 'ttl' (time-to-live) option to LevelUP for put() and batch(). Entries will be automatically cleared after the specified number of milliseconds.

See also [Tiny Level TTL](#Tiny Level TTL).

level-atomics

Adds typical atomic operations, like insert, replace and counter (increment/decrement) to LevelDB, and adds capacity to multiple parallel operations, like multi gets, multi inserts, and so on.

level-autotable

Auto incrementing keys. Read, write, delete fields in a record individually. Read, write, delete records.

level-cache

A caching module you can place in front of a levelup database. It will cache a subset of the database in an in-memory LRU cache based on configuration. It has an optional synchronous API which will return from the cache only.

level-cache also doubles as in memory implementation of the LevelUP interface so that it can be used in unit tests or other situations where you want mocks.

level-capped

Capped collections.

level-cbatch

Provides a chainable API for db.batch()

db.cbatch()
    .put('foo', 'bar')
    .del('hello')
    .exec(callback)

Note: this functionality is now provided by LevelUP core*

level-delete-range

Delete a continuous range of keys on a LevelDB.

deleteRange(db, {
    start: "foo:"
    , end: "foo;"
}, cb)

level-hooks

Implements a hook mechanism that allows you to intercept put, delete and batch operations on a db. You can then turn those operations into batches.

Useful if you want to turn a put into an atomic batch for say an automatic map operation.

It is recommended that you use use level-sublevel instead of level-hooks directly

level-indico

Yet another indexing plugin for your LevelDB. By providing only a low-level querying mechanism it gives you the power to build more complicated and optimized queries on top of it.

level-inverted-index

Create an inverted index for full-text search.

level-livefeed

A live query of a range in LevelDB. Returns a streams2 stream

var stream = livefeed(db, { start: "foo:", end: "foo;" })

level-live-stream

Like db.createReadStream() except it's live / tailable. i.e. instead of ending, it will stay open and stream changes to the database as they are inserted.

level-map-merge

Like a map-reduce but simpler. Has a batch component that runs periodically, and a real-time component that fills in the gaps. Good for generating inverted indexes.

level-mapped-index

Indexes for LevelUP built on map-reduce. Uses a custom indexing function for each index to parse and record index values for each entry. Provides getBy(index, value, function (err, data) {}) and createIndexedStream(index, value) methods on your LevelUP instance which return lists of the original entries indexed for those index/value combinations.

Level-Multiply

Overload LevelUP's get(), put() and del() so they also accept multiple keys & values:

  db.put({ foo: 'bar', boom: 'bang', whoa: 'dude' }, function (err) { /* .. */ })

  db.get([ 'foo', 'boom', 'whoa', 'wha' ], function (err, data) { /* .. */ })

  db.del([ 'foo', 'boom', 'whoa', 'wha' ], function (err) { /* .. */ })

level-namespace

Split your db up into multiple namespaces. Returns a LevelUP style db object put every key is automatically namespaced.

level-push

Saves JSON to your LevelDB using level-set with auto-generated UUID.

level-queryengine

A generic pluggable query-engine system (that supports indexes) for levelup/leveldb databases.

level-range

Find all K/V-pairs prefixed by a certain key.

range(db, 'bucket:%s:', 'bucket-name').pipe(...)

level-schedule

A durable job scheduler.

level-scuttlebutt

Persists and query scuttlebutt documents in LevelDB.

level-set

Saves JSON to your LevelDB. Converting your JSON to LevelDB key-value pair data structure.

level-static

Use LevelDB as a static file server.

level-stay

An alternative approach to storing scuttlebutts in LevelDB.

level-store

A streaming storage engine based on LevelDB.

level-subkey

The level-subkey use the path to separate sections of levelup, with hooks! these sublevels are called dynamic subkey. The level-subkey is modified from level-sublevel.

var db = require('level-subkey')(require('levelup')('/tmp/db'))
var stuff = db.subkey('stuff') //sublevel() function is deprecated, use subkey() instead.
var animal = stuff.subkey('animal')
var plant = stuff.subkey('plant')
animal.put("pig", value, function () {})
db.put("/stuff/animal/pig", value, function(err){}) //=animal.put("pig",...
animal.put("../plant/cucumber", value, function (err) {}) //=plant.put("cucumber",

level-sublevel

Adds the ability to create subsections that present the same API as LevelUP, but only write/read to a prefixed section, or bucket, of the key-space. Each section also has level-hooks installed.

level-superlevel

Sublevel partitions the database. Superlevel adds a super level to the root partition that allows accessing the entire database. This allows things like discovering sublevels and browsing the database content without knowledge of the sublevel structure.

level-trigger

Triggers for LevelUP. Runs an async job when a key changes. All jobs will eventually run, even across restarts!

level-updater

Update keys without overlapping changes - makes it possible to implement an "atomic" incrementer, JSON merger, etc.

level-version

Automatically versioned data on put. Either specify the version going in or allow it to be timestamped. By default operations act on the most recent data, or you can stream the history of a single key.

levelplus

Adds atomic updates, increments, array pushes, set additions, and user-defined atomic operations to LevelUP.

levels

levels is a light-weight LevelDB full text search for node.js (Port of TJ's reds redis search engine to levelup)

map-reduce

A map reduce implementation on top of LevelDB. Allows you to define a map reduce query that will run on top of your db.

The map reduces are incremental, and you can query the results in real-time

map-reduce-chained

Extends map-reduce and level-mapped-index to provide easy to setup chained map reduce. An example use case is to find the top 10 values after a reduce.

msgpackup

Adds the msgpack more efficient JSON serialization format as a native encoding for LevelUP.

path-engine

Query your levelup/leveldb engine using a javascript property path array syntax with INDEXES.

db.ensureIndex('car.make');
db.query(['car', 'make', 'Toyota']); // results stream

q-level

levelup with Q promises

db.batch([
    { type: 'put', key: 'a', value: 1 },
    { type: 'put', key: 'b', value: 2 },
    { type: 'put', key: 'c', value: 3 }
  ])
  .then(function() {
    return db.createReadStream()
  })
  .progress(function(data) {
    console.log(data)
  })
  .then(function() {
    return db.close()
  })

subindex

Generic pluggable indexing system for leveldb/levelup.

db.ensureIndex('car.make');
db.getBy('car.make', 'Toyota');
db.dropIndex('car.make');

Tiny Level TTL

A levelup plugin that enforces a time to live (TTL) on a given (sub)database. Supports level-sublevel 5 & 6, and level-spaces. Also respects level-lock locks.

See also [Level TTL](#Level TTL).

Replication

level-2pc

A two-phase commit protocol.

level-couch-sync

Replicate from CouchDB to LevelDB.

level-master

Master-Slave replication for LevelUP.

level-merkle

Uses merkle-trees to replicate data sets. Data must be sets, and currently, deletes are not supported.

level-replicate

master-master replication with levelup. Implements scuttlebutt style handshake and then syncs data, then replicates real time changes.

level-replicator

master-master replication, same goal as level-replicate but different approach.

Modules

Modules use LevelUP in some way, or provide useful external functionality, rather than specifically extending it.

cursor

stream cursor to iterate through a ReadStream / KeyStream / ValueStream. e.g. db.createReadStream().pipe(cursor.all(function (e, data) {}))

gun-level

Sync graph-style data real-time between browsers and servers.

inverted-index

Inverted index built upon LevelUP

level-array

The array datatype inside leveldb.

level-average

Calculate rolling averages in a LevelDB.

level-co

LevelUP wrappers for co.

level-delete-stream

A deleteStream for LevelUp.

level-encoding

Implements the encoding logic of a LevelUP-style database interface.

level-exists

Check if a datum exists without reading its value.

level-fs

Node's fs module with leveldb as backend

level-geospatial

Store key values pairs with lat/lon coordinates, and query using a radius.

level-index-update

A module to remove the old indexes in the same batch as the new ones are inserted

level-jobs

Job Queue in LevelDB for Node.js.

level-list

Map lists of data stored in a LevelDB to DOM elements.

level-live-cache

An in-memory cache that keeps up to date with its source.

level-lru-cache

Simple LRU cache

level-methods

Useful meta information about levelup methods.

level-mirror

Mirror and optionally transform data from one sublevel into another.

level-move

Move a value to another key, inside a LevelDB.

level-nearby-stream

Stream in nearby places using the browser's geolocation and level-places.

level-object

Store objects in leveldb.

level-paginate

Streaming pagination for leveldb.

level-party

Open a leveldb handle multiple times.

level-path-index

Index properties of items that live in a tree of materialized paths

level-places

Store and retrieve places near a lat/long pair.

level-queue-type

The queue datatype inside leveldb.

level-reactive

Reactive templating for data stored in a LevelDB.

level-sec

High-level API for creating secondary indexes.

level-secondary

Create and query secondary indexes.

level-serve

Streaming static file server based on LevelDB.

level-session

Framework-agnostic, LevelDB-backed web server session manager.

level-socket

Backend server that exposes a level over authenticated cross domain websockets

level-sql

An SQL front end for level-* databases.

level-sublevel-stream

Find all the sublevels of a given database, not requiring them to be in memory already.

level-subtree

Generate a tree from the sublevels of a leveldb, useful when there is no manifest.

level-sum

Calculate sums in a LevelDB and get live updates.

level-tree

Geospatial indexing for GeoJSON in LevelDB

level-trie

The TRIE data structure and search algorithm, on top of leveldb.

level-user

Client side library for authenticating with and moving data over a level-socket

levelup-iterator

Use async iterators instead of readable streams to traverse the database.

LevelTTLCache

A pass-through cache for arbitrary objects or binary data using LevelDB, expired by a TTL.

MemDB

LevelUp + MemDown.

multilevel

Expose a LevelDB over the network.

multilevel-http

Access a LevelDB instance from multiple processes via HTTP.

pull-level

pull-stream interface to LevelDB. (implement read streams, write streams, and realtime (tail/live) reads)

range-bucket

Small module to define a range of keys to use a LevelDB keys.

range-index

Range indexes for LevelDB

string-range

Manipulate string ranges for db.createReadStream()

suckit

Stream data into LevelDB via level-store over HTTP.

write-streams

level-batch-stream

A stream interface for executing batches into any levelup instance passed in. Expects a stream of arrays or single of batch objects (e.g { type: put, key: 'hello', value: 'world' }). Best when used in conjunction with batch-stream or byte-stream for tweaking batch size and concurrent-writable to tweak the concurrent writes to the database.

level-write-stream

The db.createWriteStream() interface factored out into a separate module. Only requires db.put() and db.batch(). Useful to implement a LevelUP style interface or with LevelDOWN

level-writestream

Streams2-compliant write stream for LevelUP.

level-ws

level-ws was extracted from the core LevelUP at version 0.18.0 and provides the most basic general-case WriteStream for LevelUP.

Tools

lev

A CLI REPL interface for LevelDB. Useful for exploring your LevelDB stores.

level-benchmarks

Run benchmarks against levelup-compatible engines

leveldb-repl

Super simple REPL for LevelDB. Supports filter globbing.

levelhud

LevelDB GUI with an interactive console.

levelweb

A LevelDB GUI. Includes simple data visualization tools.