Skip to content
A MongoDB compatible embeddable database and toolkit for Go.
Branch: master
Clone or download
Latest commit 1f16cc2 Nov 27, 2019
Type Name Latest commit message Commit time
Failed to load latest commit information.
bsonkit fix bug Nov 27, 2019
dbkit added timeout Nov 1, 2019
mongokit support identity based ordering Nov 27, 2019
.gitignore imporved and tested file store Oct 28, 2019
.travis.yml fix download Nov 4, 2019 added license Nov 4, 2019
Makefile renamed backend to engine Oct 18, 2019 small change Nov 5, 2019
bench_test.go cache collection Oct 29, 2019
catalog.go cleanup handles Nov 1, 2019
client.go cleaned up Nov 6, 2019
client_test.go added list databases Oct 18, 2019
collection_test.go added note Nov 27, 2019
cursor.go cleaned up Nov 6, 2019
database.go return error from transaction begin Nov 5, 2019
database_test.go added drop database Oct 18, 2019
engine.go cleaned up Nov 5, 2019
example_test.go added engine closing Oct 28, 2019
go.mod updated mongo driver Nov 6, 2019
go.sum updated mongo driver Nov 6, 2019
indexes_test.go fixed non-persistent index expiry Nov 26, 2019
lungo.go cleaned up Nov 6, 2019
lungo_test.go prepared session support Oct 31, 2019
mongo.go fix recursive call Nov 1, 2019
result.go added docs Oct 28, 2019
session.go return error from transaction begin Nov 5, 2019
session_test.go check errors Nov 4, 2019
store.go fixed non-persistent index expiry Nov 26, 2019
store_test.go support identity based ordering Nov 27, 2019
stream.go fixed typo Nov 25, 2019
stream_test.go dropped flaky test Nov 4, 2019
transaction.go support identity based ordering Nov 27, 2019
utils.go simplify Nov 5, 2019
utils_test.go added ttl index support Nov 5, 2019



Build Status Coverage Status GoDoc Release Go Report Card

A MongoDB compatible embeddable database and toolkit for Go.


To get started, install the package using the go tool:

$ go get -u


This example shows a basic usage of the mongo compatible API.


The document-oriented database MongoDB has become a widely used data store for many applications developed with the Go programming language. Both, the deprecated mgo and the official mongo driver offer a sophisticated interface to connect to a deployment and ingest and extract data using various commands. While this is enough for most projects, there are situations in which one thinks: "It would be cool if I could just do that in memory without asking the server."

Lungo tries to address this need by re-implementing the data handling mechanics in Go to be used on the client-side. This allows developers to pre- or post-process data in the application relieving the server. For example, applications may utilize this functionality to cache documents and query them quickly in memory.

But we do not need to stop there. Many developers coming from the SQL ecosystem enjoy working with SQLite as a simple alternative to bigger SQL databases. It allows running tests without setting up a database or even small production apps that write their data to a single backed-up file.

Lungo wants to offer a similar experience by implementing a full MongoDB compatible embeddable database that persists data in a single file. The project aims to provide drop-in compatibility with the API exported by the official Go driver. This way, applications may use lungo for running their tests or even low-write production deployments without big code changes.

However, one thing this project does not try to do is build another distributed database. MongoDB itself does a pretty good job at that already.


The codebase is divided into the packages bsonkit, mongokit, dbkit and the main lungo package.

  • The bsonkit package provides building blocks that extend the ones found in the official bson package for handling BSON data. Its functions are mainly useful to applications that need to inspect, compare, convert, transform, clone, access, and manipulate BSON data directly in memory.

  • On top of that, the mongokit package provides the MongoDB data handling algorithms and structures. Specifically, it implements the MongoDB querying, update, and sort algorithms as well as a btree based index for documents. All of that is then bundled as a basic in-memory collection of documents that offers a familiar CRUD interface.

  • The dbkit package provides database-centric utilities e.g. atomic file write.

  • Finally, the lungo package implements the embeddable database and the mongo compatible driver. The heavy work is done by the engine and transaction types that manage access to the basic mongokit.Collection instances. While both can be used standalone, most users want to use the generic driver interface that can be used with MongoDB deployments and lungo engines.


On a high level, lungo provides the following features (unchecked features are planned to be implemented):

  • CRUD, Index Management and Namespace Management
  • Single, Compound and Partial Indexes
  • Index Supported Sorting & Filtering
  • Sessions & Multi-Document Transactions
  • Oplog & Change Streams
  • Aggregation Pipeline
  • Memory & Single File Store

While the goal is to implement all MongoDB features in a compatible way, the architectural difference has implications on some of the features. Furthermore, the goal is to build an open and accessible codebase that favors simplicity. Check out the following sections for details on the implementation.

CRUD, Index Management and Namespace Management

The driver supports all standard CRUD, index management and namespace management methods that are also exposed by the official driver. However, to this date, the driver does not yet support any of the MongoDB commands that can be issued using the Database.RunCommand method. Most unexported commands are related to query planning, replication, sharding, and user and role management features that we do not plan to support. However, we eventually will support some of the administrative and diagnostics commands e.g. renameCollection and explain.

Leveraging the mongokit.Match function, lungo supports the following query operators:

  • $and, $or, $nor, $not
  • $eq, $gt, $lt, $gte, $lte, $ne
  • $in, $nin, $exist, $type
  • $all, $size, $elemMatch

And the mongokit.Apply function currently supports the following update operators:

  • $set, $setOnInsert, $unset, $rename
  • $inc, $mul, $max, $min
  • $currentDate

Finally, the mongokit.Project function currently supports the following projection operators:

  • $slice

Single, Compound and Partial Indexes

The mongokit.Index type supports single field and compound indexes that optionally enforce uniqueness or index a subset of documents using a partial filter expression. Single field indexes also support the automated expiry of documents aka. TTL indexes.

The more advanced multikey, geospatial, text, and hashed indexes are not yet supported and may be added later, while the deprecated sparse indexes will not. The recently introduced collation feature, as well as wildcard indexes, are also subject to future development.

Index Supported Sorting & Filtering

Indexes are currently only used to ensure uniqueness constraints and do not support filtering and sorting. This will be added in the future together with support for the explain command to debug the generated query plan.

Sessions & Multi-Document Transactions

Lungo supports multi-document transactions using a basic copy on write mechanism. Every transaction will make a copy of the catalog and clone namespaces before applying changes. After the new catalog has been written to disk, the transaction is considered successful and the catalog replaced. Read-only transactions are allowed to run in parallel as they only serve as snapshots. But write transactions are run sequentially. We assume write transactions to be fast and therefore try to prevent abortions due to conflicts (pessimistic concurrency control). The chosen approach might be changed in the future.

Oplog & Change Streams

Similar to MongoDB, every CRUD change is also logged to the local.oplog collection in the same format as consumed by change streams in MongoDB. Based on that, change streams can be used in the same way as with MongoDB replica sets.

Memory & Single File Store

The lungo.Store interface enables custom adapters that store the catalog to various mediums. The built-in MemoryStore keeps all data in memory while the FileStore writes all data atomically to a single BSON file. The interface may get more sophisticated in the future to allow more efficient storing methods.


The MIT License (MIT)

Copyright (c) 2019 Joël Gähwiler

You can’t perform that action at this time.