Available on NuGet as Spreads.LMDB.
LMDB's supported "normal" case is when a transaction is executed from a single thread. For .NET this means that if all operations on a transactions are called from a single thread it doesn't matter which thread is executing a transaction and LMDB will just work.
In some cases one my need background execution of write transactions or .NET async operations inside LMDB transactions. For this case Spreads.LMDB
fully supports async/await. Write transactions are executed in a single thread via a blocking concurrent queue. Read transactions could be used from async code, which requires forcing
attribute for environments:
A thread may use parallel read-only transactions. A read-only transaction may span threads if the user synchronizes its use. Applications that multiplex many user threads over individual OS threads need this option. Such an application must also serialize the write transactions in an OS thread, since LMDB's write locking is unaware of the user threads.
Async support is enabled by default, but could be switched off
LMDBEnvironment.Create(..., disableAsync: true); if not used.
Read-only transaction and cursor renewal
Spreads.LMDB automatically takes care of read-only transaction and cursor renewals if they are properly disposed as .NET objects. It does not allocate those objects in steady state (uses internal pools).
Working with memory safely
Warning! This library exposes
MDB_val directly as
DirectBuffer struct, the struct MUST ONLY be read when inside a transaction
(or when it points to an overflow page - but that is an undocumented hack working so far). For writes,
the memory behind
DirectBuffer MUST BE pinned.
DirectBuffer.Span property allows to access
DirectBuffer can be easily constructed from
but the span must be pinned as well if it is backed by
DirectBuffer has many methods
to read/write primitive and generic blittable struct values from any offset,
directBufferInstance.Read<ulong>(8) to read
ulong from offset
8. By default
it checks bounds, and LMDB call via P/Invoke takes much longer so there is no reason to switch the
bounds checks off, but you can still do so e.g. if you read separate bytes of large values
a lot (e.g. via indexer
directBufferInstance[offset] that returns a single byte at
Generic key/values support
unmanaged structs could be used directly as keys/values. Until
constraint and blittable helpers (at least
IsBlittable) are widly available we use
opt-in to treat a custom user-defined struct as blittable. It must have explicit
[StructLayout(LayoutKind.Sequential, Size = XX)] or defined Spreads'
BlittableSize parameter for non-generic types or
PreferBlittable set to
for generic types that could be blittable depending on a concrete type. The logic to decide
if a type is fixed-size is in TypeHelper
TypeHelper<T>.Size static property must be positive.
A database or duplicate values of a key in a single dupsorted database could be enumerated via
dataVaseInstance.AsEnumerable([several overloads]) methods that could return
DirectBuffers or generic blittable structs.
Tests show how to use the code.
Status & limitations
This library is being deployed and tested in production and is went through many performance and correctness stress tests as a part of a larger workload.
The project has required native binaries and source in
Binaries are native shared libraries compressed with
deflate and embedded into the package dll as resources (this often simplifies deployment).
The library works with original native binaries as well if not using two
TryFind helper methods.
The library does not support nested transactions yet - only because we do not use them currently. They will be added as soon as we find a real-world compelling case for them.
Issues & PRs are welcome!
MPL 2.0 (c) Victor Baybekov, 2018