Skip to content
free as in lock-free embedded database
Branch: master
Clone or download
spacejam Merge pull request #633 from Kerollmops/try-from-into
Replace handcrafted number conversions
Latest commit 0fb6097 Apr 15, 2019
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
art Create CREDITS Oct 13, 2017
benchmarks/stress2
bindings rearrange subcrates a bit Mar 1, 2018
crates Replace the hand-crafted array to u64/u32 conversions by the std one Apr 15, 2019
examples Fix CRDT example Apr 5, 2019
hack Improve instruction counter Mar 16, 2019
tests Fix race condition in Flusher shutdown Apr 8, 2019
.gitignore gitignore IntelliJ .idea directory Feb 18, 2019
.rustfmt.toml Bump rustfmt lines width to 80 Mar 23, 2019
.travis.yml
CONTRIBUTING.md Update CONTRIBUTING.md Jan 31, 2019
Cargo.toml Merge pagetable into pagecache Mar 16, 2019
ISSUE_TEMPLATE Create ISSUE_TEMPLATE Feb 11, 2018
LICENSE-APACHE ©️ dat spicy law shit Feb 22, 2019
LICENSE-MIT ©️ dat spicy law shit Feb 22, 2019
README.md Cleanup of conversions, never downcast without a check or reasonable … Apr 1, 2019
RELEASE_CHECKLIST.md Add a release checklist. Update contributor note Nov 3, 2018
code-of-conduct.md Add contributor covenant. Sep 22, 2018
rustfmt.toml global reformat Jul 22, 2018
tsan_suppressions.txt Add rayon to TSAN suppressions file Dec 18, 2018

README.md

sled

Build Status crates.io documentation chat

An (alpha) modern embedded database.

use sled::Db;

let tree = Db::start_default(path)?;

// set and get
tree.set(k, v1);
assert_eq!(tree.get(&k), Ok(Some(v1)));

// compare and swap
tree.cas(k, Some(v1), Some(v2));

// scan forward
let mut iter = tree.scan(k);
assert_eq!(iter.next(), Some(Ok((k, v2))));
assert_eq!(iter.next(), None);

// deletion
tree.del(&k);

// block until all operations are on-disk
tree.flush();

We also support merge operators!

features

  • API similar to a threadsafe BTreeMap<Vec<u8>, Vec<u8>>
  • fully atomic single-key operations, supports CAS
  • zero-copy reads
  • subscription/watch semantics on key prefixes
  • multiple keyspace support
  • merge operators
  • forward and reverse iterators
  • a crash-safe monotonic ID generator capable of generating 75-125 million ID's per second
  • zstd compression (use the compression build feature)
  • cpu-scalable lock-free implementation
  • SSD-optimized log-structured storage

goals

  1. don't make the user think. the interface should be obvious.
  2. don't surprise users with performance traps.
  3. don't wake up operators. bring reliability techniques from academia into real-world practice.
  4. don't use so much electricity. our data structures should play to modern hardware's strengths.

plans

  • LSM tree-like write performance with traditional B+ tree-like read performance
  • MVCC, serializable transactions, and snapshots
  • forward-compatible binary format
  • concurrent snapshot delta generation and recovery
  • first-class access to replication stream
  • consensus protocol for PC/EC systems
  • pluggable conflict detection and resolution strategies for PA/EL systems
  • multiple collection types like tables, BKD trees, Merkle trees, bloom filters, etc... unified under a single transactional and operational domain

Want to prioritize a specific feature or get commercial help with using sled in your project? Ferrous Systems provides commercial support for sled, and can work with you to solve a wide variety of storage problems across the latency-throughput, consistency, and price performance spectra. Get in touch!

Ferrous Systems

known issues, warnings

  • the on-disk format is going to change in non-forward compatible ways before the 1.0.0 release! after that, we will always support forward migrations.
  • quite young, should be considered unstable for the time being
  • the C API is likely to change rapidly
  • writepath is not well optimized yet. readpath is essentially wait-free and zero-copy.

contribution welcome!

want to help advance the state of the art in open source embedded databases? check out CONTRIBUTING.md!

architecture

lock-free tree on a lock-free pagecache on a lock-free log. the pagecache scatters partial page fragments across the log, rather than rewriting entire pages at a time as B+ trees for spinning disks historically have. on page reads, we concurrently scatter-gather reads across the log to materialize the page from its fragments. check out the architectural outlook for a more detailed overview of where we're at and where we see things going!

References

You can’t perform that action at this time.