A library that provides an embeddable, persistent key-value store for fast storage optimized for AWS
C++ Java Perl C Shell Makefile Other
Switch branches/tags
Clone or download
Pull request Compare This branch is 296 commits ahead, 582 commits behind facebook:master.
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
buckifier Suppress lint in old files Feb 6, 2018
build_tools Merge tag 'v5.11.3' Mar 27, 2018
cache fix gflags namespace Dec 1, 2017
cloud Print a logline when DeleteInvisibleFile deletes a file from local dir. Aug 6, 2018
cmake add missing config checks to CMakeLists.txt Dec 1, 2017
coverage Suppress lint in old files Feb 6, 2018
db Fix db_basic_test to run successfully when USE_AWS is set. May 7, 2018
docker Initial Applatix integration Mar 21, 2017
docs Blog post for WritePrepared Txn Dec 20, 2017
env Disable CloudManifest for unit tests. Aug 6, 2018
examples Pinnableslice examples and blog post Aug 24, 2017
hdfs Suppress lint in old files Feb 6, 2018
include/rocksdb Merge pull request #34 from tgudlek/tgudlek-fixup Mar 27, 2018
java Java wrapper for Native Comparators Mar 8, 2018
memtable fix gflags namespace Dec 1, 2017
monitoring fix ThreadStatus for bottom-pri compaction threads Dec 14, 2017
options Make Universal compaction options dynamic Dec 11, 2017
port FreeBSD build support for RocksDB and RocksJava Jan 26, 2018
table Suppress lint in old files Feb 6, 2018
third-party Enable MSVC W4 with a few exceptions. Fix warnings and bugs Oct 19, 2017
tools Merge tag 'v5.11.3' Mar 27, 2018
util fix ReadaheadRandomAccessFile/iterator prefetch bug Feb 25, 2018
utilities Merge tag 'v5.11.3' Mar 27, 2018
.clang-format A script that automatically reformat affected lines Jan 14, 2014
.gitignore Merge commit '266ac245affd69555643b735d19d3715c4eff91a' Nov 16, 2017
.travis.yml Skip all ApproximateSizes* unit test May 10, 2018
AUTHORS Update RocksDB Authors File Oct 18, 2017
CMakeLists.txt Added bytes XOR merge operator Mar 6, 2018
CODE_OF_CONDUCT.md Add Code of Conduct Dec 6, 2017
CONTRIBUTING.md Merge tag 'v5.11.3' Mar 27, 2018
COPYING Add GPLv2 as an alternative license. Apr 28, 2017
DEFAULT_OPTIONS_HISTORY.md options.delayed_write_rate use the rate of rate_limiter by default. May 24, 2017
DUMP_FORMAT.md First version of rocksdb_dump and rocksdb_undump. Jun 19, 2015
HISTORY.md update history and bump version Feb 25, 2018
INSTALL.md FreeBSD build support for RocksDB and RocksJava Jan 26, 2018
LANGUAGE-BINDINGS.md Add Elixir to the list of language bindings Nov 21, 2017
LICENSE.Apache Change RocksDB License Jul 15, 2017
LICENSE.leveldb Add back the LevelDB license file Jul 17, 2017
Makefile Merge tag 'v5.11.3' Mar 27, 2018
README.md Setup RocksDB Cloud .travis.yml Feb 8, 2018
ROCKSDB_LITE.md Optimistic Transactions May 29, 2015
TARGETS Merge tag 'v5.11.3' Mar 27, 2018
USERS.md Added ProfaneDB Nov 19, 2017
Vagrantfile Update Vagrant file (test internal phabricator workflow) Oct 28, 2016
WINDOWS_PORT.md Commit both PR and internal code review changes Jul 7, 2015
appveyor.yml Make Windows dep switches compatible with other builds Jan 5, 2018
issue_template.md Add a template for issues Sep 29, 2017
src.mk Merge tag 'v5.11.3' Mar 27, 2018
thirdparty.inc Make Windows dep switches compatible with other builds Jan 5, 2018

README.md

RocksDB-Cloud: A Key-Value Store for Cloud Applications

Build Status

RocksDB-Cloud is a C++ library that brings the power of RocksDB to AWS, Google Cloud and Microsoft Azure. It leverages the power of RocksDB to provide fast key-value access to data stored in Flash and RAM systems. It provides for data durability even in the face of machine failures by integrations with cloud services like AWS-S3 and Google Cloud Services. It allows a cost-effective way to utilize the rich hierarchy of storage services (based on RAM, NvMe, SSD, Disk Cold Storage, etc) that are offered by most cloud providers. RocksDB-Cloud is developed and maintained by the engineering team at Rockset Inc. Start with https://github.com/rockset/rocksdb-cloud/tree/master/cloud.

RocksDB-Cloud provides three main advantages for AWS environments:

  1. A rocksdb instance is durable. Continuous and automatic replication of db data and metadata to S3. In the event that the rocksdb machine dies, another process on any other EC2 machine can reopen the same rocksdb database (by configuring it with the S3 bucketname where the entire db state was stored).
  2. A rocksdb instance is cloneable. RocksDB-Cloud support a primitive called zero-copy-clone() that allows a slave instance of rocksdb on another machine to clone an existing db. Both master and slave rocksdb instance can run in parallel and they share some set of common database files.
  3. A rocksdb instance can leverage hierarchical storage. The entire rocksdb storage footprint need not be resident on local storage. S3 contains the entire database and the local storage contains only the files that are in the working set.

Inherits from RocksDB:

RocksDB-Cloud is API compatible and data format compatible with RocksDB.

RocksDB is developed and maintained by Facebook Database Engineering Team. It is built on earlier work on LevelDB by Sanjay Ghemawat (sanjay@google.com) and Jeff Dean (jeff@google.com)

This code is a library that forms the core building block for a fast key value server, especially suited for storing data on flash drives. It has a Log-Structured-Merge-Database (LSM) design with flexible tradeoffs between Write-Amplification-Factor (WAF), Read-Amplification-Factor (RAF) and Space-Amplification-Factor (SAF). It has multi-threaded compactions, making it specially suitable for storing multiple terabytes of data in a single database.

Start with example usage here: https://github.com/facebook/rocksdb/tree/master/examples

See the github wiki for more explanation.

The public interface is in include/. Callers should not include or rely on the details of any other header files in this package. Those internal APIs may be changed without warning.

Design discussions are conducted in https://www.facebook.com/groups/rocksdb.dev/