HTTPS clone URL
Subversion checkout URL
Clone of http://code.google.com/p/leveldb/
C++ C Other
1.2.2-patches 1.3 1.4 1.4.2-turner 1.4.9 2.0 2.1 adt-bsd-support bjs-manifest-repair bugfix/replace-file-stream-functions-with-file-descriptors develop dss-compaction-tweaks dss-stats-util eas-64-or-32-bit-counters eas-min-mem-fix end-to-end/timeseries families feature/streaming-folds-filter freebsd10-clang hotfix/jdm-ifdefout-fadvise-1.3.3 jdb-mv-hot-threads-base jdb-mv-hot-threads-new jdb-total-bytes-stat master mv-32bit-counters mv-aggressive-delete mv-always-signal mv-async-close mv-async-close-1.4 mv-att-file-handles mv-badblock-recovery mv-bloom2 mv-bloom-loader mv-bloom-size-limit mv-cache-cleanup mv-cache-corruption mv-cache-object-warming mv-cache-release-fix mv-cache-revert mv-cache-unchange mv-cache-warming-tool mv-calc-bloom-segment mv-clean-overlaps mv-compact-cache mv-compact-inversion mv-comparator-segfault-1.4 mv-compress-bypass mv-compress-msg mv-counters-update mv-dataloss-N-unittests mv-destroy-dir mv_env_riak mv-error-logging-hack mv-fadvise-control mv-filecache-tuning2 mv-filesize-tuning2 mv-flexcache mv-flexcache2 mv-flexcache3 mv-flexcache5 mv-google-1-5 mv-google-filelock mv-hardware-crc mv-hash-cache mv-hot-threads1 mv-hot-threads2 mv-imm-thread mv-initial-level-fix mv-iterator-stats mv-large-files mv-level0-bloat mv-level-dirs mv-level-work1 mv-level-work2 mv-level-work3 mv-level-work4 mv-leveldb-1.9 mv-limit-repair mv-locks-closes mv-lru2fifo mv-makefile-fpic mv-mprotect1 mv-nonblocking-get mv-numa mv-numa2 mv-option-logging mv-option-typo mv-pagecache1 mv-perf-counters mv-perf-counters2 mv-read-throttle mv-repair-by-level mv-repair-lock mv-riak-1-4-cleanup mv-russell-counters mv-skiplist-upsize mv-snappy-count mv-spin-locks mv-sst-fadvise mv-sst-rewrite mv_sst_stats mv-thread-model mv-throttle2 mv-throttle-3 mv-throttle-3a mv-throttle-3b mv-throttle-4 mv-throttle-5 mv-tiered-locks2 mv-tiered-rwlock mv-timed-grooming mv-tuning-3b mv-tuning7-1.4.9 mv-valgrind-1 mv-valgrind-cleanup mv-verify-compactions mv-warnings-cleanup1 mv-write-buffers mv-write-patch-1.3 mv-write-sizing mv-write-throttle mv-write-throttle2 pp-hot-threads-backport pp-skiplist-sequential-insert prototype/add_new_stats prototype/compactions prototype/range-deletes prototype/timeseries rz-mv-option-logging-use-names sbv-db-value-holder sv-mavericks-port upstream
Nothing to show
Mv timed grooming
leveldb: A key-value store Authors: Sanjay Ghemawat (email@example.com) and Jeff Dean (firstname.lastname@example.org) The original Google README is now README.GOOGLE. ** Introduction This repository contains the Google source code as modified to benefit the Riak environment. The typical Riak environment has two attributes that necessitate leveldb adjustments, both in options and code: - production servers: Riak often runs in heavy Internet environments: servers with many CPU cores, lots of memory, and 24x7 disk activity. Basho's leveldb takes advantage of the environment by adding hardware CRC calculation, increasing Bloom filter accuracy, and defaulting to integrity checking enabled. - multiple databases open: Riak opens 8 to 128 databases simultaneously. Google's leveldb supports this, but its background compaction thread can fall behind. leveldb will "stall" new user writes whenever the compaction thread gets too far behind. Basho's leveldb modification include multiple thread blocks that each contain prioritized threads for specific compaction activities. Details for Basho's customizations exist in the leveldb wiki: http://github.com/basho/leveldb/wiki ** Branch pattern This repository follows the Basho standard for branch management as of November 28, 2013. The standard is found here: https://github.com/basho/riak/wiki/Basho-repository-management In summary, the "develop" branch contains the most recently reviewed engineering work. The "master" branch contains the most recently released work, i.e. distributed as part of a Riak release. ** Basic options needed Those wishing to truly savor the benefits of Basho's modifications need to initialize a new leveldb::Options structure similar to the following before each call to leveldb::DB::Open: leveldb::Options * options; options=new Leveldb::Options; options.filter_policy=leveldb::NewBloomFilterPolicy2(16); options.write_buffer_size=62914560; // 60Mbytes options.total_leveldb_mem=2684354560; // 2.5Gbytes (details below) options.env=leveldb::Env::Default(); ** Memory plan Basho's leveldb dramatically departed from Google's original internal memory allotment plan with Riak 2.0. Basho's leveldb uses a methodology called flexcache. The technical details are here: https://github.com/basho/leveldb/wiki/mv-flexcache The key points are: - options.total_leveldb_mem is an allocation for the entire process, not a single database - giving different values to options.total_leveldb_mem on subsequent Open calls causes memory to rearrange to current value across all databases - recommended minimum for Basho's leveldb is 340Mbytes per database. - performance improves rapidly from 340Mbytes to 2.5Gbytes per database (3.0Gbytes if using Riak's active anti-entropy). Even more is nice, but not as helpful. - never assign more than 75% of available RAM to total_leveldb_mem. There is too much unaccounted memory overhead (worse if you use tcmalloc library). - options.max_open_files and options.block_cache should not be used.