@gfosco gfosco released this Jul 4, 2018 · 253 commits to master since this release

Assets 2

5.14.2 (7/3/2018)

Bug Fixes

  • Change default value of bytes_max_delete_chunk to 0 in NewSstFileManager() as it doesn't work well with checkpoints.
  • Set DEBUG_LEVEL=0 for RocksJava Mac Release build.

5.14.1 (6/20/2018)

Bug Fixes

  • Fix block-based table reader pinning blocks throughout its lifetime, causing memory usage increase.
  • Fix bug with prefix search in partition filters where a shared prefix would be ignored from the later partitions. The bug could report an eixstent key as missing. The bug could be triggered if prefix_extractor is set and partition filters is enabled.

5.14.0 (5/16/2018)

Public API Change

  • Add a BlockBasedTableOption to align uncompressed data blocks on the smaller of block size or page size boundary, to reduce flash reads by avoiding reads spanning 4K pages.
  • The background thread naming convention changed (on supporting platforms) to "rocksdb:", e.g., "rocksdb:low0".
  • Add a new ticker stat rocksdb.number.multiget.keys.found to count number of keys successfully read in MultiGet calls
  • Touch-up to write-related counters in PerfContext. New counters added: write_scheduling_flushes_compactions_time, write_thread_wait_nanos. Counters whose behavior was fixed or modified: write_memtable_time, write_pre_and_post_process_time, write_delay_time.
  • Posix Env's NewRandomRWFile() will fail if the file doesn't exist.
  • Now, DBOptions::use_direct_io_for_flush_and_compaction only applies to background writes, and DBOptions::use_direct_reads applies to both user reads and background reads. This conforms with Linux's open(2) manpage, which advises against simultaneously reading a file in buffered and direct modes, due to possibly undefined behavior and degraded performance.
  • Iterator::Valid() always returns false if !status().ok(). So, now when doing a Seek() followed by some Next()s, there's no need to check status() after every operation.
  • Iterator::Seek()/SeekForPrev()/SeekToFirst()/SeekToLast() always resets status().

New Features

  • Introduce TTL for level compaction so that all files older than ttl go through the compaction process to get rid of old data.
  • TransactionDBOptions::write_policy can be configured to enable WritePrepared 2PC transactions. Read more about them in the wiki.
  • Add DB properties "rocksdb.block-cache-capacity", "rocksdb.block-cache-usage", "rocksdb.block-cache-pinned-usage" to show block cache usage.
  • Add Env::LowerThreadPoolCPUPriority(Priority) method, which lowers the CPU priority of background (esp. compaction) threads to minimize interference with foreground tasks.
  • Fsync parent directory after deleting a file in delete scheduler.
  • In level-based compaction, if bottom-pri thread pool was setup via Env::SetBackgroundThreads(), compactions to the bottom level will be delegated to that thread pool.

Bug Fixes

  • Fsync after writing global seq number to the ingestion file in ExternalSstFileIngestionJob.
  • Fix WAL corruption caused by race condition between user write thread and FlushWAL when two_write_queue is not set.
  • Fix BackupableDBOptions::max_valid_backups_to_open to not delete backup files when refcount cannot be accurately determined.
  • Fix memory leak when pin_l0_filter_and_index_blocks_in_cache is used with partitioned filters
  • Disable rollback of merge operands in WritePrepared transactions to work around an issue in MyRocks. It can be enabled back by setting TransactionDBOptions::rollback_merge_operands to true.
  • Fix bug with prefix search in partition filters where a shared prefix would be ignored from the later partitions. The bug could report an eixstent key as missing. The bug could be triggered if prefix_extractor is set and partition filters is enabled.

Java API Changes

  • Add BlockBasedTableConfig.setBlockCache to allow sharing a block cache across DB instances.
  • Added SstFileManager to the Java API to allow managing SST files across DB instances.

@gfosco gfosco released this Jun 18, 2018 · 414 commits to master since this release

Assets 2

Bug Fixes

  • Fix regression bug of Prev() with ReadOptions.iterate_upper_bound.

@gfosco gfosco released this Jun 18, 2018 · 459 commits to master since this release

Assets 2

Bug Fixes

  • Fix regression bug of Prev() with ReadOptions.iterate_upper_bound.

@ajkr ajkr released this Jun 7, 2018 · 5 commits to 5.13.fb since this release

Assets 2

5.13.3 (6/6/2018)

Bug Fixes

  • Fix assertion when reading bloom filter of SST files containing range deletions but no data

@gfosco gfosco released this May 25, 2018 · 414 commits to master since this release

Assets 2

5.13.2 (5/25/2018)

Public API Change

  • Introduced CompressionOptions::kDefaultCompressionLevel, which is a generic way to tell RocksDB to use the compression library's default level. It is now the default value for CompressionOptions::level. Previously the level defaulted to -1, which gave poor compression ratios in ZSTD.

Bug Fixes

  • Fix segfault caused by object premature destruction (PR #3898)
  • Fix an issue with unnecessary capture in lambda expressions (PR #3904)

@gfosco gfosco released this May 14, 2018 · 414 commits to master since this release

Assets 2

5.13.1 (4/30/2018)

New Features

  • Add Env::LowerThreadPoolCPUPriority(Priority) method, which lowers the CPU priority of background (esp. compaction) threads to minimize interference with foreground tasks.
  • Eliminate use of temporary directories in BackupEngine to improve reliability on distributed file systems.

5.13.0 (3/20/2018)

Public API Change

  • RocksDBOptionsParser::Parse()'s ignore_unknown_options argument will only be effective if the option file shows it is generated using a higher version of RocksDB than the current version.
  • Remove CompactionEventListener.

New Features

  • SstFileManager now can cancel compactions if they will result in max space errors. SstFileManager users can also use SetCompactionBufferSize to specify how much space must be leftover during a compaction for auxiliary file functions such as logging and flushing.
  • Avoid unnecessarily flushing in CompactRange() when the range specified by the user does not overlap unflushed memtables.
  • If ColumnFamilyOptions::max_subcompactions is set greater than one, we now parallelize large manual level-based compactions.
  • Add "rocksdb.live-sst-files-size" DB property to return total bytes of all SST files belong to the latest LSM tree.

Bug Fixes

  • Fix a leak in prepared_section_completed_ where the zeroed entries would not removed from the map.
  • Fix WAL corruption caused by race condition between user write thread and backup/checkpoint thread.
  • Fsync after writing global seq number to the ingestion file in ExternalSstFileIngestionJob.
  • Fix memory leak when pin_l0_filter_and_index_blocks_in_cache is used with partitioned filters

Java API Changes

  • Add BlockBasedTableConfig.setBlockCache to allow sharing a block cache across DB instances.

@gfosco gfosco released this Apr 23, 2018 · 459 commits to master since this release

Assets 2

5.12.4 (4/23/2018)

Bug Fixes

  • Fix memory leak in two_level_iterator

5.12.3 (4/23/2018)

Bug Fixes

  • Fix memory leak when pin_l0_filter_and_index_blocks_in_cache is used with partitioned filters

5.12.2 (3/23/2018)

Bug Fixes

  • Fsync after writing global seq number to the ingestion file in ExternalSstFileIngestionJob.

Java API Changes

  • Add BlockBasedTableConfig.setBlockCache to allow sharing a block cache across DB instances.

5.12.1 (3/16/2018)

Public API Change

  • RocksDBOptionsParser::Parse()'s ignore_unknown_options argument will only be effective if the option file shows it is generated using a higher version of RocksDB than the current version.

New Features

  • Avoid unnecessarily flushing in CompactRange() when the range specified by the user does not overlap unflushed memtables.

Bug Fixes

  • Fix WAL corruption caused by race condition between user write thread and backup/checkpoint thread.

5.12.0 (2/14/2018)

Public API Change

  • Iterator::SeekForPrev is now a pure virtual method. This is to prevent user who implement the Iterator interface fail to implement SeekForPrev by mistake.
  • Add include_end option to make the range end exclusive when include_end == false in DeleteFilesInRange().
  • Add CompactRangeOptions::allow_write_stall, which makes CompactRange start working immediately, even if it causes user writes to stall. The default value is false, meaning we add delay to CompactRange calls until stalling can be avoided when possible. Note this delay is not present in previous RocksDB versions.
  • Creating checkpoint with empty directory now returns Status::InvalidArgument; previously, it returned Status::IOError.
  • Adds a BlockBasedTableOption to turn off index block compression.
  • Close() method now returns a status when closing a db.

New Features

  • Improve the performance of iterators doing long range scans by using readahead.
  • Add new function DeleteFilesInRanges() to delete files in multiple ranges at once for better performance.
  • FreeBSD build support for RocksDB and RocksJava.
  • Improved performance of long range scans with readahead.
  • Updated to and now continuously tested in Visual Studio 2017.

Bug Fixes

  • Fix DisableFileDeletions() followed by GetSortedWalFiles() to not return obsolete WAL files that PurgeObsoleteFiles() is going to delete.
  • Fix Handle error return from WriteBuffer() during WAL file close and DB close.
  • Fix advance reservation of arena block addresses.
  • Fix handling of empty string as checkpoint directory.
Apr 23, 2018
5.12.3 release with memory leak fix for pinned partition filters

@gfosco gfosco released this Apr 3, 2018 · 459 commits to master since this release

Assets 2

5.12.2 (3/23/2018)

Bug Fixes

  • Fsync after writing global seq number to the ingestion file in ExternalSstFileIngestionJob.

Java API Changes

  • Add BlockBasedTableConfig.setBlockCache to allow sharing a block cache across DB instances.

5.12.1 (3/16/2018)

Public API Change

  • RocksDBOptionsParser::Parse()'s ignore_unknown_options argument will only be effective if the option file shows it is generated using a higher version of RocksDB than the current version.

New Features

  • Avoid unnecessarily flushing in CompactRange() when the range specified by the user does not overlap unflushed memtables.

Bug Fixes

  • Fix WAL corruption caused by race condition between user write thread and backup/checkpoint thread.

5.12.0 (2/14/2018)

Public API Change

  • Iterator::SeekForPrev is now a pure virtual method. This is to prevent user who implement the Iterator interface fail to implement SeekForPrev by mistake.
  • Add include_end option to make the range end exclusive when include_end == false in DeleteFilesInRange().
  • Add CompactRangeOptions::allow_write_stall, which makes CompactRange start working immediately, even if it causes user writes to stall. The default value is false, meaning we add delay to CompactRange calls until stalling can be avoided when possible. Note this delay is not present in previous RocksDB versions.
  • Creating checkpoint with empty directory now returns Status::InvalidArgument; previously, it returned Status::IOError.
  • Adds a BlockBasedTableOption to turn off index block compression.
  • Close() method now returns a status when closing a db.

New Features

  • Improve the performance of iterators doing long range scans by using readahead.
  • Add new function DeleteFilesInRanges() to delete files in multiple ranges at once for better performance.
  • FreeBSD build support for RocksDB and RocksJava.
  • Improved performance of long range scans with readahead.
  • Updated to and now continuously tested in Visual Studio 2017.

Bug Fixes

  • Fix DisableFileDeletions() followed by GetSortedWalFiles() to not return obsolete WAL files that PurgeObsoleteFiles() is going to delete.
  • Fix Handle error return from WriteBuffer() during WAL file close and DB close.
  • Fix advance reservation of arena block addresses.
  • Fix handling of empty string as checkpoint directory.

@miasantreble miasantreble released this Mar 12, 2018 · 2 commits to 5.11.fb since this release

Assets 2

5.11.2 (02/24/2018)

Bug Fixes

  • Fix bug in iterator readahead causing blocks to incorrectly be considered truncated (corrupted).

5.11.1 (02/22/2018)

New Features

  • Follow rsync-style naming convention for BackupEngine tempfiles. This enables some optimizations when run on GlusterFS.

5.11.0 (01/08/2018)

Public API Change

  • Add autoTune and getBytesPerSecond() to RocksJava RateLimiter

New Features

  • Add a new histogram stat called rocksdb.db.flush.micros for memtable flush.
  • Add "--use_txn" option to use transactional API in db_stress.
  • Disable onboard cache for compaction output in Windows platform.
  • Improve the performance of iterators doing long range scans by using readahead.

Bug Fixes

  • Fix a stack-use-after-scope bug in ForwardIterator.
  • Fix builds on platforms including Linux, Windows, and PowerPC.
  • Fix buffer overrun in backup engine for DBs with huge number of files.
  • Fix a mislabel bug for bottom-pri compaction threads.
  • Fix DB::Flush() keep waiting after flush finish under certain condition.