Skip to content

Home

Jump to bottom
wolfkdy edited this page Aug 30, 2021 · 12 revisions

Welcome to MongoRocks wiki

MongoRocks is MongoDB with RocksDB as a storage engine. It's plugging into MongoDB though the storage engine API, which was released as part of MongoDB 3.0: http://docs.mongodb.org/manual/faq/storage/. The lastest stable version of MongoRocks is v3.2, v3.4, v4.0.3, and v4.2.5

To report bugs, ask questions or leave feedback, it's suggested to open a github issue.For tech support or emergency issues, it is prefered to use github issue board and/or contact wolfkdy1989@gmail.com.

Repositories

In version 3.X, RocksDB code is part of main Mongo repository. In 3.2 and going forward, the code for MongoRocks will be a separate module. There are currently two repositories:

  • https://github.com/mongodb-partners/mongo-rocks -- This is the repository for MongoRocks module for versions 3.2 and going forward. It's still in active development, as is MongoDB version 3.2.
  • https://github.com/mongodb-partners/mongo -- This is the fork of MongoDB repository and it's used for developing version 3.0. There is one development branch and many release branches:
  • v3.0-fb -- Development branch. This is where all the new commits and fixes go.
  • v3.0.8-mongorocks -- MongoRocks 3.0.8 release. We will keep releasing new versions in format 'v3.0.x-mongorocks'

In version 4.0, the latest stable version of mongoRocks is v4.0.3, please refer to this install manual for more details of intergrating mongoRocks4.0.3 with mongoDB-4.0.3.

In version 4.2, the latest stable version of mongoRocks is v4.2.5.

This all matters to you if you wish to compile from source. MongoRocks is oritented to experienced developers who are willing/familiar to compile mongodb from source. Currently we don't provide binaries.

Features

For the most part, running MongoDB with RocksDB storage engine should be transparent to the user. However, there are some cool features and configuration options that can make your experience even better.

Backups

RocksDB's files are immutable. This means that backups are easy and fast: 1. Find a list of live files, 2. hard link to a different directory (copy if the destination is on the different file system). To issue a backup, you can call:

db.adminCommand({setParameter:1, rocksdbBackup: "/var/lib/mongodb/backup/1"})

(Yes, we're aware that it's a bit silly to use setParameter API to issue backups. We're planning to move to MongoDB's command API in 3.2)

This will create a directory /var/lib/mongodb/backup/1 (it should not exist before) and hardlink all the relevant files. You can then copy those files to S3 or HDFS in the background. We're building a tool that will incrementally backup MongoRocks to S3. Keep an eye on MongoRocks Google Group for the announcement.

Compact the database instance

RocksDB's writes are very fast because bulk of the work is done in the background in a process called compaction. Compactions are automatically triggered when the state of LSM tree becomes non-ideal. However, you can also trigger the compaction manually. After the compaction is done your reads will be faster and space used on disk a bit smaller (approximately 10%). To schedule a manual compaction, you can call:

db.adminCommand({setParameter:1, rocksdbCompact: 1})

Configuration

Configuring RocksDB is a bit of an art. We hope that the default configuration will be good for most cases, but you can always get better performance by tuning, especially if your workload is special in some way.

There are couple of parameters you configure:

  • --rocksdbCacheSizeGB or storage.rocksdb.cacheSizeGB -- size of RocksDB's block cache. By default 30% of RAM. We keep uncompressed pages in the block cache and compressed pages in the kernel's page cache. You can also configure block cache size dynamically by calling: db.adminCommand({setParameter:1, rocksdbRuntimeConfigCacheSizeGB: 10})
  • --rocksdbCompression or storage.rocksdb.compression -- compression. By default this is snappy. Other available options are none and zlib. If your binary doesn't support the requested compression, opening the database will fail.
  • --rocksdbConfigString or storage.rocksdb.configString -- through this parameter you can configure all other RocksDB options. The format is the same as option string described here: https://github.com/facebook/rocksdb/wiki/Option-String-and-Option-Map#option-string
  • --rocksdbMaxWriteMBPerSec or storage.rocksdb.maxWriteMBPerSec -- default is 1024. RocksDB compactions can create spiky writes to IO, which can cause higher P99 storage read latency. You can use this option to smooth our the writes. For example, if you set this to 100MB/s, RocksDB will make sure to never write more than 100MB/s to storage. That way writes will be smoother and there will be storage bandwidth available for reads to go through. You can also change this value dynamically by calling db.adminCommand({setParameter:1, rocksdbRuntimeConfigMaxWriteMBPerSec:30})
  • --rocksdbCrashSafeCounters or storage.rocksdb.crashSafeCounters -- false by default. This means that if your database performs an unclean shutdown, the counters for number of records in a collections might be wrong. You can correct them with MongoDB's validate call. This is similar to WiredTiger behavior. If you set this option to true, then we'll make sure that counters are correct even after a crash. Write performance might suffer a bit, of course.

Things to be careful about

In this section, we'll write about any issues that might happen when running MongoRocks and how to fix them. Currently, we're aware of one thing to be careful about.

Number of file descriptors

By default, Linux kernel allows each process to use only 1024 file descriptors. MongoRocks is configured in such a way that it's using 32MB files, so if your database is 1TB in size, you'll need 32K files. Before running MongoRocks, please increase the number of file descriptors that MongoDB process can use. Here's the recommended setting of ulimit from MongoDB's docs: http://docs.mongodb.org/manual/reference/ulimit/#recommended-ulimit-settings

Monitoring and debugging

Run db.serverStatus()["rocksdb"] and enjoy. We'll write a separate wiki page explaining what all of this means. In the meantime, you can start here: https://github.com/facebook/rocksdb/wiki/RocksDB-Tuning-Guide#compaction-stats

Clone this wiki locally