When run in two consecutive times DB::Open has quite different time cost

[zergvszerg]

Note: Please use Issues only for bug reports. For questions, discussions, feature requests, etc. post to dev group: https://groups.google.com/forum/#!forum/rocksdb or https://www.facebook.com/groups/rocksdb.dev

Expected behavior

Our application do checkpoint-then-open two times in a row. We notice that the first call into DB::Open take a lot longer than the second.
We expect these two runs cost about the same amount of time.

Actual behavior

We notice the first and longer call into DB::Open takes 2 times longer in VersionEditHandler and 6 times longer in VersionSet::ProcessManifestWrites.

Steps to reproduce the behavior

1st run

• CreateCheckpoint() to create a new db instance A.
• costs 99ms to Open() the new db instance A.

2nd run

• CreateCheckpoint() to create a new db instance B, which is the same as A.
• costs 32ms to Open() the new db instance B.

And there is no write on the parent db instance during the whole time so B should be the same as A.

[hx235]

Since you observed the different Open() time, could you provide a RocksDB UT repro and corresponding measurement?

[zergvszerg]

Sorry, I'm unable to reproduce it by UT case. So far this Open() time difference only occur in our online service when performing the checkpoint-then-open action 9 out of 10 times.