Keyvaluestore enhance #1282

merged 21 commits into from Feb 23, 2014


None yet

2 participants


KeyValueStore enhancement:

  1. Add more KeyValueStore options
  2. Fix some KeyValueStore bugs
  3. Add more tests to test ObjectStore
  4. Make lock clear
  5. Remove unused or filestore-copied codes
  6. Add perf counter to KeyValueStore
yuyuyu101 added some commits Feb 18, 2014
@yuyuyu101 yuyuyu101 Fix deadlock caused by hold collection obj
Collection is a special object in KeyValueStore, if exists collection modify,
this object will be hold. When exists check collection, it will be dead lock to
try to access this object.

Now lookup transaction cache first to ensure no collection object exists

Signed-off-by: Haomai Wang <>
@yuyuyu101 yuyuyu101 Move perf counter and add op queue reserve throttle
The perf counter of FileStore can shared with other ObjectStore backend, so move
it to ObjectStore and adjust position to let other KeyValueStore refer to.

Signed-off-by: Haomai Wang <>
@yuyuyu101 yuyuyu101 Add test for omap interface
Signed-off-by: Haomai Wang <>
@yuyuyu101 yuyuyu101 Add get_*_with_header to StripObjectHeader
In some situations, we need to ensure header is held and try to read, so read
interfaces need to accept header argument to allow.

Signed-off-by: Haomai Wang <>
@yuyuyu101 yuyuyu101 Use rename operation to make colection_move atomic
Signed-off-by: Haomai Wang <>
@yuyuyu101 yuyuyu101 Adjust some interfaces to optimize caller codes
Signed-off-by: Haomai Wang <>
@yuyuyu101 yuyuyu101 Add read/write operation to
Signed-off-by: Haomai Wang <>
@yuyuyu101 yuyuyu101 Return 0 when the offset of read exceed the length of object
Signed-off-by: Haomai Wang <>
@yuyuyu101 yuyuyu101 Fix incorrect read and truncate
If the first chunk of object is not read entirely, the current implementation
will contain it entirely which make wrong.

Signed-off-by: Haomai Wang <>
@yuyuyu101 yuyuyu101 Unify object level lock in GenericObjectMap
Before we copy lock implementation from DBObjectMap which provide with two locks
for header. Here just unify it to make ease.

Signed-off-by: Haomai Wang <>
@yuyuyu101 yuyuyu101 Optimize write call add enhance error detect
A single write call may need several keys in the backend kv store. Let get these
keys one time.

Signed-off-by: Haomai Wang <>
@yuyuyu101 yuyuyu101 Remove unnecessary "check_coll" check
The goal of "check_coll" is aimed to ensure the collection is exists. But
the create and delete of collection already ensured by OSD, just remove check.

Signed-off-by: Haomai Wang <>
@yuyuyu101 yuyuyu101 Implement collection_rename interface in KeyValueStore
Signed-off-by: Haomai Wang <>
@yuyuyu101 yuyuyu101 Make SequencePosition sync by each transaction
Each object modify will increase SequencePosition and sync it to disk

Signed-off-by: Haomai Wang <>
@yuyuyu101 yuyuyu101 Remove m_eio in KeyValueStore
KeyValueStore use kv backend to detect errors and nearly can't know the actual
reason for error.

Signed-off-by: Haomai Wang <>
@yuyuyu101 yuyuyu101 Remove filestore_inject_stall
Signed-off-by: Haomai Wang <>
@yuyuyu101 yuyuyu101 Add backend check option to KeyValueStore
Signed-off-by: Haomai Wang <>
@yuyuyu101 yuyuyu101 Add config tracker to KeyValueStore
Signed-off-by: Haomai Wang <>
@yuyuyu101 yuyuyu101 Remove eio inject codes in KeyValueStore
Signed-off-by: Haomai Wang <>
@yuyuyu101 yuyuyu101 Add KeyValueStore op thread options
Signed-off-by: Haomai Wang <>
@liewegas liewegas commented on an outdated diff Feb 22, 2014
@@ -650,6 +650,13 @@
OPTION(journal_aio, OPT_BOOL, true)
OPTION(journal_force_aio, OPT_BOOL, false)
+OPTION(keyvaluestore_queue_max_ops, OPT_INT, 50)
+OPTION(keyvaluestore_queue_max_bytes, OPT_INT, 100 << 20)
+OPTION(keyvaluestore_check_backend, OPT_BOOL, 0) // Expensive debugging check on sync
liewegas Feb 22, 2014 Member


@liewegas liewegas merged commit 90ebdcc into ceph:master Feb 23, 2014
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment