Skip to content

Commit

Permalink
MB-16181: Compress system events for ephemeral buckets
Browse files Browse the repository at this point in the history
For ephemeral vbuckets, compress the system-event Item as it will
be retained in memory. This can reduce the memory overhead that results
from the creation of scopes and collections, especially when long names
are used.

The actual improvement really depends on the length and content of
the name, i.e. can snappy actually achieve good compression and note
that the 'compressValue' result in no change to the Item if it could
not compress the value.

For reference we have 4 types of System-Event (4 different value
formats) and they are all 'flatbuffer' types. The flatbuffer types that
define the create system-events store the name and are of interest to
this change.

The following table shows the size of the CreateCollection value as name
increases (max name length is 251).

name length | size
===================
0           | 40
2           | 40
4           | 44
6           | 44
8           | 48
10          | 48
12          | 52
14          | 52
16          | 56
18          | 56
251         | 288

From that table the CreateCollection value size is:
    40 + 4 * (strlen(name) / 4)

The fixed 40 bytes doesn't compress, so the result is all in the name.

Note:

CreateScope is very similar to CreateCollection, it is slightly smaller
but stores a name which can be upto 251 bytes.

DropCollection/DropScope do not store a name and are currently 24 bytes
and those 24 bytes do not compress.

Change-Id: Id293e6eda133e82be4251454f1db02a40ff2a2c6
Reviewed-on: http://review.couchbase.org/c/kv_engine/+/143918
Reviewed-by: Dave Rigby <daver@couchbase.com>
Tested-by: Build Bot <build@couchbase.com>
  • Loading branch information
jimwwalker authored and daverigby committed Jan 27, 2021
1 parent c518609 commit 57fd17b
Show file tree
Hide file tree
Showing 2 changed files with 5 additions and 0 deletions.
4 changes: 4 additions & 0 deletions engines/ep/src/ephemeral_vb.cc
Original file line number Diff line number Diff line change
Expand Up @@ -998,6 +998,10 @@ uint64_t EphemeralVBucket::addSystemEventItem(
}

item->setVBucketId(getId());

// ephemeral keeps system events in-memory, so keep it compressed.
item->compressValue();

auto htRes = ht.findForWrite(item->getKey());
auto* v = htRes.storedValue;
auto& hbl = htRes.lock;
Expand Down
1 change: 1 addition & 0 deletions engines/ep/src/sizes.cc
Original file line number Diff line number Diff line change
Expand Up @@ -132,6 +132,7 @@ int main(int, char **) {
display("queued_item", sizeof(queued_item));
display("Collections::VB::ManifestEntry",
sizeof(Collections::VB::ManifestEntry));
display("cb::ExpiryLimit", sizeof(cb::ExpiryLimit));

std::cout << std::endl << "Histogram Ranges" << std::endl << std::endl;

Expand Down

0 comments on commit 57fd17b

Please sign in to comment.