Skip to content

Commit

Permalink
[common] Add encrypted-file-node-size slab into slab allocator
Browse files Browse the repository at this point in the history
Allocating the file nodes (`file_node_t`) of Gramine's encrypted files
can be one source of overhead for certain workloads (e.g., RocksDB
compaction). To address this performance bottleneck, this commit
introduces a new slab size (8288B, considering the alignment) tailored
for the size of the file node (8235B) and therefore a new free list for
8256B blocks managed by the slab allocator. Additionally, one more slab
size (~4KB) is added, to have proper powers-of-2 slab levels.

Note that while this change can improve performance for certain memory
allocation patterns, it introduces internal fragmentation and may also
increase memory overhead and external fragmentation if the application
rarely makes allocations of the file node size. However, our limited
experiments did not show any issues.

In addition, this commit fixes an issue where in-memory files are not
zeroed out when resized.

Signed-off-by: Kailun Qin <kailun.qin@intel.com>
  • Loading branch information
kailun-qin authored and dimakuv committed Feb 12, 2024
1 parent c06a4ae commit 9288b49
Show file tree
Hide file tree
Showing 2 changed files with 10 additions and 4 deletions.
7 changes: 4 additions & 3 deletions common/include/slabmgr.h
Original file line number Diff line number Diff line change
Expand Up @@ -88,14 +88,15 @@ static_assert(IS_ALIGNED(offsetof(struct slab_area, raw), 16),
MIN_MALLOC_ALIGNMENT)

#ifndef SLAB_LEVEL
#define SLAB_LEVEL 8
#define SLAB_LEVEL 10
#endif

#ifndef SLAB_LEVEL_SIZES
#define SLAB_LEVEL_SIZES \
16, 32, 64, 128 - SLAB_HDR_SIZE, 256 - SLAB_HDR_SIZE, 512 - SLAB_HDR_SIZE, \
1024 - SLAB_HDR_SIZE, 2048 - SLAB_HDR_SIZE
#define SLAB_LEVELS_SUM (4080 - SLAB_HDR_SIZE * 5)
1024 - SLAB_HDR_SIZE, 2048 - SLAB_HDR_SIZE, 4096 - SLAB_HDR_SIZE, \
8288 - SLAB_HDR_SIZE
#define SLAB_LEVELS_SUM (16464 - SLAB_HDR_SIZE * 7)
#else
#ifndef SLAB_LEVELS_SUM
#error "SLAB_LEVELS_SUM not defined"
Expand Down
7 changes: 6 additions & 1 deletion libos/src/fs/libos_fs_mem.c
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,12 @@ static int mem_file_resize(struct libos_mem_file* mem, size_t buf_size) {
if (!buf)
return -ENOMEM;

memcpy(buf, mem->buf, MIN(buf_size, (size_t)mem->size));
size_t min_size = MIN(buf_size, (size_t)mem->size);
memcpy(buf, mem->buf, min_size);

/* if the new buffer is larger, zero out the new portion */
if (buf_size > min_size)
memset(buf + min_size, 0, buf_size - min_size);

free(mem->buf);
mem->buf = buf;
Expand Down

0 comments on commit 9288b49

Please sign in to comment.