The reason for the ItemHandle

[skyelves]

In CacheLib, to write an object, we need to first allocate an memory and then write data to allocated memory and insert item handle into cache.
To read an object, we first read the item handle from the cache and then read the value.
I am wondering the design of an extra abstraction of ItemHandle. Why do we need the ItemHandle? Is it for better concurrency or any other reasons?

[1a1a11a]

My guess is for async read for NVM

[sathyaphoenix]

I am wondering the design of an extra abstraction of ItemHandle. Why do we need the ItemHandle? Is it for better concurrency or any other reasons?

Item exposes the allocation for object in cache. ItemHandle exposes the Item to end user. ItemHandle exists for several reasons. Mostly it acts as a wrapper abstraction to how Item is exposed through our APIs. For example,

• ItemHandle enables zero-copy access to Item and the co-ordination with memory management of Item's resources. Every ItemHandle represents an outstanding reference to Item handed to user or internal components.
• It hides the async nature of Item when Item is present in NVM and not in DRAM yet.

Conceptually, CacheLib users interact with Item through the ItemHandle.

[skyelves]

Can you elaborate a little bit more on "It hides the async nature of Item"? How does ItemHandle achieve that?

[sathyaphoenix]

@skyelves : perhaps this documentation can help with that https://cachelib.org/docs/Cache_Library_User_Guides/HybridCache#accessing-items-on-cache

[haowu14]

Hi @skyelves. ItemHandle helps manage Item's lifetime. Async read from NVM like @1a1a11a is one example of that.
But if your question is why CacheLib needs to allocate before insert, it's because CacheLib uses slab allocation and manages its own memory.

[sathyaphoenix]

If a pointer is returned, the cache can not know when the application is done using the pointer so that the Item can be freed or evicted internally. To do this by returning pointer, the application has to them remember to call another potential API to let the cache know when the pointer is no longer accessed by the application. ItemHandle enables this in a simpler way by wrapping the pointer into a C++ object with RAII semantics. When the application is done with the handle, the handle is destroyed in the stack and the handle destructor lets the cache decrement the refcount. It is equivalent to an API that returns an std::shared_ptr instead of T* in c++

[skyelves]

That makes a lot of sense!