KV Cache

C++ Implementation of a Key-Value cache.

The cache is composed by a primary cache (RAM Cache) and a secondary cache (Storage Cache).

The RAM Cache has a maximum size. When the maximum capacity has been reached, KV Pairs will be evicted based on the selected cache eviction strategies to make place to new elements. Evicted elements will then be moved to the secondary cache.

Tools and Required Software

Before the project can be used, you should check that all the necessary softwares and tools are installed. Mandatory are the following:

- C++ compiler compatible with C++20

• gcc 7+

  Install command

  • Debian/Ubuntu:

      sudo apt install build-essential
    

  • Windows:

      choco install mingw -y
    

  • MacOS:

      brew install gcc
    

• clang 6+

  Install command

  • Debian/Ubuntu:

      bash -c "$(wget -O - https://apt.llvm.org/llvm.sh)"
    

  • Windows:

    Visual Studio 2019 ships with LLVM (see the Visual Studio section). However, to install LLVM separately:

      choco install llvm -y
    

    llvm-utils for using external LLVM with Visual Studio generator:

      git clone https://github.com/zufuliu/llvm-utils.git
      cd llvm-utils/VS2017
      .\install.bat
    

  • MacOS:

      brew install llvm
    

• Visual Studio 2019 or higher

  Install command + Environment setup

  On Windows, you need to install Visual Studio 2019 because of the SDK and libraries that ship with it.

  Visual Studio IDE - 2019 Community (installs Clang too):

    choco install -y visualstudio2019community --package-parameters "add Microsoft.VisualStudio.Workload.NativeDesktop --includeRecommended --includeOptional --passive --locale en-US"
  

  Put MSVC compiler, Clang compiler, and vcvarsall.bat on the path:

    	choco install vswhere -y
    	refreshenv
  
    	# change to x86 for 32bit
    	$clpath = vswhere -products * -latest -prerelease -find **/Hostx64/x64/*
    	$clangpath = vswhere -products * -latest -prerelease -find **/Llvm/bin/*
    	$vcvarsallpath =  vswhere -products * -latest -prerelease -find **/Auxiliary/Build/*
  
    	$path = [System.Environment]::GetEnvironmentVariable("PATH", "User")
    	[Environment]::SetEnvironmentVariable("Path", $path + ";$clpath" + ";$clangpath" + ";$vcvarsallpath", "User")
    	refreshenv
  

- CMake at least version 3.15 or higher

Install Command

• Debian/Ubuntu:

    sudo apt-get install cmake
  

• Windows:

    choco install cmake -y
  

• MacOS:

    brew install cmake
  

- Conan (Also Vcpkg could be used but isn't tested)

Install Command

• Via pip - https://docs.conan.io/en/latest/installation.html#install-with-pip-recommended

    pip install --user conan
  

Note: Check that pip is installed with python3, otherwise there will be errors when configuring the cmake project

• Windows:

    choco install conan -y
  

• MacOS:

    brew install conan
  

C++20 is required since the following features are used in the library or in the client:

• Concepts
• Designated Initializer
• Map.contains
• Using enum construct

To setup the environment on linux ubuntu, there are two scripts that can be used:

1. For installing conan

scripts/conan_install.sh

2. For Installing the required Cpp tools

scripts/required_cpp_tools_install.sh

Optional Tools

This project may also run with other software for static code analysis, documentation and related things. These are optional but they are strongly suggested for development since they improve code quality.

Furthermore, a clang-format file is present the root folder to have a standard formatting for the codebase.

• Doxygen

  Install Command

  • Debian/Ubuntu:

      sudo apt-get install doxygen
      sudo apt-get install graphviz
    

  • Windows:

      choco install doxygen.install -y
      choco install graphviz -y
    

  • MacOS:

      brew install doxygen
      brew install graphviz
    

• ccache

  Install Command

  • Debian/Ubuntu:

      sudo apt-get install ccache
    

  • Windows:

      choco install ccache -y
    

  • MacOS:

      brew install ccache
    

• Cppcheck

  Install Command

  • Debian/Ubuntu:

      sudo apt-get install cppcheck
    

  • Windows:

      choco install cppcheck -y
    

  • MacOS:

      brew install cppcheck
    

• include-what-you-use

  Install Command

  Follow instructions here: https://github.com/include-what-you-use/include-what-you-use#how-to-install

To install these optional tools environment on linux ubuntu the following script can be run:

For Installing the required Cpp tools

scripts/optional_cpp_tools_install.sh

Build

This software uses cmake for generating the build system configuration.

In particular, starting from the root directory, the command to generate the build configuration is:

• Single Configuration Builders (Make, Ninja)

    Single configuration builders won't know about configurations, so cmake has to create different build directories for each one of them.
  
    - Release
    ```Bash
    cmake -S . -B ./build/Release -DCMAKE_BUILD_TYPE:STRING=Release
    ```
  
    - Debug
    ```Bash
    cmake -S . -B ./build/Debug -DCMAKE_BUILD_TYPE:STRING=Debug
    ```
  
    - Release with Debug Symbols
    ```Bash
    cmake -S . -B ./build/RelWithDebInfo
    ```
  
    - Release with Minimum Size
    ```Bash
    cmake -S . -B ./build/MinSizeRel
    ```
  

• Multiconfiguration Builders (Visual Studio, Ninja Multi-Configuration)

    Multiconfiguration builders will work directly in a single build folder and the actual configuration has to be specified at compile time.
    ```Bash
    cmake -S . -B ./build
    ```
  

Then, to actually compile the code, you can use directly cmake via the following commands launched from the root directory:

• Single Configuration Builders (Make, Ninja)

  Single configuration builders won't know about configurations, so the build process has to be launched separately from each build folder.

  • Release

     cmake --build ./build/Release

  • Debug

     cmake --build ./build/Debug

  • Release with Debug Symbols

     cmake --build ./build/RelWithDebInfo

  • Release with Minimum Size

     cmake --build ./build/MinSizeRel

• Multiconfiguration Builders (Visual Studio, Ninja Multi-Configuration)

  Multiconfiguration builders will work directly in a single build folder and the actual configuration has to be specified at build time.

  • Release

     cmake --build ./build -- /p:configuration=release

  • Debug

     cmake --build ./build -- /p:configuration=Debug

  • Release with Debug Symbols

     cmake --build ./build -- /p:configuration=RelWithDebInfo

  • Release with Minimum Size

     cmake --build ./build -- /p:configuration=MinSizeRel

CMake will automatically pick the predefined C++ compiler. On Linux systems you can change it via the "CXX" system variable.

Automatic Tests can be run from the build directly via CMake.

• Single Configuration Builders (Make, Ninja)

  ```Bash cd build/Debug ctest -C Debug ```
• Multiconfiguration Builders (Visual Studio, Ninja Multi-Configuration)

  ```Bash cd build ctest -C Debug ```

This library uses the Catch2 framework for automatic testing.

Continuous Integration

This project uses appveyor for running continuous integration. After every commit, the source code is compiled and tested on the following environments:

• Linux Ubuntu 20.04 with GCC 9.3.0
• Windows Server 2022 with MSBuild VC142

Static Code Analysis and Sanitizers

The cmake configuration supports various tools to be run automatically post build

• What You See is What You Get
• Clang-Tidy
• Cppcheck

To add them to the build process, uncomment the relative options in the root's cmake file from line 22 to 43. These tools must be installed on the develompent system and cmake must be able to find them on the current system path. See the section on optional tools for installation instructions.

Library Documentation

The library supports doxygen for automatically generating code documentation. See the section on optional tools for instructions on installing doxigen on your system. The "doxygen-docs" target will be automatically generated by cmake if the option is uncommented in the root's cmake file on line 33. For example, if you generate a make configuration, the command to create the documentation after the build has been completed would be

cd build/Release
make doxygen-docs

If the process is successfull, the produced documentation will be found in the following directory:

build/Release/html

Test Client

With the library a small sample client is deployed. The following are the possible options:

Usage:
    KVCacheClient [--memCacheMaxSizeMB=<mb>] 
				  [--memoryCacheType=<memType>] 
				  [--storageCacheType=<storageType>] 
				  [--cacheEvictionStrategy=<cacheEviction>] 
				  [--storagePath=<path>] 
				  [--fileLoad=<filePath>] 
    KVCacheClient (-h | --help)
    KVCacheClient --version
 
 Options:
          --memCacheMaxSizeMB=<mb>                  Max size as Integer of the Memory Cache in MB. [Default 1 MB].
          --memoryCacheType=<memType>               Type of MemoryCache: (UNORDERED_MAP | ORDERED_MAP). [Default is UNORDERED_MAP].
          --storageCacheType=<storageType>          Type of StorageCache: (NONE | LINEAR_FILE). [Default is NONE].
          --cacheEvictionStrategy=<cacheEviction>   Algorithm for Cache Eviction when the MemoryCache is full: (FIFO | LIFO | LRU | MRU). [Default is FIFO].
          --storagePath=<path>                      Path of the StorageCache. Default is current directory.
          --fileLoad=<filePath>                     Path of a file to load into the cache. The file will need to have the KV Pairs on consecutive lines
          -h --help                                 Show this screen.
          --version                                 Show version.

In the example folder there are two usefull files. The first is a text file with a set of kv pairs (for the fileLoad option) and the second small bash script for launching the application. In general the full path of the compiled binary will be:

• Single Configuration Builders (Make, Ninja)

  - Release ```Bash /build/Release/src/KVCacheClient ```

  • Debug

     /build/Debug/src/KVCacheClient

  • Release with Debug Symbols

     /build/RelWithDebInfo/src/KVCacheClient

  • Release with Minimum Size

     /build/MinSizeRel/src/KVCacheClient

• Multiconfiguration Builders (Visual Studio, Ninja Multi-Configuration)

  ```Bash /build/src/KVCacheClient ```