RAMI: Robust Adaptive Memory Infrastructure

RAMI is a sophisticated memory management system designed to optimize memory usage through advanced techniques like deduplication, compression, and intelligent allocation. It integrates deeply with the operating system via a kernel module and provides a comprehensive suite of tools for management, monitoring, and analysis.

Features

• Memory Deduplication: Identifies and eliminates redundant data blocks in memory, reducing overall memory footprint.
• Memory Compression: Compresses memory pages to store more data in less physical RAM, improving system efficiency.
• Adaptive Memory Allocation: Intelligently allocates and manages memory resources to optimize performance and resource utilization.
• Kernel-level Integration: Utilizes a Linux kernel module for low-level memory interception and efficient operation.
• Security Features: (Optional) Includes encryption, fingerprint guarding, and hash salting for enhanced data security.
• Command-Line Interface (CLI): ramictl provides a powerful interface for system configuration, control, and status monitoring.
• Prometheus Exporter: Exposes detailed system metrics for real-time monitoring and integration with Prometheus.
• Web Dashboard: A user-friendly web interface for visualizing memory usage, performance metrics, and system status.

Components

The RAMI system is composed of several interconnected components:

• RAMI Daemon (rami_daemon): The core background service responsible for executing memory deduplication, compression, and allocation policies.
• RAMI Kernel Module (rami_module.ko): A loadable kernel module that intercepts memory operations and redirects them to the RAMI system for processing.
• RAMI CLI (ramictl): A user-space command-line tool for interacting with the RAMI daemon, configuring settings, and querying system status.
• Prometheus Exporter (rami_exporter): A telemetry component that collects and exposes RAMI's operational metrics in a Prometheus-compatible format.
• Dashboard API: A Flask-based backend serving data to the web dashboard.
• Web UI: A web-based graphical interface for monitoring and managing the RAMI system.

Setup and Installation

To set up and install the RAMI system, execute the setup_rami.sh script. This script will:

1. Update system packages and install necessary build tools and libraries (e.g., build-essential, libssl-dev, lz4, zstd, python3, pipenv, libmicrohttpd-dev, bpfcc-tools, linux-headers, libpolly-18-dev).

2. Set up the BCC (BPF Compiler Collection) environment.

3. Install Python dependencies for the dashboard API using pipenv.

4. Compile and load the RAMI kernel module.

5. Compile core RAMI libraries and user-space applications (daemon, CLI).

6. Create a mount point for the RAMI memory pool at /mnt/rami_pool.

   Important: External Storage Configuration For optimal performance and persistent storage of the RAMI memory pool, it is highly recommended to dedicate an external block device (e.g., a separate partition or disk) and configure it to be automatically mounted at /mnt/rami_pool at system boot.

   Steps:

   1. Identify your block device: Use lsblk or fdisk -l to find the device you wish to dedicate (e.g., /dev/sdb1).
   2. Format the device (if necessary): For example, to format with ext4: sudo mkfs.ext4 /dev/sdb1
   3. Get the UUID of the device: sudo blkid /dev/sdb1 (Note down the UUID).
   4. Edit /etc/fstab: Add an entry to /etc/fstab to ensure the device mounts automatically.

      UUID=<YOUR_DEVICE_UUID> /mnt/rami_pool ext4 defaults 0 2
      

      Replace <YOUR_DEVICE_UUID> with the actual UUID obtained in step 3.
   5. Mount the device: After editing /etc/fstab, run sudo mount -a to mount all entries, or reboot your system.

./setup_rami.sh

Launching the RAMI System

After installation, you can launch the RAMI system components. Note that the RAMI Daemon and CLI (ramictl) require root privileges to interact with the kernel module and /dev/rami.

To launch all components (Prometheus exporter, Dashboard API, and RAMI Daemon) in the background:

# Ensure the Dashboard API's virtual environment is correctly set up (if issues arise)
# cd user/dashboard/api && rm -rf venv && python3 -m venv venv && ./venv/bin/pip install flask
# Then, launch the system:
sudo ./launch_rami.sh

Alternatively, for a quick start without recompilation (assuming components are already built), you can use start_rami.sh or start_rami_system.sh. Remember to run the daemon and CLI with sudo if starting them individually.

Note on Dashboard API: If you encounter issues with the Dashboard API (e.g., ModuleNotFoundError), ensure its Python virtual environment is correctly set up by navigating to user/dashboard/api and running: rm -rf venv && python3 -m venv venv && ./venv/bin/pip install flask

Configuration

The RAMI system can be configured via the config/rami.conf file. Key configurable options include:

• log_level: Set the logging verbosity (e.g., info, debug).
• deduplication.enabled: Enable or disable memory deduplication.
• compression.enabled: Enable or disable memory compression.
• security.enabled: Enable or disable security features.
• telemetry.port: Configure the port for the Prometheus exporter.

Usage

Command-Line Interface (ramictl)

The ramictl utility provides various commands to interact with the RAMI system. (Further details on ramictl commands will be provided in the docs/api_reference.md and docs/architecture.md.)

Web Dashboard

Access the web dashboard by navigating to the appropriate URL (typically http://localhost:5000 if running locally, as per Flask's default).

Directory Structure

• config/: Configuration files for RAMI.
• core/: Core functionalities including allocator, compression, deduplication, libraries, security, and telemetry.
• daemon/: RAMI Daemon source and binary.
• docs/: Project documentation.
• kernel/: Linux kernel module source.
• logs/: Log files generated by RAMI components.
• tests/: Unit, integration, and fuzz tests.
• user/: User-facing components including the web dashboard and ramictl CLI.