LlmMemory::Hippocampus

LlmMemory::Hippocampus Class

The LlmMemory::Hippocampus class provides methods to store and query documents. It supports chunking of large documents and uses different types of storage and embedding techniques.

Usage

The following is an example of how to use the LlmMemory::Hippocampus class.

# Initialize the Hippocampus
hippocampus = LlmMemory::Hippocampus.new(
  embedding_name: :openai, 
  chunk_size: 1024, 
  chunk_overlap: 50, 
  store_name: :redis, 
  index_name: "my_index"
)

# Define some documents
documents = [
  {
    content: "This is the first document. It talks about Ruby programming.",
    metadata: {title: "Ruby Programming", author: "John Doe"}
  },
  {
    content: "This is the second document. It talks about Python programming.",
    metadata: {title: "Python Programming", author: "Jane Doe"}
  }
]

# Memorize the documents
hippocampus.memorize(documents)

# Query the documents
results = hippocampus.query("Ruby programming", limit: 5)

# Print the results
results.each do |result|
  puts "Content: #{result[:content]}"
  puts "Metadata: #{result[:metadata]}"
  puts "---"
end

# Forget all the documents
hippocampus.forget_all

Class Initialization

initialize(embedding_name: :openai, chunk_size: 1024, chunk_overlap: 50, store_name: :redis, index_name: "llm_memory") -> Hippocampus

Creates a new instance of the LlmMemory::Hippocampus class.

Parameters:

• embedding_name (Symbol): The name of the embedding to be used. The default is :openai.
• chunk_size (Integer): The size of chunks into which documents will be divided if their length exceeds this value. The default is 1024.
• chunk_overlap (Integer): The overlap size between chunks. The default is 50.
• store_name (Symbol): The name of the store to be used. The default is :redis.
• index_name (String): The name of the index in the store. The default is "llm_memory".

Raises an error if the specified embedding or store is not found.

Instance Methods

validate_documents(documents) -> nil

Validates the format of the documents.

Parameters:

• documents (Array[Hash]): An array of hashes where each hash represents a document with :content (String) and :metadata (Hash).

Raises an error if the documents do not meet the required format.

memorize(docs) -> nil

Stores the documents after validating, chunking, and adding vectors to them.

Parameters:

• docs (Array[Hash]): An array of hashes where each hash represents a document with :content (String) and :metadata (Hash).

query(query_str, limit: 3) -> Array[Hash]

Queries the store and returns the results.

Parameters:

• query_str (String): The query string.
• limit (Integer): The maximum number of results to return. The default is 3.

Returns an array of hashes where each hash represents a document with :content (String), :metadata (Hash), and a :vector.

forget_all -> nil

Drops the index, effectively deleting all stored data.

make_chunks(docs) -> Array[Hash]

Divides documents into chunks if their content length exceeds the chunk_size.

Parameters:

• docs (Array[Hash]): An array of hashes where each hash represents a document with :content (String) and :metadata (Hash).

Returns an array of hashes where each hash represents a document with :content (String) and :metadata (Hash). The content of each document may represent a chunk of the original document if its length exceeded the chunk_size.

Exceptions

The LlmMemory::Hippocampus class will raise exceptions if certain conditions are not met:

• If the embedding or store specified during initialization is not found, it will raise an error message: "Embedding '#{embedding_name}' not found." or "Store '#{store_name}' not found.", respectively.
• If the documents do not meet the required format during the validate_documents or memorize method, it will raise an error message: "Your documents need to have an array of hashes (content: string and metadata: hash)".

Further Information

The LlmMemory::Hippocampus class is part of the LlmMemory module, which is designed for managing memory in a language model. It provides a unified interface for various embeddings and stores, making it easy to change these components as needed. The class uses the Singleton design pattern to ensure that only one instance of each component is created.

The LlmMemory::Hippocampus class uses the Strategy design pattern for embeddings and stores, allowing these components to be easily swapped out for different implementations. The specific embedding and store to be used are specified during class initialization and can be changed dynamically if needed.