RAGSearchUnity: Semantic search in Unity!

RAGSearchUnity allows to implement semantic search within the Unity engine.
It is a Retrieval Augmented Generation (RAG) system empowered by the best open-source retrieval models available.
RAGSearchUnity is built on top of the awesome sharp-transformers and usearch libraries.

_{At a glance  •  How to help  •  Setup  •  How to use  •  Examples  •  License}

At a glance

• 💻 Cross-platform! Windows, Linux and macOS
• 🏠 Runs locally without internet access. No data ever leave the game!
• ⚡ Blazing fast search with Approximate Nearest Neighbors (ANN)
• 🤗 Support of the best retrieval models
• 🔧 Easy to setup and use
• 💰 Free to use for both personal and commercial purposes

Tested on Unity: 2021 LTS, 2022 LTS, 2023

How to help

• Join us at Discord and say hi!
• ⭐ Star the repo and spread the word about the project!
• Submit feature requests or bugs as issues or even submit a PR and become a collaborator!

Setup

• Open the Package Manager in Unity: Window > Package Manager
• Click the + button and select Add package from git URL
• Use the repository URL https://github.com/undreamai/RAGSearchUnity.git and click Add

How to use

RAGSearchUnity implements a super-fast similarity search functionality with a Retrieval-Augmented Generation (RAG) system.
This works as follows.

Building the data You provide text inputs (a phrase, paragraph, document) to add to the data
Each input is split into sentences (optional) and encoded into embeddings with a deep learning model.

Searching You can then search for an query text input.
The input is again encoded and the most similar text inputs or sentences in the data are retrieved.

To use search:

• create an empty GameObject for the embedding model 🔍.
  In the GameObject Inspector click Add Component and select the Embedding script).
• select the model you prefer from the drop-down list to download it (bge small, bge base or MiniLM v6).

In your script you can then use it as follows 🦄:

using RAGSearchUnity;

public class MyScript : MonoBehaviour
{
  public Embedding embedding;
  SearchEngine search;

  void Game(){
    ...
    string[] inputs = new string[]{
      "Hi! I'm a search system.", "the weather is nice. I like it.", "I'm a RAG system"
    };
    // build the embedding
    EmbeddingModel model = embedding.GetModel();
    search = new SearchEngine(model);
    foreach (string input in inputs) search.Add(input);
    // get the 2 most similar phrases
    string[] similar = search.Search("hello!", 2);
    // or get the 2 most similar sentences
    string[] similarSentences = search.SearchSentences("hello!", 2);
    ...
  }
}

• Finally, in the Inspector of the GameObject of your script, select the Embedding GameObject created above as the embedding property.

You can save the data along with the embeddings:

search.Save("Embeddings.zip");

and load them from disk:

SearchEngine search = SearchEngine.Load(model, "Embeddings.zip");

You can also specify the delimiters to use or no splitting:

// use ".", "!", "?" as delimiters
search = new SearchEngine(model, ".!?");
// don't split sentences
search = new SearchEngine(model, null);

If you want to manage multiple independent searches, RAGSearchUnity provides the MultiSearchEngine class for ease of use:

MultiSearchEngine multisearch = new MultiSearchEngine(model);

// add a text for a specific search
multisearch.Add("hi I'm Luke", "search1");
multisearch.Add("Searching, searching, searching...", "search1");
multisearch.Add("hi I'm Jane", "search2");

// search for similar text in all searches
string[] similar = multisearch.Search("hello!", 2);
// search for similar texts within a specific search
string[] similar = multisearch.Search("hi there!", 1, "search1");

That's all ✨!

Examples

The HamletSearch sample contains an example search system for the Hamlet play 🎭. To install the sample:

• Open the Package Manager: Window > Package Manager
• Select the RAGSearchUnity Package. From the Samples Tab, click Import next to the sample.

The sample can be run with the Scene.unity scene it contains inside their folder.
In the scene, select the Embedding GameObject and download one of the models (Download model).
Save the scene, run and enjoy!

License

The license of RAGSearchUnity is MIT (LICENSE.md) and uses third-party software and models with MIT and Apache licenses (Third Party Notices.md).