Hybrid search with experimental features using Custom Embedding API as an embedder

[miguelisidoro]

Hello,

We are thinking on implementing a custom REST API that serves as an embedder. but there is little documentation unfortunately.

Can you confirm that with a custom REST API as an embedder:

• When indexing the content, we don't supply the vector but the vector is generated by the embedder
• When searching, we don't need to supply the vector and the vector for the "q" parameter is generated by the embedder
• Indexing times are much faster because Meilisearch is doing the work by calling our custom API to generate the embedding vector without any time penalty instead of us having to generate the embedding vector that will slow down the indexing process

One of our concerns is that we will need to rebuild our existing search index and if we use a user provided approach, for each document, we will need to call the Azure Open AI embedding API to generate the embedding vector that will be set in the _vectors field of the index.

If on the other hand, use a REST API approach using a custom REST API that calls Azure Open AI Embedding API, we don't need to call the Azure Open AI Embedding API directly and indexing times are much faster.

Can you confirm what I said above?

About the format of the response required by Meilisearch when using a custom REST API as an embedder. Can you give us an example of a response that we would have to supply so that it can be processed correctly by Meilisearch?

About the request itself, how does Meilisearch do a request to the custom REST API as an embedder? How do I know how data is passed to the embedder? Is it a GET or a POST (I assume POST)? Can you give us an example of a request in the following situations?

• When content is indexed
• When performing a search

Another thing: in https://www.meilisearch.com/docs/learn/experimental/vector_search#generate-auto-embeddings, REST Option, it is referred the following:

"model is a mandatory field indicating a compatible model.

documentTemplate is an optional field. Use it to customize the data you send to the embedder. It is highly recommended you configure a custom template for your documents."

What are we supposed to set in the model field?

documentTemplate although optional, is recommended. Are there any examples of document templates in the documentation or that you can supply?

Thanks

[dureuill]

Hello, @miguelisidoro, 👋

Your setup does not seem 100% clear to me. To be clear, you would implement a custom REST server that would serve as a proxy to Azure OpenAI, is that correct?

When indexing the content, we don't supply the vector but the vector is generated by the embedder

Correct

When searching, we don't need to supply the vector and the vector for the "q" parameter is generated by the embedder

Correct

IIndexing times are much faster because Meilisearch is doing the work by calling our custom API to generate the embedding vector without any time penalty instead of us having to generate the embedding vector that will slow down the indexing process

I'm not sure I understand this affirmation. Under the assumption that Meilisearch is going to call your custom API that calls the Azure OpenAI API, why do you expect that to be faster than you directly calling the Azure OpenAI API? It feels like I'm missing a piece here.

If on the other hand, use a REST API approach using a custom REST API that calls Azure Open AI Embedding API, we don't need to call the Azure Open AI Embedding API directly

Correct, the Azure OpenAI embedding API will be called by your REST proxy which will be called by Meilisearch, if I understand your setup correctly

and indexing times are much faster.

Why?

About the format of the response required by Meilisearch when using a custom REST API as an embedder. Can you give us an example of a response that we would have to supply so that it can be processed correctly by Meilisearch?

• The response must be in the JSON format
• If the request was an array of texts to embed, then the response is expected to contain an array of objects containing the embeddings
• If the request was a single text to embed, then the response is expected to contain a single object containing the resulting embedding
• The number of embeddings in the response is expected to be the same as the number of texts in the request
• Other than that, the format is rather flexible and can be specified using the pathToEmbeddings and embeddingObject parameters in the embedder configuration.
• You can probably pass through the response from AzureOpenAI unmodified.
• Let's assume that the parameters of the embedders are as the following: pathToEmbeddings = ["A", "B", "C"] and embeddingObject = ["D", "E", "F"]. Then the response is parsed in the following way:
  1. Look at the fields specified by the array of strings in pathToEmbeddings. In the example Meilisearch will lookup at field A.B.C from the response.
  2. The second step depends on the value of inputType:
  • if inputType == "text", then Meilisearch will look at the fields specified by the array of strings in embeddingObject, in the example A.B.C.D.E.F. Fshould be the embedding (array of floats)
  • if inputType == "textArray", then C must be an array, and Meilisearch will look into each item of this C array. In each item, Meilisearch will lookup D.E.F. F should be the embedding (array of floats)

Looking at a sample response from OpenAI Azure in their documentation:

{
  "object": "list",
  "data": [
    {
      "object": "embedding",
      "embedding": [
        0.018990106880664825,
        -0.0073809814639389515,
        .... (1024 floats total for ada)
        0.021276434883475304,
      ],
      "index": 0
    }
  ],
  "model": "text-similarity-babbage:001"
}

then you should have:

{
   "inputType": "textArray",
   "pathToEmbeddings": ["data"],
   "embeddingObject": ["embedding"]
   // other parameters elided
}

for correctly parsing the response.

About the request itself, how does Meilisearch do a request to the custom REST API as an embedder? How do I know how data is passed to the embedder?

• Meilisearch makes a POST call to the configured url with a JSON payload as body
• Meilisearch sets the Content-Type header to application/json
• Meilisearch sets the Authorization header to Bearer ${api_key} where ${api_key} is the value of the apiKey field in the configuration
• The JSON payload is constituted of two elements:
  1. The value of the query field from the configuration, copied verbatim.
  2. The text(s) to embed, injected in the JSON payload at the path described by the array of string in inputField.

So, assuming that your REST proxy would behave exactly like Azure OpenAI expected that it would take an API key as its Authorization header rather than a non-standard header like Azure does, then the corresponding embedding parameters could be:

"apiKey": "<API-key-of-your-custom-REST-embedder>",
"inputField": ["input"],
"inputType": "textArray",
"query": {}

Can you give us an example of a request in the following situations?

The request will look the same in both situations, except that the text to embed will be sourced from:

1. directly the query when embedding for a search
2. the result of rendering the documentTemplate for the document when embedding for a document

model is a mandatory field indicating a compatible model.

This is an unfortunate error in the current documentation. query.model is not required for REST embedders. As query is copied verbatim when making requests to url, you should only put there fields that make sense for your embedder.

documentTemplate although optional, is recommended. Are there any examples of document templates in the documentation or that you can supply?

I'm not sure about the current documentation, but there is a recent Docs branch that adds an example, see here

This new branch also contains more explanation about the documentTemplate. The gist of it is to include fields from your documents that exist in all documents and that are semantically relevant, and truncate them if they might be too long (You should try to keep your template under 40 to 50 semantically relevant words for best results).

So, to recap, the configuration of your embedder would looks something like the following:

{
  "source": "rest",
  "url": "url-to-your-REST-proxy",
  "apiKey": "api-key-to-your-REST-proxy",
  "documentTemplate": "something containing relevant {{doc.field}}s, truncated if necessary",
  "inputField": ["input"],
  "inputType": "textArray",
  "query": {},
   "pathToEmbeddings": ["data"],
   "embeddingObject": ["embedding"]
}

[miguelisidoro]

Hello @dureuill,

Thanks for response.
I get your doubts, sorry for not being clear.
Your response is big and I am still in middle of processing it :).

Our REST API, indeed would act as a proxy to Azure Open AI.

About the index time, my thoughts were the following:

• If we use user provided approach, in the indexing process, we are responsible to call Azure Open AI and we must wait for the Azure Open AI Embedded API response to have the content indexed. In a rebuld index process where we are indexing the whole search index, this will take more time since we have this call to Azure Open AI Embedding API
• If by the other hand, we supply a proxy API, and assuming the call to our proxy API is asynchronous, the time to rebuild the index would be inferior in synchronous processing time.

I see that according to your response, it will take more time regardless because Meilisearch will call our API that will call Azure Open AI Embedding API which is slower than calling Azure Open AI directly.

Thanks

[dureuill]

Hello again,

it will take more time regardless because Meilisearch will call our API that will call Azure Open AI Embedding API which is slower than calling Azure Open AI directly.

the difference might be negligible though, you need to test this and see if you are satisfied with the indexing performance. Using a REST embedder is definitely going to be more convenient than using a userProvided one. In particular, any change to the document that modifies the document template is guaranteed to trigger a regeneration of the embedding, whereas you have to change the vector manually when using a user provided embedder.

[miguelisidoro]

Hello @dureuill ,

We are not going to implement a custom embedding API to call Azure Open AI Embedding API.
Instead, we are going to move forward with the userProvided option.

Thanks