Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature Request: Implement MMR Search in Qdrant Database for LangChain #3676

Closed
Kubululo opened this issue Dec 15, 2023 · 3 comments · Fixed by #5467
Closed

Feature Request: Implement MMR Search in Qdrant Database for LangChain #3676

Kubululo opened this issue Dec 15, 2023 · 3 comments · Fixed by #5467
Labels
auto:enhancement A large net-new component, integration, or chain. Use sparingly. The largest features

Comments

@Kubululo
Copy link

Title: Add MMR Search Capability to Qdrant in LangChain

Details:
Requesting the implementation of Maximal Marginal Relevance (MMR) search within the Qdrant database framework in LangChain. MMR search, already present in other vector stores in LangChain, is currently missing in the Qdrant integration. This addition would enhance document retrieval capabilities by providing diverse and relevant results, bringing Qdrant's functionality in line with other supported vector stores in LangChain.

Copy link

dosubot bot commented Dec 15, 2023

🤖

Hello @Kubululo! It's nice to meet you on GitHub. I'm Dosu, a friendly bot here to assist you. While we wait for a human maintainer, I'm here to help you with bug solutions, answer any queries you might have, and guide you on your journey to becoming a contributor. Let's dive into code and collaboration together!

Thank you for your feature request. Implementing Maximal Marginal Relevance (MMR) search within the Qdrant database framework in LangChain is indeed possible and would certainly enhance the document retrieval capabilities of LangChain.

Currently, LangChain's Qdrant integration supports a form of search functionality, as evident in the similaritySearchVectorWithScore method of the QdrantVectorStore class. This method searches for vectors in the Qdrant database that are similar to a given query vector and returns the score and payload (metadata and content) for each similar vector. However, this is not explicitly an MMR search, which is a specific method for diversifying search results.

Here is the code snippet that supports this:

async similaritySearchVectorWithScore(
  query: number[],
  k?: number,
  filter?: QdrantSchemas["Filter"]
): Promise<[Document, number][]> {
  if (!query) {
    return [];
  }

  await this.ensureCollection();

  const results = await this.client.search(this.collectionName, {
    vector: query,
    limit: k,
    filter,
  });

  const result: [Document, number][] = (
    results as QdrantSearchResponse[]
  ).map((res) => [
    new Document({
      metadata: res.payload.metadata,
      pageContent: res.payload.content,
    }),
    res.score,
  ]);

  return result;
}

Source: libs/langchain-community/src/vectorstores/qdrant.ts

To implement MMR search in Qdrant, a similar method would need to be created in the Qdrant vector store class. This method would use Qdrant's search functionality to find the most relevant documents, and then apply the MMR algorithm to rank the results.

As an example, here is how MMR search is implemented in the Supabase vector store:

const resultOne = await vectorStore.maxMarginalRelevanceSearch(
  "Hello world",
  { k: 1 }
);

Source: examples/src/indexes/vector_stores/supabase_with_maximum_marginal_relevance.ts

This feature request is valuable and would bring Qdrant's functionality in line with other supported vector stores in LangChain. However, as a technical support rep, I'm unable to create pull requests or issues. I recommend reaching out to the maintainers of the LangChain repository with this feature request.

I hope this information is helpful. If you have any further questions, please don't hesitate to ask.

Best regards,

Dosu

Sources


This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

@dosubot dosubot bot added the auto:enhancement A large net-new component, integration, or chain. Use sparingly. The largest features label Dec 15, 2023
@Kubululo
Copy link
Author

Kubululo commented Jan 8, 2024

Any update?

@dosubot dosubot bot added the stale Issue has not had recent activity or appears to be solved. Stale issues will be automatically closed label Apr 8, 2024
@dosubot dosubot bot closed this as not planned Won't fix, can't repro, duplicate, stale Apr 15, 2024
@dosubot dosubot bot removed the stale Issue has not had recent activity or appears to be solved. Stale issues will be automatically closed label Apr 15, 2024
@jstjoe
Copy link

jstjoe commented May 11, 2024

I hope this can be reopened and implemented. It seems that Qdrant themselves support it with their client, and LangChain Python supports a number of different flavors of it too: https://api.python.langchain.com/en/latest/vectorstores/langchain_community.vectorstores.qdrant.Qdrant.html#langchain_community.vectorstores.qdrant.Qdrant.amax_marginal_relevance_search

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
auto:enhancement A large net-new component, integration, or chain. Use sparingly. The largest features
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants