Best pipeline for similarity question search.

[JimReno]

Thank you for all the Haystack contributors. It is a really great job!
I want to build a QA system that searches questions based on the user's query. In other words, the QA system prepares a question and an answer as a key-value pair. The retriever module only needs to find the top-K match question and return the question-answer pair to the user. Is there any practical pipeline I can refer to?
I have read the discussion2328. It uses a ranker to score two sentences. And Document Search Method but I still haven't dived into them. Any ideas or suggestions are appreciated!

[julian-risch]

Hi @JimReno what you describe sounds like our FAQPipeline. Here is a tutorial about it: https://haystack.deepset.ai/tutorials/existing-faqs Please have a look and let me know if it fits your use case. We also have a blogpost about FAQ Search but because of the fast development of Haystack be aware that some code examples from the blogpost are outdated by now so I'd suggest you focus more on the tutorial than the blogpost.

[JimReno]

Thank you very much @julian-risch . I have used the tutorial4 for FAQ task. Sorry for that I didn't read the document carefully. Also, I have some questions to bother you.
1、I have installed cuda11.3 to support GPU. But the running outputs seems not use gpu when it is inferencing. (Please see the picture below)

2、I have tested the covid datasets and it worked well. So I tried to build my own FAQ system using Chinese QA pairs. I used the default all-MiniLM-L6-v2 model and load the csv dataset locally. The Dataset has only 80 QA pairs but it worked very bad. Any idead about fixing this issue? I have tried paraphrase-multilingual-MiniLM-L12-v2 model but it seems not working correctly.
With great respect to you! Thanks for the great work you have done for haystack.

[julian-risch]

You're welcome! If you are using Chinese data, you could try to find a model specifically trained for Chinese. For example this model here: https://huggingface.co/uer/sbert-base-chinese-nli or one of the other 300 Chinese models. Here is a search for models on the model hub with a language filter: https://huggingface.co/models?language=zh&sort=downloads

The problem that the GPU is not used probably has nothing to do with Haystack. In python, you could run the following to see whether pytorch can access the GPU:

import torch
torch.cuda.is_available()

I assume this will return False for you?

[JimReno]

Thank you @julian-risch . Your advice saved me! Now I use sentence-transformers/distiluse-base-multilingual-cased for question embedding. It seems to have a nice result. The dataset I used is very small. So I decide to finetune the model later.

A little more help please. Now I am going to buid a REST API for the FAQ system. It seems Haystack has already privde a tutorial on REST. But I want to build the FAQ system with out docker. I checked the .yaml config on issue1361 but it won't work.
Maybe the option1 in REST API is a alternative way. I still can't figure out how to write this code. Sorry that I am a newbie in QA domain.
Thank you anyway. You have helped me so much!

[JimReno]

Now I have built the FAQ system using the tutorial on the official site. I built a REST API using Flask instead of FastAPI. The inference time cost about 350ms.

The next step is enabling Pytorch GPU support and finetune the multilingual model on my own dataset. Thank you @julian-risch . I've learned a lot during the deployment of Haystack. ;)

[julian-risch]

Great to hear that and good luck with your project! If you want, you can mark one of the answers in this discussion. Thereby the question is shown as "answered".