Frontend:
- Vue & Nuxt
Backend:
- Flask
- FAISS vector DB storage
- custom script to scrape URLs on a website
- Scraping
The first task is to scrap all data available on a website. We took an approach of looking at sitemap.xml and downloading all pages followed by cleaning up and storing the images.
- Generating Embeddings
We are using all-mpnet-base-v2 to generate embeddings and storing it in FAISS Vector storage database.
- Flask Backend
When a query comes we use it to generate vector embedding and keyword based search and figure out the relevant parts.
- LLM
We use Mistral AI's latest LLM that came just a few days back. We tried 13B-Llama2, but this was faster on our machine.
We used this repository to easily run, look at performance and create an API for the model.
- Frontend
We did not have too much time for making it look good, and adding the features that we wanted to, but it is a functioning reponsive chatbot app.
- Everything runs locally (from generating embeddings, to storing, and also running LLM)
- We have performed OCR on images and added also added it into generating embeddings.
- Alongside using similarity search, we wrote a simple token based keyword search to answer direct questions, which otherwise could be missed.
- Run an API inference of any LLM you wish (we used oobabooga's text-generation-webui with Mistral-AI-7B)
embeddings/data_get.pyhas the availibility to change the website URL and download all pages on that website. It also performs OCR on images and stores that.Create_Embeddings/generate_embeddings.pywould generate embeddings for the downloaded website pages. Followed bymerge_dbs.pywhich merges various faiss databases which were created, while preserving true source of data.- On starting
Create_Embeddings/flask_endpoint.pyit creates an API endpoint where on querying a question it returns relevant parts. - Go to
/frontendand install npm packagesyarn installfollowed byyarn dev.
- Divyansh Singh
- Param Gandhi
- Ritwik Sharma
- Vimarsh Shah
