Sahayak
: Virtual Customer Service is a web-based application designed to streamline customer support for Amazon
. It replaces traditional chatbots by taking user input through voice and providing accurate responses based on the query. With support for multiple languages, it offers a simple and efficient way to help customers around the world.
Youtube Video Link: https://youtu.be/kjU2W9N35rs
Website Link: https://vcs-hackon-kohl.vercel.app/
About us Link: https://vcs-about-us.vercel.app/
Our application allows users to ask queries directly through voice input instead of typing, providing a more natural and efficient interaction compared to traditional chatbots.
Users can receive assistance in their native language simply by changing the language preference, making the service accessible to a global audience.
Users can quickly get answers by selecting a relevant category on the main page. Categories include AWS support, online retail, refund requests, Prime-related queries, and general inquiries.
Users can ask questions related to previous queries during the same call. They can also seek clarification on responses provided by our AI voice assistant, ensuring comprehensive and relevant answers.
Our model offers different language accents based on user preference, enhancing the clarity and understanding of the responses.
Our model generates responses based on the user's query, taking into account the user's current mood and sentiment.
This project is divided into two main parts: the frontend and the backend.
The frontend is developed using ReactJS
and Redux Toolkit
. It is structured as follows:
The backend is developed using Python with FastAPI. It handles the API endpoints, voice processing, and response generation. The structure is as follows:
Key Features:
-
Voice to Text Conversion and Sentiment Analysis:
Utilizes OpenAI's Whisper for converting user voice input into text and analyzing sentiment to understand the user's query context. -
Query Similarity Search:
Compares the current user query with previous queries to find the top similar ones using similarity search algorithms. -
Top K Similar Queries:
Retrieves the top K similar queries based on the similarity score to find the most relevant past interactions. -
Integration with LLM Model:
Fetches answers from a service database for the top similar queries and uses a Large Language Model (LLM) to generate responses. -
Text to Audio Conversion:
Uses Amazon Polly to convert the text response into audio, delivering it based on the user's language and accent preference.
Follow these steps to set up and run the frontend of the project:
Clone the project repository to your local machine. Use the following command in your terminal:
git clone https://github.com/Tushar-022/vcsHackon/tree/master/frontend.git
cd hackon
npm install
Create a .env file in the root directory of the frontend project and add the following environment variables:
REACT_APP_API_KEY=your-api-key
REACT_APP_AUTH_DOMAIN=your-auth-domain
REACT_APP_PROJECT_ID=your-project-id
REACT_APP_STORAGE_BUCKET=your-storage-bucket
REACT_APP_MESSAGING_SENDER_ID=your-messaging-sender-id
REACT_APP_APP_ID=your-app-id
REACT_APP_MEASUREMENT_ID=your-measurement-id
REACT_APP_EMAIL_CONFIRMATION_REDIRECT=your-email-confirmation-redirect
FAST_REFRESH=true
Finally, start the development server by running:
npm start
This command will start the application and open it in your default web browser at http://localhost:3000.
Follow these steps to set up and run the backend of the project:
Clone the project repository to your local machine. Use the following command in your terminal:
git clone https://github.com/AnupamMittal-21/HackOn.git
cd vcsHackon/backend
Setting up a virtual environment helps to isolate the project dependencies from your global Python environment:
python -m venv venv
source venv/bin/activate # On Windows, use `venv\Scripts\activate`
Install all the necessary dependencies listed in the requirements.txt file:
pip install -r requirements.txt
Finally, start the development server by running:
python main.py
- React.js
- Redux Toolkit
- react-media-recorder
- react-formik
- Firebase
- Python
- FastAPI
- Whisper OpenAI (voice-to-text conversion and sentiment analysis)
- LLM (Large Language Model, for natural language processing)
- Amazon Polly (text-to-audio conversion)
- Chroma DB (used for embeddings or related functionalities)
In the future, we plan to integrate video solutions within our application. Users will be able to access video tutorials related to their queries. They can pause the video at any point and ask the virtual assistant about specific steps being implemented at that timestamp. Here’s how it will work:
-
Video Playback and Interaction: Users can view instructional videos relevant to their queries within the application.
-
Timestamp Interaction: Users can pause the video and ask the virtual assistant questions about the specific steps demonstrated at that moment.
-
Visual and Text Integration: Implement image and video recognition capabilities to handle customer queries that involve visual data.
-
Enhanced Personalization: Improve personalization algorithms by incorporating machine learning techniques that adapt to user preferences and behaviors in real-time.
Sahayak, developed by Team Dynamo, enhances customer support through state-of-the-art AI and NLP technologies. It addresses current challenges and paves the way for future innovations in virtual customer service.
For more information about our team members, please visit our LinkedIn profiles: