YouTube Conversationalist Google Chrome extension for YouTube videos that produces summaries and a chat discussing the video.
The aim is to optimize video content consumption by providing concise summaries and key point highlights. This application is particularly useful for users who need to quickly grasp the essence of lengthy videos without watching them in their entirety.
Source Video used for below demos: Formula 1 Cars
Multiple Summary Versions
Chat Bot Feature
- Video Summarization: Automatically summarizes videos and highlights the key points discussed, allowing users to quickly understand the main content without viewing the entire video.
- Interactive Chat System: Users can interact with the video content by asking questions at specific timestamps. This feature is under development and aims to make video content more accessible and engaging.
- Audio Extraction: The application fetches the audio track from the video using the YouTube API.
- Audio to Vector Conversion: Using Deepgram, the audio is converted into textual data transcripts with timestamps for every word.
- Vector Search: The embedded vectors are then generated by together.ai and stored and indexed in MongoDB Atlas cloud database instance.
- Retrieving Information: Utilizes Retrieval-Augmented Generation (RAG) AI techniques to query the MongoDB Atlas Vector search and fetch responses to user prompts based on the video content.
- User Interface: Finally, the frontend was built and hosted with Streamlit that integrates the model with the database for querying video information.
- Skill Level Summary and Chat: Choose the depth of analysis and skill level you'd like in the summary and chat explanations
- Emotion Analysis: Identify and respond to the emotional content of the video