Skip to content

Building an AI-powered tool for auto converting audio from lectures/meetings into markdown notes using state-of-the-art large language models from OpenAI

License

Notifications You must be signed in to change notification settings

teyang-lau/YOListenO

Repository files navigation

You Only Listen Once

made-with-python made-with-Markdown Generic badge GitHub license
Open in Streamlit



You Only Listen Once (YOLO) 🧠 is an AI-powered tool by utilizing state-of-the-art AI deep learning models from OpenAI including Whisper and GPT3.5 to automatically convert audio/video lectures and meetings into markdown notes, which can subsequently be copied into most markdown editor and notetaking apps.


YOListenO Web Application

Check out the You Only Listen Once web application hosted on streamlit!


Lecture Video Example

Machine.Learning.Engineering.for.Production.MLOps.Specialization.Course.1.Week.1.Lesson.2.mp4

Video is from DeepLearningAI's youtube channel.


Final Markdown Example (raw markdown is also available in the app!)

Machine Learning Project Life Cycle

Summary

This transcript discusses the major steps of a machine learning project life cycle, including scoping, data collection, model training, deployment, and maintenance.

Main Points

  • The machine learning project life cycle is an effective way to plan out all the steps needed to build a machine learning system and minimize surprises.
  • The major steps of a machine learning project include scoping, data collection, model training, deployment, and maintenance.
  • During the scoping phase, you define the project and decide what to work on.
  • During the data collection phase, you acquire the data you need for your algorithm, define the data, establish a baseline, and label and organize the data.
  • During the model training phase, you select and train the model, perform error analysis, and update the model as needed.
  • During the deployment phase, you deploy the system in production, monitor the system, track the data, and maintain the system.
  • Maintenance may involve going back to perform more error analysis, retrain the model, or update the data.

Action Items

  • Define the project during the scoping phase.
  • Acquire the data needed for the algorithm during the data collection phase.
  • Select and train the model, perform error analysis, and update the model as needed during the model training phase.
  • Deploy the system in production, monitor the system, track the data, and maintain the system during the deployment phase.
  • Go back to perform more error analysis, retrain the model, or update the data during the maintenance phase.

Follow Up Questions

  • What are some best practices for data collection?
  • How do you select and train the model?
  • What are some common challenges during the deployment phase?
  • How do you monitor and maintain the system after deployment?
  • How do you update the model and data during the maintenance phase?

Potential Arguments

  • The machine learning project life cycle may not be applicable to all machine learning projects.
  • The life cycle may not account for unexpected challenges or changes in the project.
  • The life cycle may not address ethical considerations in machine learning.

Meeting Video Example (only audio)

Effective.Meetings_.Simulated.Exercise.for.Chairing.Minute.Taking.mp4

Audio is from Apropos Productions Ltd.'s youtube channel.


Final Markdown Example (raw markdown is also available in the app!)

Date:

Attending

Announcements

  • Frank Lyons is running late
  • Gary Cope and Karl Madden have emailed apologies
  • Joey Ballon Wayza is struggling with personal issues

Agenda

Agenda 1 — Car Parking Issues

  • Lucy Strokes highlights problems with staff arguing over spaces
  • Suggestion to send an email to staff informing them that there are only five spaces belonging to the company
  • Consider priority parking for those who need their cars during work hours, such as the sales team
  • Sue and Jason to be allocated the two remaining spaces, with Jason's car taking up two spaces
  • Three spaces reserved for clients and visitors

Agenda 2 — Staff Morale

  • Concerns about low sales figures and increased sick leave
  • Lack of training and effective appraisals
  • Hold a separate meeting to discuss the predominant reason for low morale and to work on tackling it
  • Hold a team challenge or awards night to boost morale
  • Send out a questionnaire to gather feedback from staff

Agenda 3 — Technical and Training Issues

  • Technical problem that occurred on Friday caused by an external energy issue
  • Pay the electricity bill on time
  • Training issue to be discussed in the next two weeks
  • Report from Gary shows that they had a tough year but are doing well in contract sales
  • Down on revenue and handset sales abroad

Action Items

  • Designate priority parking spaces and revoke parking privileges for those who park inconsiderately
  • Hold a separate meeting to discuss the predominant reason for low morale and to work on tackling it
  • Send out a questionnaire to gather feedback from staff
  • Pay the electricity bill on time
  • Discuss training issue in the next two weeks

Project Directory Structure

.
├── images              <- storing images
├── notebooks           <- notebooks for explorations / debugging / training
├── samples             <- storing sample audio and videos
├── scripts             <- all source code, modules and scripts
├── requirements.txt    <- installing dependencies   
└── streamlit_app.py    <- streamlit deployment script

About

Building an AI-powered tool for auto converting audio from lectures/meetings into markdown notes using state-of-the-art large language models from OpenAI

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Languages