Created by Dylan Klein
This prototype was built in order to automatically generate subtitles for videos (intended for editing in Final Cut Pro X). The aim is to save a significant amount of time during the editing process. The subtitles generated need to be engaging, and for this reason, YouTube closed captions are not suitable.
This demo was built in collaboration with the YouTube channel DarrenLevyOfficial (show title: "Funny Uber Rides").
This demo has the following requirements:
- Subtitles MUST be available to be edited in Final Cut Pro X.
- Subtitles MUST start and stop during the spoken word period to which the text belongs.
- Subtitles MUST appear near the speaker's face in the video.
- Process MUST save a significant amount of time and be easy for a video editor to use.
This code was written in Python 3.7.4. Dependencies are:
ffmpeg(ffmpeg needs to be installed on the OS - not a Python install, eg.$ brew install ffmpeg)noisereducegoogle-cloud-speech(May need to run:$ pip install --upgrade google-cloud-speech)tkmessageboxpillowpandas
-
Run the following command to install (most of) the above dependencies:
$ pip install -r requirements.txt -
Then run the following command to install
ffmpeg:$ brew install ffmpeg -
Create a Google Cloud Platform account, activate the Speech-to-Text API, save your API credentials as:
.creds/google_creds.json -
To execute the program, simply run:
$ python __init__.py
The program has the following inputs:
- Video file (.mp4 or .m4v formats)
- User to select number of speakers in video clip
The program has the following outputs:
- A Comma Separated Values file (.csv) - displaying the generated captions
- A Final Cut Pro XML file (.fcpxml) - to be opened in Final Cut Pro X
The following diagram displays the overall architecture behind the prototype:
Navigate to the example directory to view example input and output files.
Navigate to the prototyping directory to view the Jupyter notebook files which helped me to prototype various components of the application.