FabBits

FabBits is a standalone cross-platform software capable of finding certain interesting bits from movies/shows, soccer, and basketball. Following are the things it will be able to detect -

• Action sequences in movies/shows - ✅
• Summary of movies/shows - ✅
• Actor-specific scenes in movies/shows - ✅
• Jokes in sitcoms - ✅
• Slo-mos in Sports - ❌
• Goals in Soccer - ✅
• Goal misses in Soccer - ⭕
• Three pointers in Basketball - ✅

Requirements

You need the following things to run FabBits -

1. Python3
2. OpenCV - Used for image and video processing
3. Moviepy - Used for video editing and audio processing
4. PyQt5 - Used to make the GUI
5. Scipy - Used for audio processing
6. Tesserocr - Used for, well, OCR
7. Pillow - Used to preprocess images for OCR

The python dependencies can be installed by running -

pip3 install scipy
pip3 install opencv-python
pip3 install moviepy
pip3 install pyqt5
pip3 install Pillow
pip3 install tesserocr

or if you are the Anaconda kind -

conda install -c conda-forge scipy
conda install -c conda-forge opencv
conda install -c conda-forge moviepy
conda install -c anaconda pyqt 
conda install -c conda-forge pillow
conda install -c simonflueckiger tesserocr 

Usage

Once that's done, run the main GUI by - python3 main.py

To find your FabBit of choice -

• Click MOVIES or SPORTS button for their respective use-cases
• Select the use-case from the sidebar
  • A pop-up dialog will ask for the actor if actor-specific scene was chosen
• Click on Choose File to select the input video
• Click on Find FabBits
• Move the slider in the blue areas, which are the extracted FabBits, and play the video
• Click on Save FabBits to save the extracted stuff into a video file

You can also run the respective files of use-cases to get their FabBit, like - python3 goal_detector.py soccer_match.mp4

Blog

I have mentioned almost everything about how I built FabBits on my blog. Give it a read if you want to build something involving the use-cases I have done!

Samples

This drive folder contains a list of videos and the video output FabBits produced.

Contributing

Pull requests are welcome! Although for major changes, please open an issue first to discuss what you would like to change.

References

Audio-Based Action Scene Classification Using HMM-SVM Algorithm by Khin Myo Chit, K Zin Lin

Action Scene Detection with Support Vector Machines; Liang-Hua Chen, Chih-Wen Su et a

A Scoreboard Based Method for Goal events Detecting in Football Videos; Song Yang, Wen Xiangming et al

Detecting Soccer Goal Scenes from Broadcast Video using Telop Region; Naoki Ueda, Masao Izumi

Anatomy of a Laugh Track

Primary Detection Methods for Laugh Tracks

Real-Time Event Detection in Field Sport Videos

Finding Celebrities in Billions of Webpages; Xiao Zhang, Lei Zhang, Xin-Jing Wang, Heung-Yeung Shum; IEEE Transaction on Multimedia, 2012

C.H. Demarty,C. Penet, M. Soleymani, G. Gravier. VSD, a public dataset for the detection of violent scenes in movies: design, annotation, analysis and evaluation. In Multimedia Tools and Applications, May 2014

C.H. Demarty, B. Ionescu, Y.G. Jiang, and C. Penet. Benchmarking Violent Scenes Detection in movies. In Proceedings of the 2014 12th International Workshop on Content-Based Multimedia Indexing (CBMI), 2014

M. Sjöberg, B. Ionescu, Y.G. Jiang, V.L. Quang, M. Schedl and C.H. Demarty. The MediaEval 2014 Affect Task: Violent Scenes Detection. In Working Notes Proceedings of the MediaEval 2014 Workshop, Barcelona, Spain (2014)

C.H. Demarty,C. Penet, G. Gravier and M. Soleymani. A benchmarking campaign for the multimodal detection of violent scenes in movies.In Proceedings of the 12thinternational conference on Computer Vision – Volume Part III (ECCV’12),Andrea Fusiello, Vittorio Murino, and Rita Cucchiara (Eds), Col. Part III. Springer Verlag, Berlin