Project Request

Automatic captioning of videos based on cooking by understanding the action behind the scene and providing Nutrtional information based on the ingredients used.

https://github.com/nehavish006

Define You

GSSOC Participant
Contributor - Snekha C | Contributor

Project Name

Description

Video Captioning is a task of automatic captioning a video by understanding the action and event in the video which can help in the retrieval of the video efficiently through text. Video Captioning is an encoder decoder mode based on sequence to sequence learning. Automated video caption generator helps searching of videos in websites better and make content easier. The video information is considered as a sequence of images with 10 to 12 seconds short video clips. In the proposed system, Convolutional Neural Networks (VGG 16) is used for feature extraction and LSTM is used for encoding and decoding the features. Greedy search algorithm is used for predicting the efficient caption and gives speedy word extraction. Finally, the language converter is used for specific people language to understand the captions.

Scope

Objectives:

• To help the people with Deaf and hard of hearing individuals to watch videos helps people to focus on and remember the information more easily.
• To build the feature extraction using the convolutional neural network (VGG 16).
• To examine how LSTM is used for encoding and decoding the features.
• To explore how the greedy search algorithm predict the efficient caption.

Deliverables:

• A trained CNN RNN model capable of analyzing the video and generating the captions.
• Well-documented guidelines, including dataset preparation, training, inference, and any specific requirements.

Timeline

Start date : The date of assignment
End date: June 15 2023

Video Links or Support Links

Sandeep Samleti , Ashish Mishra , Alok Jhajhria , Shivam Kumar Rai, Gaurav Malik, 2021, "Real Time Video Captioning Using Deep Learning", International Journal Of Engineering Research & Technology (IJERT) Volume 10, Issue 12 (December 2021).