Captionate - Image Captioning Toolkit

Task is to design a Deep Learning+NLP based toolkit (python-library) which can input images and generate textual description of the Image. Further addons to the project can be a web application where the user can upload an image and get the caption for input image or an Android/iOS application where the user can capture/upload an image and get the caption generated using this toolkit.

Skills Required: Python, Object-Oriented Programming, NLP, Famous DL Libraries like Tensorflow or Keras. Optional Skills include Full stack web app, Android/iOS App Development.

Resources to get started

1. COCO Dataset
2. Flicker 8k Dataset
3. Flicker 30k Dataset
4. Implementation in PyTorch
5. Keras Implementation
6. Another Keras Implementation
7. Another Implementation using PyTorch
8. Understanding Project at Root Level
9. Research Paper on Convolutional Image Captioning
10. Intro to RNN
11. Understanding RNN & LSTM
12. Understanding CNN - Part 2 link in article.

Senior Mentors for this Project

• Nikhil Pandey
• Ronak Sakhuja
• Rajat