Purpose

Facilitate image caption generation of unseen visual object categories (MSCOCO datasets) with knowledge guided attention mechanism.

Dependencies

Keras 2.0.8 Theano 0.9.0.dev

Installation Instructions

1. git clone https://github.com/adityamogadala/KGA.git
2. Caption Model Training - kga_caption_model.py
3. Language Model Training - language_model.py
4. Generation of Image Captions (Greedy) - kga_greedy_inference.py
5. Generation of Image Captions (Beam) - kga_beam_inference.py

Citation

Aditya Mogadala, Umanga Bista, Lexing Xie, Achim Rettinger. In 15th Extended Semantic Web Conference (ESWC). Springer International Publishing. (2018)