Image Captioning with External Knowledge

Description

This repository contains the code of the models developed for my PhD dissertation, "Image Captioning with External Knowledge".

• geo-aware: contains the code of the geo-aware captioning model, trained on the GeoRic dataset
• knowledge-aware: contains the code of the encyclopedic knowledge-aware captioning model, trained on the K-GeoRic dataset
• news-knowledge-aware: contains the code of the knowledge-aware captioning model applied to the domain of news images, trained on the NYTimes800k dataset

Contacts

For questions contact Sofia Nikiforova: s.nikiforova@uu.nl.

Acknowledgments

This PhD project was carried out at Utrecht University, under the supervision of Prof. Dr. Yoad Winter, Dr. Tejaswini Deoskar and Dr. Denis Paperno. The work was funded by the European Research Council (ERC) under the European Union's Horizon 2020 research and innovation programme (grant agreement No 742204).