Link to Google Colab notebook for image captioning and story generation: https://colab.research.google.com/drive/1pvdBQilg-jG8V-qvB3k4QuFyQL7E-9ao
Link to Google Drive folder containing other necessary code and files: https://drive.google.com/open?id=1j19xzpgDrP8IBJlMXMgAFQeBOal2Hd-3
Image captioning is based off of: https://github.com/yunjey/pytorch-tutorial/tree/master/tutorials/03-advanced/image_captioning
Story generation is based off of OpenAI's GPT-2; specifically, based our code on this: https://colab.research.google.com/github/ilopezfr/gpt-2/blob/master/gpt-2-playground_.ipynb
The image captioning uses the resnet-152 model pretrained on the ILSVRC-2012-CLS image classification dataset. The encoder is a convolutional neural network (CNN) and decoder a long short-term memory (LSTM) network. You can download a pretrained model here: https://www.dropbox.com/s/ne0ixz5d58ccbbz/pretrained_model.zip?dl=0. Extract pretrained_model.zip to image_captioning/models/.
- Please use Google Chrome and get the following extension.
- You must have the Firebase CLI installed. If you don't have it install it with
npm install -g firebase-tools
and then configure it withfirebase login
. - On the command line run
firebase use --add storygen-849ff
. - On the command line run
firebase serve
using the Firebase CLI tool to launch a local server.
© Google, 2016. Licensed under an Apache-2 license.
Base Source Code here.