Skip to content
master
Go to file
Code

Latest commit

 

Git stats

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

README.md

COMICS

code to download comics data and train models described in The Amazing Mysteries of the Gutter: Drawing Inferences Between Panels in Comic Book Narratives.

email miyyer@umd.edu and varunm@cs.umd.edu with any comments/problems/questions/suggestions.

dependencies:

  • requires python 2.7, lasagne, theano, h5py, cv2, glob2

to download / unzip / preprocess COMICS data:

  • bash setup.sh (downloads raw panel images, OCR transcriptions, etc., and preprocesses them into an hdf5 file)
  • if you don't want to download everything at once, you can download individual files at https://obj.umiacs.umd.edu/comics/index.html.

to train models after preprocessing (example for text cloze):

  • python models/text_cloze.py (make sure to run on GPU; see run.sh for our theano flags)
  • see description of hyperparameters by running python models/text_cloze.py --help
  • note that low-quality data is only filtered out in dev/test data (by throwing out examples with too many UNK tokens). during training, all data is used.

results:

method text cloze easy text cloze hard visual cloze easy visual cloze hard character coherence
text only 63.4 52.9 55.9 48.4 68.2
image only 51.7 49.4 85.7 63.2 70.9
image text 68.6 61.0 81.3 59.1 69.3

if you use the COMICS data and/or code, please cite:

@InProceedings{Iyyer:Manjunatha-Comics2017,
    Title = {The Amazing Mysteries of the Gutter: Drawing Inferences Between Panels in Comic Book Narratives},
    Booktitle = {IEEE Conference on Computer Vision and Pattern Recognition},
    Author = {Mohit Iyyer and Varun Manjunatha and Anupam Guha and Yogarshi Vyas and Jordan Boyd-Graber and Hal {Daum\'{e} III} and Larry Davis},
    Year = {2017},
}

About

COMICS data / code / annotations

Resources

License

Releases

No releases published

Packages

No packages published
You can’t perform that action at this time.