Using PyTorch, I took a pretrained ResNet152 model and further trained it on a collection of 40,000 images of works of art from the openly licensed collections at Rijksmuseum and The Metropolitan Museum of Art. On average, it guesses the age of a work within 66 years.
This project is referenced in my article, AI Literacy: How artificial intelligence classifies and predicts our world.
Please see the Jupyter Notebook in this repository for a complete walkthrough of the performance of this model.
I've never worked with convolutional neural networks before, so take this as an example of a beginner's exploration rather than any sort of best practice. PyTorch is relatively new software and I've found it useful to read other people's code, as there isn't much of it out in the world yet.
- ResNet 152 (avg error +/- 66 years, 222MB)
You can either use one of the pre-trained models above, or train your own convnet on a corpus of artwork where you have metadata as well as imagery (and appropriate licensing!).
The metadata
directory contains two CSV files mapping year-of-work to IDs from Rijksmuseum and the Met. You can use those
IDs to acquire the full images from the providers. If you want to train on another task besides year, get the
full metadata sets from the providers and use those.
If you use one of the pre-trained models, guess.py
can accept either an image by URL or a CSV in the
form of true year, path-to-image-file, like so:
year,path
1803,/home/liza/data/rijksmuseum-images/214/RP-T-FM-241.jpeg
1886,/home/liza/data/met-images/435/435649.jpg
1480,/home/liza/data/met-images/337/337494.jpg
(The train.py
script will automatically generate a test-after-split.csv
file in this format, by pulling out 10% of the training data as a validation set.)
If a URL is passed, the script will return the prediction and a PIL object. The CSV form will output its predictions to another CSV, as:
year,diff,pred,path
1803,6,1797,/home/liza/data/rijksmuseum-images/214/RP-T-FM-241.jpeg
1886,304,1582,/home/liza/data/met-images/435/435649.jpg
1480,158,1638,/home/liza/data/met-images/337/337494.jpg
1700,159,1859,/home/liza/data/met-images/365/365519.jpg
suitable for analysis, as in evaluate.ipynb
Happy guessing!
Liza Daly https://twitter.com/liza