I built and trained a custom deep learning model to recognise paintings by artist — then wrapped it in an AI-powered web app that explains its own decisions in natural language.
This is not a pre-built API or a wrapper around someone else's classifier. The model at the core of ARTisan is a ResNet18 I fine-tuned myself on a curated dataset of paintings, trained to distinguish between 19 artists based purely on visual style. Everything from the training pipeline to the Grad-CAM explainability layer to the Groq AI integration was built from scratch.
The classifier is a ResNet18 convolutional neural network, originally pretrained on ImageNet, that I fine-tuned on a painting dataset across 19 artists. Fine-tuning means I replaced the final classification layer, froze the early feature extraction layers, and retrained the network on my own data — teaching it to distinguish Rembrandt's chiaroscuro from Van Gogh's impasto, or Dali's surrealist compositions from Magritte's.
| Training Detail | Value |
|---|---|
| Base Model | ResNet18 (ImageNet pretrained) |
| Approach | Transfer learning + fine-tuning |
| Classes | 19 artists |
| Input Size | 224 × 224 px |
| Output | Softmax probability distribution |
| Explainability | Grad-CAM heatmaps via pytorch-grad-cam |
After training, I added a Grad-CAM layer — this generates a heatmap over the original painting showing exactly which brushstrokes, regions, and compositional elements the model weighted most heavily in its prediction. Then I connected Groq AI to turn those raw model outputs into natural language analysis.
Identify — Runs the painting through the classifier, returns confidence score, Grad-CAM heatmap, and an AI-written explanation of why the model made that call
Compare — Extracts deep feature embeddings from two paintings and computes cosine similarity — measuring stylistic closeness at the feature level, not the pixel level
Style — Aggregates top-3 prediction probabilities to detect the dominant art movement, with full AI historical context
Explain — Open-ended AI interpretation of any painting including personal artwork, with no classifier involved — just Groq reading the canvas
Botticelli, Chagall, Dali, Degas, Gauguin, Goya, Klee, Leonardo, Magritte, Matisse, Modigliani, Picasso, Rembrandt, Renoir, Rubens, Sisley, Titian, Van Gogh, Warhol
| Technology | Purpose |
|---|---|
| Python 3.12 | Core application logic |
| PyTorch + ResNet18 | Custom trained artist classifier |
| pytorch-grad-cam | Visual explainability heatmaps |
| Gradio | Web application interface |
| Groq API (Llama 3.3 70B) | Natural language analysis |
| python-dotenv | API key management |
Prerequisites
- Python 3.12
- A free Groq API key from console.groq.com
- The trained model file
artist_classifier_v2.pth
git clone https://github.com/Megatron148/artisan.git
cd artisan
pip3.12 install torch torchvision gradio pytorch-grad-cam groq python-dotenv Pillow
echo "GROQ_API_KEY=your_api_key_here" > .env
Update MODEL_PATH in app.py to point to your artist_classifier_v2.pth, then:
python3.12 app.py
Open http://127.0.0.1:7860 in your browser.
artisan/
│
├── app.py # App logic — inference, Grad-CAM, AI integration, UI
├── artist_classifier_v2.pth # Trained model weights (not committed — too large for GitHub)
├── .env # Groq API key (not committed)
└── README.md # This file
- Training and fine-tuning convolutional neural networks on custom datasets
- Transfer learning — adapting pretrained models to new domains
- Grad-CAM for visual explainability in deep learning models
- Building cosine similarity search from deep feature embeddings
- Integrating production AI APIs into a Python application
- Designing multi-tab interfaces with Gradio
- Expand to 50+ artists
- Upload your own images to add new artists to the classifier
- Side-by-side Grad-CAM comparison between two paintings
- Mobile-optimized interface
- Art history timeline view by movement
Keegan Doyle — @Megatron148
Interdisciplinary Studies — Exploring the intersection of AI, computer vision, and art history
