Standing on the shoulders of giants:
- Based on github.com/hamidkazemi22/vit-visualization
- This repo is a lightweight, CLIP ViT feature visualization only, implementation thereof.
- Contains ALL (!) CLIP models + some speed optimization. Requires CUDA.
- Uses OpenAI / CLIP
-
Check / install
requirements.txt -
Check the comments in "run_visualization.py" -> Easy lightweight configuration
-
Configure: model, optimizer, single-feature vs. multi-feature/layer, intermediate steps, ...
-
From the console, use "python run_visualization.py"
-
Saves output in neatly organized and named way (subfolders)
-
Work in progress
-
Currently non-deterministic. A "rose neuron" gets you a rose; but a slightly different one each time.
- CLIP has been trained on "pretty much the entire internet".
- One neuron gets you a cat, a rose; another may encode explicit / sensitive / offensive / violent content.
- Use responsibly / at your own discretion.
- For more information, refer to the CLIP Model Card.