! This paper is under review on the experimental track of the Journal of Visualization and Interaction.While convolutional neural networks have been widely studied, their inner workings remain hard to interpret for humans. Tailored explainability methods for these models often focus on saliency. For language models and generative models in general, on the other hand, analysis approaches frequently aim at exploring the latent spaces of the model, which is less commonly done for image classification models. At the same time, inspecting neighboring samples within intermediate representations generated by the layers of CNNs has already proven useful to enhance model robustness.
In this work, we derive tailored visualizations for a convolutional neural network classifier where we treat the multidimensional output range of each layer as an embedding space. Guided by the previous results on the significance of locality, we visualize effects on the local surrounding of samples in Euclidean space, focusing on the layer-wise transformation the network applies to the data. By comparing distances between samples across layers, we contribute a global and two local views showing the neighborhood development of samples throughout the network. Our visualizations help to gain insights into what the model has learned and the contribution of each layer to the classification. Our global visualization captures aspects of the interplay between data and model architecture, where future work might result in useful tools for analyzing, comparing or improving neural network architectures.
We used the QuickDraw dataset for our experiments: https://github.com/googlecreativelab/quickdraw-dataset
The code for re-running our experiments can be found here: https://github.com/moritzdueck/collaiborate
Moritz Dück: Conceptualization, Methodology, Investigation, Visualization, Software, Writing - Original Draft Johannes Knittel: Conceptualization, Supervision, Writing - Review & Editing Hendrik Strobelt: Conceptualization, Supervision Mennatallah El-Assady: Conceptualization, Supervision
The authors declare that there are no competing interests.