This project examines how image classification models can learn from soft similarity information. Using transfer learning, we train classifier heads on top of ResNet50 to learn to classify the CIFAR10 test images based on a custom soft contrastive objective function, which dually minimizes the cross entropy loss and the contrastive loss, or distance between embeddings of similar images. The similarity scores are given as soft values, generated by the human-annotated labels in the CIFAR10H dataset, or hard binary flags as in SimCLR or SupCon methods to serve as the two baselines.
Cifar10H dataset: https://github.com/jcpeterson/cifar-10h
SimCLR: https://github.com/google-research/simclr
SupCon: https://github.com/google-research/google-research/tree/master/supcon