Cluster similar images

[JoshVarty]

For image related tasks it might be useful to cluster similar images together. We may want to count these images across the train and test set and see whether or not the distributions are equal.

[JoshVarty]

Thoughts:

• Train a network
• Remove the head
• Take the output for each image
• Run PCA on each vector
• Run K-Means clustering on all of our vectors

[JoshVarty]

It isn't working. Possibly because the features our network learns focus on "Does this image have cancer or not"

Possible workarounds

• Same approach but cluster 1 and 0 separately. (Might not work for test set though)
• Try either earlier layers of the network or simply use the raw input

[JoshVarty]

Tried with more success on the input image. Things to try:

1. Try without PCA
2. Try with different number of PCA components
3. Can we seed the K-means clusters?
   • I think we can use init. See here