This repository contains a SIFT-based algorithm for whale photo identification written in MATLAB.
ATTENZIONE! This code is not cleaned and not optimized.
HOW TO RUN
Main script for running:
Input: directory 'DIRNAME' containing
- whale images in .jpg
- subdirectory 'masks' with inside-whale masks (if there is an image /DIRNAME/ABC.jpg, then there should be a mask /DIRNAME/masks/ABSMask.jpg). The image within the inside-whale mask has white (255,255,255) pixels in the region inside the whale, and black pixels (0,0,0) everywhere else.
- optional - .xls file containing two colums of the form: "whale_image_file_name whale_id" (gold standard)
Output: Matches between each two images stored in 'DIRNAME':
- dump of Matlab results in .mat
- image names and number of matches in .xls
- visualization of matches in .png
For each image find SIFT features (
- Calculate SIFT features in the dilated inside-whale region, i.e. mask dilated by mask_height * mask_dilation_rad_percent
- Find features with centers inside the eroded inside-whale region, i.e. mask eroded by mask_height * mask_erosion_rad_percent
Find matches of SIFT features for each two images (
Reduce number of matches based on kNN_k (parameter) nearest neighbors of each match (
kNN_matches_reductionwith parameters kNN_k, KNN_RED_PARAM)
For each two images i1 and i2 Iterate For each match (m1,m2), where m1 is a feature in i1 and m2 is the matched feature in i2 find kNN_k nearest neighbors of m1 in i1 (NN1) and m2 in i2 (NN2) compute how many NN1 are not matched to some NN2 (diff(m1,m2)) If there is a match (mi,mj) with diff(mi,mj) > KNN_RED_PARAM (parameter) Then remove match (mk,mt) with max diff(mk,mt) Else stop iteration
- Reduce number of matches based on pairwise angles between kNNangl_k (parameter) nearest neighbors of each match (
For each two images i1 and i2 Iterate For each match (m1,m2), where m1 is a feature in i1 and m2 is the matched feature in i2 find kNNangl_k nearest neighbors of m1 in i1 (NN1) take matches of the features NN1 in i2 (NM2) compute differences between all angles (m11,m1,m12) and (m21,m2,m22), where a) m11, m12 are in NN1 and m21,m22 are in NM2, b) m11 is matched to m21, m12 is matched to m11 compute the average difference over kNNangl_k maximal differences (average_diff(m1,m2)) If there is a match (mi,mj) with average_diff(mi,mj) > KNNANGL_RED_THRESH (parameter) Then remove match (mk,mt) with max average_diff(mk,mt) reduce number of matches based on kNNangl_k (parameter) nearest neighbors of each match (`kNN_matches_reduction` with parameters kNNangl_k, KNN_RED_PARAM) Else stop iteration
- Remove matched feature configurations that are horizontally flipped (
For each two images i1 and i2 find the most left and right features (l and r) in i1 that are mapped to some features in i2 find mappings (l,lm) and (r,rm), where lm and rm are features in i2 If lm > rm on the horizontal scale Then remove all matches between i1 and i2
- For each two images, output matches
In the real use case, algorithm step 1 is done after an image is uploaded and the whale region is selected. For each image, SIFT features (double matrix) are stored in the DB. Other steps are done when matching a new image with images stored in the DB.