In this project, we tried to solve the Landmark Recognition Challenge by Google hosted on Kaggle. The idea is to build a model that recognizes the correct landmark in a dataset of challenging test images. A major challenge for this application is to obtain a large annotated dataset. It is extremely difficult to add annotations manually for all images. So, to build the model, we utilize the annotated data provided by them. This data has been constructed by clustering visually similar photos and matching them based on local features.
For image recognition and classification, deep CNN is the state-of-the-art approach for training the model. The reason for high popularity of CNN is because it takes advantage of local spatial coherence in the input images. Moreover, they get trained using fewer weights compared to other regular neural nets. However, the issue with normal deep CNNs is that they do not scale well with the increase in number of classes. This indicates that they directly cannot work for our dataset which consists of more than 15k classes. In our huge dataset, some categories are easier to classify than others. If we take an example from CIFAR dataset, it is easy to tell the difference between an apple and a car. But it is hard to distinguish a car from a bus. Here, we can say that car and bus belongs to the same higher level category of vehicles while apple belongs to the category of fruits. If we leverage this categorization, then we will be able to scale our model well with the increase in number of classes.
We implement hierarchical deep CNN which consists of a two level hierarchy. These hierarchy can either be learnt from the data or it can be predefined. Once the hierarchies have been established, we train our model to first classify the incoming images into a coarse category and then direct the flow to the corresponding fine category classifier. The HD-CNN model is modular and it can built on top of any existing building block CNN. Thus, the model consists of a coarse to fine classification followed by probabilistic integration of predictions from both classifiers.