This repository contains code for training neural networks over Google Street View images with the task of country classification. It has been done as Project Work of the course Machine Learning for Computer Vision at the University of Bologna (report can be read at report.pdf).
Previous approaches only use north, east, west and down frames for country classification.
A lot of countries, anyway, contain meta-data in their coverage, meaning that they use different cars or different cameras. This knowledge is often useful to narrow down the possible countries for a specific guess.
The following images shows the Google car with metal bars on top, typical of specific countries like Guatemala or Ghana.
Our hypothesis is that meta-data could improve the metrics, so we perform tests by providing the down frame as well.
The dataset is obtained by using create_data.py. For each sampled location, we get north, east, south, west, down frames.
The neural networks have been tested on 4 countries (Andorra, Canada, Denmark, Ghana), with 1000 location sampled per country (5000 images each).
To obtain imagery for different countries, change the country_names list inside create_data.py with the desired countries (you can get a list of the countries covered by Streetview here).
The code may sometimes fail to obtain a valid Streetview location since it randomly samples a position in a given country and searches over a certain radius.
The general approach is shown in the following figure
The classifications and softmax scores for each country are fed into a LambdaMart model, which is also provided with the score of a binary classificator Neural Network that detects Google Car meta data (using down frames).
The trained neural networks are
- A simple convolutional network
- ResNet50
- ResNet50 with pre-trained weights (ImageNet)
The results are provided in the following table. The models with the prefix m are the ones that use meta-data.
| Model | Precision | Recall | F1 |
|---|---|---|---|
| CNN | 0.50 | 0.52 | 0.50 |
| mCNN | 0.51 | 0.53 | 0.51 |
| ResNet50 | 0.50 | 0.50 | 0.50 |
| mResNet50 | 0.53 | 0.53 | 0.53 |
| ResNet50 IN | 0.71 | 0.71 | 0.71 |
| mResNet50 IN | 0.73 | 0.73 | 0.73 |