Deep Learning for Recognizing Cities from their Google Street View Images
Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
data
feature map
matconvnet
reference
scripts
svm
urban_viz
.gitignore
1.301430818,103.864462_0.png
README.md
teaser.png

README.md

cv_streetview

Code repository for deep learning urban visual characteristics analysis

Overview

The goal of this research project is to evaluate how well cities/districts can be characterized by their visual properties using the readily available city imaginary data set provided by Google street view.

In order to find out the image features that would affect a deep learning algorithm to distinguish between different cities, we trained several image categorization neural networks using AlexNet and VGG-S CNN structures, and analyzed the visual clues that helps the algorithm make categorization decisions. We found out that the algorithm precisely grasps the differences between different city types, and has almost 92% accuracy* on test set for the best model.

As an exploration of latent urban characteristics that can be reflected in the visual aspect, the research proposes an alternative paradigm of computational urban morphological research where environmental affordances revealed in streetlevel urban imagery are taken into consideration.

CNN recognizing cities from Google streetview.

Dataset

About 160,000 images were requested using Googole Streetview API (data requesting code here). Samples of the dataset can be found in the "data" folder. 2/3 of the images were used for training, 1/6 for validating, and 1/6 for testing.

Demo

The project is built within MatConvNet framework. To run the demo, please

  1. Clone the repository.

  2. Follow this guide to install MatConvNet.

  3. Download the trained model from here

  4. Run matconvnet/examples/run_city_demo.m to predict the the city of a given image (make sure you change the paths of the trained model as well as the image to point to the model you just downloaded and the image you want to test).

Work in progress

This project is a work in progress and you can find the latest updates in visual feature extraction and analysis mainly in the scripts/ folder.

(* Please refer to the paper for details and caveats, etc.)