See the full website + gallery: http://opendot.github.io/ml4a-invisible-cities/
“With cities, it is as with dreams: everything imaginable can be dreamed, but even the most unexpected dream is a rebus that conceals a desire or, its reverse, a fear. Cities, like dreams, are made of desires and fears, even if the thread of their discourse is secret, their rules are absurd, their perspectives deceitful, and everything conceals something else.”
― Italo Calvino, Invisible Cities
The idea is to create an imaginary city from a hand-drawn sketch. Trained with aerial images of real cities, a neural network can transform this into a realistic bird-eye-view city. Then, switching between different cities references it would be possible to generate different views of the same imaginary city.
How it works
We were fascinated by the possibility of generating new and non-existent but realistic images using conditional adversarial neural networks that remembers a certain set of features from the things it has seen in the past: the same process that we humans undergo when we dream.
Taking inspiration from the given examples, we applied a pre-defined color scheme to geographic data (OpenStreetMap) using Mapbox Studio: roads, green spaces, buildings, water were styled with different colours (black, green, red, blue), so that the neural network (NN) could compare these to aerial images and learn the different features.
Training, evaluating, running
We then used vvvv as a tool to collect both satellite imagery and associated labeled map tiles. We trained a conditional generative adversarial network to recontruct the satellite imagery from its map tiles.
It then produces a set of images according to the unique characteristics of each city: the same blue shade will translate to a venetian canal or a simple river, red will became a 17th century villa or a 50s modernist house in the hills of L.A.
To encompass the variability of all geographic features, we left the background as plain white. This translated to unexpected results as the NN could interpret the same white patch of land as an airport, a maize field or a dumpster.
City style transfer
With this technique, we fed map tiles of one city to the generative model of another city, producing sattelite imagery of the former in the style of the latter.
Here we feed completely handdrawn tiles to the models, producing hallucinations of cities.
- Gene Kogan
- Gabriele Gambotto
- Ambhika Samsen
- Andrej Boleslavsky
- Michele Ferretti
- Damiano Gui
- Fabian Frei