Skip to content
Permalink
Browse files

Added some things lol

  • Loading branch information...
Aryan Mann
Aryan Mann committed Apr 26, 2019
1 parent 84837e4 commit 6053f453cc1eee4365919de85f7737c08d847262
Showing with 149 additions and 0 deletions.
  1. +58 −0 README.md
  2. +91 −0 app.py
  3. BIN images/chicago.jpg
  4. BIN images/style01.jpg
@@ -3,6 +3,64 @@

Any visit to an art museum leaves the observer with a sense of the style of each individual work. This project aims to create a process by which the style of one painting is transferred to another algorithmically. The result is a beautiful fusion between the content and style between two art pieces.

## VGG-19 Architecture
# Notes

#### Layers of the VGG-19 Image Classification Network
```
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
input_1 (InputLayer) (None, None, None, 3) 0
_________________________________________________________________
block1_conv1 (Conv2D) (None, None, None, 64) 1792
_________________________________________________________________
block1_conv2 (Conv2D) (None, None, None, 64) 36928
_________________________________________________________________
block1_pool (MaxPooling2D) (None, None, None, 64) 0
_________________________________________________________________
block2_conv1 (Conv2D) (None, None, None, 128) 73856
_________________________________________________________________
block2_conv2 (Conv2D) (None, None, None, 128) 147584
_________________________________________________________________
block2_pool (MaxPooling2D) (None, None, None, 128) 0
_________________________________________________________________
block3_conv1 (Conv2D) (None, None, None, 256) 295168
_________________________________________________________________
block3_conv2 (Conv2D) (None, None, None, 256) 590080
_________________________________________________________________
block3_conv3 (Conv2D) (None, None, None, 256) 590080
_________________________________________________________________
block3_conv4 (Conv2D) (None, None, None, 256) 590080
_________________________________________________________________
block3_pool (MaxPooling2D) (None, None, None, 256) 0
_________________________________________________________________
block4_conv1 (Conv2D) (None, None, None, 512) 1180160
_________________________________________________________________
block4_conv2 (Conv2D) (None, None, None, 512) 2359808
_________________________________________________________________
block4_conv3 (Conv2D) (None, None, None, 512) 2359808
_________________________________________________________________
block4_conv4 (Conv2D) (None, None, None, 512) 2359808
_________________________________________________________________
block4_pool (MaxPooling2D) (None, None, None, 512) 0
_________________________________________________________________
block5_conv1 (Conv2D) (None, None, None, 512) 2359808
_________________________________________________________________
block5_conv2 (Conv2D) (None, None, None, 512) 2359808
_________________________________________________________________
block5_conv3 (Conv2D) (None, None, None, 512) 2359808
_________________________________________________________________
block5_conv4 (Conv2D) (None, None, None, 512) 2359808
_________________________________________________________________
block5_pool (MaxPooling2D) (None, None, None, 512) 0
=================================================================
Total params: 20,024,384
Trainable params: 20,024,384
Non-trainable params: 0
_________________________________________________________________
```

## References
- ***"A Neural Algorithm of Artistic Style"*** by L. Gatys, A. Ecker, and M. Bethge: http://arxiv.org/abs/1508.06576.
- ***"Perceptual Losses for Real-Time Style Transfer and Super-Resolution"*** by J. Johnson, A. Alahi, Li Fei-Fei: https://arxiv.org/abs/1603.08155
91 app.py
@@ -0,0 +1,91 @@
# Artistic Style Transfer by Aryan Mann and Stefan Ilic

import keras.backend as k
from keras.applications import vgg19 as icn
from keras.applications.imagenet_utils import preprocess_input
from keras.preprocessing import image
from keras import layers as kl
from keras.utils import plot_model
from keras.models import Model, Sequential
from matplotlib import pyplot as pp
import numpy as np

## Properties
image_size = (512, 512)

## Define the Loss functions
def prepare_vgg_input(img):
"""
Given an image, process it for input into the VGG network
:param img: An image
:return: An image fit for use as input into the VGG network
"""
img = image.img_to_array(img)
img = np.expand_dims(img, axis=0)
return preprocess_input(img)


def vgg_input_to_image(img):
img = img[0, :, :, :]
return img


def content_loss(features_from_content, features_from_output):
"""
The Content Loss described in Gatys paper
:param features_from_content: VGG-layer features from the content image
:param features_from_output: VGG-layer features from the output image
:return: The content loss i.e. sum of squared differences
"""
return k.sum(k.square(features_from_content - features_from_output))


def style_loss(style_of_content, style_of_output):
"""
The Style Loss described in Gatys paper
:param style_of_content:
:param style_of_output:
:return: The style loss
"""

def gram_matrix(matrek):
return k.dot(matrek, k.transpose(matrek))

# RGB <--> BRG change. Thanks VGG-19!
content_style = k.batch_flatten(k.permute_dimensions(style_of_content, (2, 0, 1)))
output_style = k.batch_flatten(k.permute_dimensions(style_of_output, (2, 0, 1)))

gram_content = gram_matrix(content_style)
gram_output = gram_matrix(output_style)

# What are those N,M terms? Something about L2 Regularization
return k.sum(k.square(gram_content - gram_output))


## See the layers!



### Get output tensor for a specific layer
def model_from_layer(layer_name):
return Model(inputs=model.input, outputs=model.get_layer(layer_name).output)


#### TIME TO DO THINGS ####



### Load the VGG Model
model = icn.VGG19(weights='imagenet', include_top=False)

### Load the Image
content_image = image.load_img('images/chicago.jpg', target_size=(224, 224, 3))
content_image = prepare_vgg_input(content_image)

## Get output from a certain layer of the VGG

block1_conv1_model = model_from_layer('block3_conv2')
resultant_img = block1_conv1_model.predict(content_image)

pp.imshow(resultant_img)
pp.show()
BIN +11.6 KB images/chicago.jpg
Binary file not shown.
BIN +226 KB images/style01.jpg
Binary file not shown.

0 comments on commit 6053f45

Please sign in to comment.
You can’t perform that action at this time.