🖼 Create artwork from your casual sketch with GAN and style transfer
Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
build
config
server
src
static
.babelrc
.editorconfig
.gitignore
.postcssrc.js
LICENSE
README.md
index.html
package.json
yarn.lock

README.md

Sketch to Art 🎨

You could be an artist with AI

[Live Demo]

Introduction

This project can transform your casual sketch to beautiful painting/artwork using modern AI technology.

Screenshots

Theories

To achieve the goal, there are mainly two steps in the pipeline:

  • Reconstruct and generate real image from the sketch
  • Arbitary style transfer to beautify the result with given result

Sketch Reconstruction

The principle behind this is called Conditional Adversarial Networks, known as pix2pix, which is able to generate image based on the given image.

Style Transfer

It became known to us with the appearance of Prisma app. Typically, we generate an individual model for each pre-defined style. Here, we want to go further by using any new picture as the style. So, we adopted the method, Universal Style Transfer via Feature Transforms proposed in NIPS2017, which enables us to perform arbitary style transfer.

Installation

Backend

The server side is powered by Python and Flask. You can see this simpler example. Navigate to the server directory and all the files concerning the service and neural networks are there. The two main files:

  • app_pix.py for pix2pix translation
  • app_stylize.py for arbitrary style transfer

Prerequisites

Make sure you have Python installed. And the following key packages are needed:

tensorflow
keras
pillow
flask
gevent

Run

Simply run with python:

$ python app_xxx.py

And you could see the output indicating the port it's listening (5001 and 5002). Go to http://localhost:5001 and you should see the returned information.

Frontend

You should installed:

# Clone the repo
$ git clone git@github.com:mtobeiyf/sketch-to-art.git
$ cd sketch-to-art

# Install dependencies
$ yarn  # or npm install

# Run
$ yarn dev  # or npm run dev

Open your favorite browser at http://localhost:8080, the site is there.

Acknowledgments

This is a project of Digital Image Processing taught by Dr. Jia Yan.

WCT-TF

pix2pix-tensorflow

pytorch-CycleGAN-and-pix2pix

Neural-Style-Transfer-Papers

Vue.js

Authors

Shuaibin Zhang, Haoran Su, Tangbo Liu, Xin Fu

License

Copyright © 2018, Fing

Released under the MIT License.