We used Generative Adversarial Network (GAN) model proposed in Learning to Cartoonize Using White-box Cartoon Representations (CVPR 2020) by Xinrui Wang and Jinze Yu.
Our idea was to test if it is reasonably possible to perform model inferences in the browser clients with CPUs only. Without needing to send any of user's data (images) to servers.
App preview: Upload an image or try examples
Here's the application flow and architecture:
TensorFlow's Saved models are converted to TensorFlow.js models. Images are padded and scaled to 256px before they are fed to the model. This rescaling is done to speed up the processing and might reduce the quality too.
Model footprint is ~1.5MB. These models in the browsers without GPU acceleration could manage to cartoonize, but takes anywhere between 5-10 seconds for processing. This is much slower than tflite models performance in mobile devices. However, web browsers benefit from users not needing to install anything or transmitting data outside of their systems.
This work was possible due to
- Margaret Maynard-Reid and Sayak Paul's work on How to Create a Cartoonizer with TensorFlow Lite
- Xinrui Wang and Jinze Yu's original work on White-box CartoonGAN
Xinrui Wang and Jinze Yu are the original authors of White-box CartoonGAN.
@InProceedings{Wang_2020_CVPR,
author = {Wang, Xinrui and Yu, Jinze,
title = {Learning to Cartoonize Using White-Box Cartoon Representations,
booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
month = {June}, year = {2020} }
- Original White-box Cartoonization Repo
- Cartoonizer with TensorFlow Lite Repo
- Live application
- Repo, Notebook, Colab