Selective Style Transfer for Text

Accepted to ICDAR 2019 PDF

Authors: Raul Gomez, Ali Furkan Biten, Lluis Gomez, Jaume Gibert, Marçal Rusiñol, Dimosthenis Karatzas

End-To-End Model

To be released soon.

Two Stage Model

Requirements:

tensorflow
caffe
magenta (only to train the style transfer model)

Models

Download the models and put them in data/models/.

Magenta Scene Text Style Transfer Model

TextFCN Model

Stylizing images

Images are assumed to be in data/img/.

Style Transfer

Stylize the entire images using magenta scene text model. To stylize images you don't need a complete magenta installation, it's enough with the magenta code included in magenta/. (Notice we have modified some code in image_stylization_transform.py, so a raw magenta won't work). Results are saved in data/styleTransfer/.

python style_images.py

Text Segmentation

Get text segmentation heatmaps using TextFCN. Results are saved in data/heatmaps/.

python get_TextFCN_heatmaps.py

Selective Text Style Transfer

Do weighted blending to get the final results of selective style transfer two stage model. Results are saved in data/SelectivestyleTransfer/.

python weighted_blending.py

Training

To train the magenta style transfer model follow the original instructions using the source style images found in src_styles/.