A data-driven line extractor for 2D anime, manga and illustration using Keras.
- Online demo ver. alpha
- Looking for nico-opendata to make public dataset, if you have nico-opendata please open issue.
- New LineDistiller is on the schedule, it should work for illustration without line-drawings.
Extraction of high quality lines from 2D illustration is very crucial for further deep learning work and its application such as colorization, sketch simplication and animation. This work was started for solving this basic problem and we received inspiration from Deep Extraction of Manga Structural Lines  and sketchKeras . Our approach build upon network in . The residual block in  was changed to bottleneck residual block for better performance and less model size. We also tried original network in  and U-net like encoder-decoder network with highpass input like . Among these models, the first model are better in both model size and result.
- Keras2 (Tensorflow backend)
- Set up directories.
The data should be organized into directories. The directory tree is:
data/raw, contains the images for train
data/contour_s, contains the line images for train
data/v_raw, contains the images for validation
data/v_contour_s, contains the line images for validation
data/predict, contains the images for prediction
data/result, contains the output results from prediction
train.py uses keras
ImageDataGenerator, please organize sub folders as it needs
Download the model from release and put it in the same folder with code, then rename to
train.pyfor train and
model_180102.h5: Model trained using MSE loss with private dataset for 25 epochs, loss ~0.001. model_180121.h5: Model trained using MSE loss with private dataset for 25 epochs, loss ~0.001.
- The model only works with illustrations which have lines. It will not perform edge detection, it just extracts lines from the original image. So it can ignore edges between different color fills and shadings (often cause gradient changes).
- Due to small dataset, the performance for general illustration could be worse. The model should be trained with argumented data contains different brightness and line width.
- The model can also extract text in dark color from image with complex background, see example in zhihu link.
- Parameter EPOCHS and BATCH_SIZE in
train.pyhave been optimized.
- Using tanh as activation of last layer results better performance but may cause some artifacts.
- Prefered sizes of image are HD (1920x1080) and SD (1280x720). Memory cost for HD image may be heavy.
- The input is preprocessed and the output is postprocessed for better result.
- CLAHE may cause artifacts around high contrast area.
- To avoid artifacts in white background, input should be preprocessed with contract stretch operation(input 0-255 to output 0-230). This works in model_180102.h5 and model_180121.h5. And This issue will be fix in next release.
- Train on large and various dataset.
- Classify lines and output them with classes such as boundray line, shading line.
From Project HAT by Hepesu With