AwesomeAnimeResearch

signal processing or machine learning related to anime or manga

Papers

Summary of published or preprint papers

Dataset

Paper	Conference	Links
The Amazing Mysteries of the Gutter: Drawing Inferences Between Panels in Comic Book Narratives	CVPR2017	Github
Building a Manga Dataset ”Manga109” with Annotations for Multimedia Applications	IEEE MultiMedia,2020	HP
DanbooRegion: An Illustration Region Dataset	ECCV2020	Github
Unconstrained Text Detection in Manga: a New Dataset and Baseline	ECCVW2020	Github
Cartoon Face Recognition: A Benchmark Dataset	ACM MM2020	Github
DAF:RE: A CHALLENGING, CROWD-SOURCED, LARGE-SCALE, LONG-TAILED DATASET FOR ANIME CHARACTER RECOGNITION		Github

Image Generation

Subcategory	Paper	Conference	Links
Generation	Towards the Automatic Anime Characters Creation with Generative Adversarial Networks	Comiket92	HP
	Full-body High-resolution Anime Generation with Progressive Structure-conditional Generative Adversarial Networks	ECCVW2018	HP
	Generate Novel Image Styles using Weighted Hybrid Generative Adversarial Nets	IJCNN2018
	Towards Diverse Anime Face Generation: Active Label Completion and Style Feature Network	EUROGRAPHICS2019
	An Adaptive Control Algorithm for Stable Training of Generative Adversarial Networks	IEEE Access2019
	Overcoming Long-term Catastrophic Forgetting through Adversarial Neural Pruning and Synaptic Consolidation
	Autoencoding Generative Adversarial Networks		Github
	Classification Representations Can be Reused for Downstream Generations
	GAN Memory with No Forgetting	NeurIPS2020	Github
	Generating Full-Body Standing Figures of Anime Characters and Its Style Transfer by GAN
	HistoGAN: Controlling Colors of GAN-Generated and Real Images via Color Histograms	CVPR2021	Github
	Efficient Continual Adaptation for Generative Adversarial Networks
	Generating "Ideal" Anime Opening Frames Using Neural Networks	ElConRus2021
	CoPE: Conditional image generation using Polynomial Expansions
	StyleGAN-NADA: CLIP-Guided Domain Adaptation of Image Generators		Github
	DisUnknown: Distilling Unknown Factors for Disentanglement Learning	ICCV2021	Github
Few-shot	Image Generation From Small Datasets via Batch Statistics Adaptation	ICCV2019	Github
	FEW-SHOT ADAPTATION OF GENERATIVE ADVERSARIAL NETWORKS		Github
	MineGAN: effective knowledge transfer from GANs to target domains with few images	CVPR2020	Github
	Freeze the Discriminator: a Simple Baseline for Fine-Tuning GANs	CVPRW2020	Github
	DATA INSTANCE PRIOR FOR TRANSFER LEARNING IN GANS
	MineGAN++: Mining Generative Models for Efficient Knowledge Transfer to Limited Data Domains
Interpretability	RPGAN: GANs Interpretability via Random Routing		Github
	Unsupervised Discovery of Interpretable Directions in the GAN Latent Space	ICML2020	Github
	Closed-Form Factorization of Latent Semantics in GANs	CVPR2021	Github
	Unsupervised Discovery of Disentangled Manifolds in GANs
	Do Generative Models Know Disentanglement? Contrastive Learning is All You Need		Github
	Surrogate Gradient Field for Latent Space Manipulation	CVPR2021
	EigenGAN: Layer-Wise Eigen-Learning for GANs		Github
	Discovering Interpretable Latent Space Directions of GANs Beyond Binary Attributes	CVPR2021

Image-to-image Translation

Subcategory	Paper	Conference	Link
Face2anime	Improving Shape Deformation in Unsupervised Image-to-Image Translation	ECCV2018	Github
	Twin-GAN – Unpaired Cross-Domain Image Translation with Weight-Sharing GANs		Github
	DA-GAN: Instance-level Image Translation by Deep Attention Generative Adversarial Networks	CVPR2018	Github
	Landmark Assisted CycleGAN for Cartoon Face Generation
	Generating Anime from Real Human Image with Adversarial Training	ICASERT2019
	Auto-Encoding for Shared Cross Domain Feature Representation and Image-to-Image Translation
	Few-shot Knowledge Transfer for Fine-grained Cartoon Face Generation
	Unsupervised Image-to-Image Translation via Pre-trained StyleGAN2 Network		Github
	A Note on Data Biases in Generative Models	NeurIPSW2020
	Turn Real People into Anime Cartoonization	ICCECE2021
	AniGAN: Style-Guided Generative Adversarial Networks for Unsupervised Anime Face Generation		Github
	Multi-CartoonGAN for Conditional Artistic Face Translation	JSAI2021
	GANs N’ Roses: Stable, Controllable, Diverse Image to Image Translation (works for videos too!)		Github
	FINE-TUNING STYLEGAN2 FOR CARTOON FACE GENERATION		Github
	AgileGAN: Stylizing Portraits by Inversion-Consistent Transfer Learning	ACM Trans. Graph. 2021	Github
Selfie2anime	U-GAT-IT: Unsupervised Generative Attentional Networks with Adaptive Layer-Instance Normalization for Image-to-Image Translation	ICLR2020	Github
	Breaking the cycle—Colleagues are all you need	CVPR2020	Github
	AttentionGAN: Unpaired Image-to-Image Translation using Attention-Guided Generative Adversarial Networks	IJCNN2019	Github
	Unpaired Image-to-Image Translation using Adversarial Consistency Loss	ECCV2020	Github
	Feature Quantization Improves GAN Training	ICML2020	Github
	Interpolation based Anime Face Style Transfer	MAPR2020
	SPatchGAN: A Statistical Feature Based Discriminator for Unsupervised Image-to-Image Translation
	Good Artists Copy, Great Artists Steal: Model Extraction Attacks Against Image Translation Generative Adversarial Networks
	Unaligned Image-to-Image Translation by Learning to Reweight	ICCV2021
Photo2anime	CartoonGAN: Generative Adversarial Networks for Photo Cartoonization	CVPR2018	Github
	Comixify: Transform video into a comics		Github
	GANILLA: Generative adversarial networks for image to illustration translation		Github
	CartoonRenderer: An Instance-based Multi-Style Cartoon Image Translator	MMM2020
	Learning to Cartoonize Using White-box Cartoon Representations	CVPR2020	Github
	Generative Adversarial Networks for photo to Hayao Miyazaki style cartoons		Github
	AnimeGAN: a novel lightweight GAN for photo animation		Github
	GAN-based Multi-Style Photo Cartoonization	IEEE TVCG2021
Sketch2anime	SmartPaint: a co-creative drawing system based on generative adversarial networks	FITEE2019
	PI-REC: Progressive Image Reconstruction Network With Edge and Color Domain		Github
	Modeling Artistic Workflows for Image Generation and Editing	ECCV2020	Github
	Deep Sketch-guided Cartoon Video Inbetweening	IEEE2021
	How to train your conditional GAN: An approach using geometrically structured latent manifolds
Photo2manga	MangaGAN: Unpaired Photo-to-Manga Translation Based on The Methodology of Manga Drawing	AAAI2021
Anime2costume	Anime-to-Real Clothing: Cosplay Costume Generation via Image-to-Image Translation
Style transfer	Anime Style Space Exploration Using Metric Learning and Generative Adversarial Networks
	Disentangling Style and Content in Anime Illustrations		Github
	RAG: Facial Attribute Editing by Learning Residual Attributes	IEEE Access2019
	StarGAN Based Facial Expression Transfer for Anime Characters	CSICC2020
	Few-shot Semantic Image Synthesis Using StyleGAN Prior		Github

Automatic Line Art Colorization

Subcategory	Paper	Conference	Links
NoHint	Automatic Sketch Colorization with Tandem Conditional Adversarial Networks	ISCID2018
	Do You Like Sclera? Sclera-region Detection and Colorization for Anime Character Line Drawings	IJNDC2019
	Colorization for Anime Sketches with Cycle-Consistent Adversarial Network	IJPE2019
	Seg2pix: Few Shot Training Line Art Colorization with Segmented Image Data	Appl. Sci. 2021
	Semi-automatic Manga Colorization Using Conditional Adversarial Networks		Github
	Stylized-Colorization for Line Arts	ICPR2021
	Generative Probabilistic Image Colorization
Atari	Manga Colorization	ACM Trans. Graph. 2006
	TexToons: Practical Texture Mapping for Hand-drawn Cartoon Animations	NPAR2011
	Outline Colorization through Tandem Adversarial Networks
	A Fast and Efficient Semi-guided Algorithm for Flat Coloring Line-arts	EG2018
	Auto-painter: Cartoon Image Generation from Sketch by Using Conditional Wasserstein Generative Adversarial Networks	Neurocomputing2018	Github
	User-Guided Deep Anime Line Art Colorization with Conditional Adversarial Networks	ACMMC2018	Github
	Two-stage Sketch Colorization	ACM Trans. Graph. 2018	Github
	Anime Sketch Coloring with Swish-gated Residual U-net and Spectrally Normalized GAN		Github
	PaintsTorch: a User-Guided Anime Line Art Colorization Tool with Double Generator Conditional Adversarial Network	CVMP2019
	Interactive Anime Sketch Colorization with Style Consistency via a Deep Residual Neural Network	TAAI2019
	Semi-Auto Sketch Colorization Based on Conditional Generative Adversarial Networks	CISP-BMEI2019
	MANGAN: ASSISTING COLORIZATION OF MANGA CHARACTERS CONCEPT ART USING CONDITIONAL GAN	ICIP2019	Github
	Two-Stage Sketch Colorization With Color Parsing	IEEE Access2019
	User Guided Digital Artwork Colorization	CSAI2019
	Cartoon image colorization based on emotion recognition and superpixel color resolution	ICCST2020
	Automatic Colorization of Anime Style Illustrations Using a Two-Stage Generator	Appl. Sci. 2020
	Line Art Colorization with Concatenated Spatial Attention	CVPRW2021
	User-Guided Line Art Flat Filling with Split Filling Mechanism	CVPR2021	HP
	Deep Edge-Aware Interactive Colorization against Color-Bleeding Effects
	Dual Color Space Guided Sketch Colorization	IEEE Trans. on Img. Proc. 2021
Reference	Unsupervised Colorization of Black-and-White Cartoons	NPAR2004
	Reference-based Manga Colorization by Graph Correspondence Using Quadratic Programming	SIGGRAPH2014
	Deep Manga Colorization with Color Style Extraction by Conditional Adversarially Learned Inference	IIAI2017
	Style Transfer for Anime Sketches with Enhanced Residual U-net and Auxiliary Classifier GAN	ACPR2017
	cGAN-based Manga Colorization Using a Single Training Image	ICDAR2017	Github
	Comicolorization: Semi-Automatic Manga Colorization	SIGGRAPH2017	Github
	Attentioned Deep Paint		Github
	Fully Automatic Colorization for Anime Character Considering Accurate Eye Colors	SIGGRAPH2019
	Semantic Example Guided Image-to-Image Translation	IEEE Transactions on Multimedia2020
	Graph Matching based Anime Colorization with Multiple References	SIGGRAPH2019
	Reference-Based Sketch Image Colorization using Augmented-Self Reference and Dense Semantic Correspondence	CVPR2020
	Multi-Density Sketch-to-Image Translation Network
	Deep-Eyes: Fully Automatic Anime Character Colorization with Painting of Details on Empty Pupils	EG2020
	Colorization of Line Drawings with Empty Pupils	PG2020
	Anime Sketch Colorization by Component-based Matching using Deep Appearance Features and Graph Representation	ICPR2021
	Anime Style Transfer With Spatially-Adaptive Normalization	ICME2021
	Exploring Sketch-based Character Design Guided by Automatic Colorization
	PAINTING STYLE-AWARE MANGA COLORIZATION BASED ON GENERATIVE ADVERSARIAL NETWORKS
Tag	Tag2Pix: Line Art Colorization Using Text Tag With SECat and Changing Loss	ICCV2019	Github
Video	Automatic Temporally Coherent Video Colorization		Github
	Artist-Guided Semiautomatic Animation Colorization	ICCVW2019
	Deep Line Art Video Colorization with a Few References
	Line Art Correlation Matching Feature Transfer Network for Automatic Animation Colorization	WACV2021

Automatic Character Lighting

Paper	Conference	Links
Ink-and-Ray: Bas-Relief Meshes for Adding Global Illumination Effects to Hand-Drawn Characters	ACM Trans. Graph. 2014
Deep Normal Estimation for Automatic Shading of Hand-Drawn Characters	ECCVW2018	Github
Automatic Illumination Effects for 2D Characters	NIPSW2018
2D shading for cel animation	NPAR2018	HP
Augmenting Hand-Drawn Art with Global Illumination Effects through Surface Inflation	CVMP 2019	HP
Generating Digital Painting Lighting Effects via RGB-space Geometry	ACM Trans. Graph. 2020	Github
Learning to Shadow Hand-drawn Sketches	CVPR2020	Github

Automatic Character Editing

Paper	Conference	Links
Spatially Controllable Image Synthesis with Internal Representation Collaging		Github
Erasing Appearance Preservation in Optimization-based Smoothing	ECCV2020	Github
Cross-Domain and Disentangled Face Manipulation with 3D Guidance		Github
L2M-GAN: Learning to Manipulate Latent Space Semantics for Facial Attribute Editing	CVPR2021
Deep Unfolding with Normalizing Flow Priors for Inverse Problems
Sketch-based Anime Hairstyle Editing with Generative Inpainting	NicoInt2021
DyStyle: Dynamic Neural Network for Multi-Attribute-Conditioned Style Editing
Improving The Quality Of Illustrations: Transforming Amateur Illustrations To A Professional Standard	ICIP2021

Automatic Sketch Editing

Paper	Conference	Links
Temporal Noise Control for Sketchy Animation	NPAR2011
Learning to Simplify: Fully Convolutional Networks for Rough Sketch Cleanup	ACM Trans. Graph. 2016	Github
Deep Extraction of Manga Structural Lines	ACM Trans. Graph. 2017	Github
Mastering Sketching: Adversarial Augmentation for Structured Prediction	ACM Trans. Graph. 2018	Github
Real-Time Data-Driven Interactive Rough Sketch Inking	ACM Trans. Graph. 2018	Github
Unpaired Sketch-to-Line Translation via Synthesis of Sketches	SIGGRAPH2019
Perceptual-aware Sketch Simplification Based on Integrated VGG Layers	TVCG2019
Progressive Full Data Convolutional Neural Networks for Line Extraction from Anime-Style Illustrations	Appl. Sci. 2020
SketchMan: Learning to Create Professional Sketches	MM2020	Github
One-shot Line Extraction from Color Illustrations	NicoInt2021

Automatic Animation Inbetweening

Paper	Conference	Links
A filter based approach for inbetweening (Japanese)		Movie
DiLight: Digital light table – Inbetweening for 2D animations using guidelines	CMV2017
Optical Flow Based Line Drawing Frame Interpolation Using Distance Transform to Support Inbetweenings	ICIP2019
Deep Animation Video Interpolation in the Wild	CVPR2021	Github

Talking Head Animation

Paper	Conference	Links
MakeItTalk: Speaker-Aware Talking-Head Animation	SIGGRAPH Asia2020
CPTNet: Cascade Pose Transform Network for Single Image Talking Head Animation	ACCV2020

Manga Application

Subcategory	Paper	Conference	Links
Generation	Synthesis of Screentone Patterns of Manga Characters	ISM2019
	Manga Filling Style Conversion with Screentone Variational Autoencoder	SIGGRAPH2020
	Generating Manga from Illustrations via Mimicking Manga Workflow	CVPR2021	HP
	Hair Shading Style Transfer for Manga with cGAN	ICAART2020
Restoration	Exploiting Aliasing for Manga Restoration	CVPR2021	Github
Inpainting	Seamless Manga Inpainting with Semantics Awareness
Text Detection	Method for Real Time Text Extraction of Digital Manga Comic	IJIP2011
	CNN based Extraction of Panels/Characters from Bengali Comic Book Page Images	ICDARW2019
	Deep Learning-Based Classification of the Polar Emotions of “Moe”-Style Cartoon Pictures	TST2020
	Learning from the Past: Meta-Continual Learning withKnowledge Embedding for Jointly Sketch, Cartoon, andCaricature Face Recognition	MM2020
Landmark Detection	Facial Landmark Detection for Manga Images

Representation Learning

Paper	Conference	Links
Illustration2Vec: A Semantic Vector Representation of Illustrations	SIGGRAPH2015	Github

Pose Estimation

Paper	Conference	Links
Pose Estimation of Anime/Manga Characters: A Case for Synthetic Data	MANPU2016
Transfer Learning for Pose Estimation of Illustrated Characters	WACV2022

Image Retrieval

Paper	Conference	Links
Sketch-based Manga Retrieval using Manga109 Dataset	MTA2017
AugNet: End-to-End Unsupervised Visual Representation Learning with Image Augmentation		Github

Character Recognition

Paper	Conference	Links
Progressive Deep Feature Learning for Manga Character Recognition via Unlabeled Training Data	ACM TURC2019
CNN based Extraction of Panels/Characters from Bengali Comic Book Page Images	ICDARW2019
Deep Learning-Based Classification of the Polar Emotions of “Moe”-Style Cartoon Pictures	TST2020
Learning from the Past: Meta-Continual Learning withKnowledge Embedding for Jointly Sketch, Cartoon, andCaricature Face Recognition	MM2020
Graph Jigsaw Learning for Cartoon Face Recognition
ACFD: Asymmetric Cartoon Face Detector	IJCAI2020

3D Character Creation

Paper	Conference	Links
Automatic Generation of 3D Natural Anime-like Non-Player Characters with Machine Learning	ICCW2020

Robotics

Paper	Conference	Links
Making Robots Draw A Vivid Portrait In Two Minutes

Speech Synthesis

Paper	Conference	Links
Comic-Guided Speech Synthesis	ACM Trans. Graph. 2019	HP

Survey

Paper	Conference
A Survey of Comics Research in Computer Science
Computational Approaches to Comics Analysis
Image Colorization: A Survey and Dataset

Projects

Summary of github or other types of projects that are related to anime or manga except above.

Dataset

• Layered Temporal Dataset for Anime Drawings

Image Generation

• makegirlsmoe
• ANIME305/Anime-GAN-tensorflow
• jayleicn/AnimeGAN
• FangYang970206/Anime_GAN
• pavitrakumar78/Anime-Face-GAN-Keras
• forcecore/Keras-GAN-Animeface-Character
• tdrussell/IllustrationGAN
• m516825/Conditional-GAN
• bchao1/Anime-Generation

Image-to-Image Tranlation

• Aixile/chainer-cyclegan
• SystemErrorWang/FacialCartoonization
• experience-ml/cartoonize
• racinmat/anime-style-transfer
• TachibanaYoshino/AnimeGANv2
• XiaoSanGit/Real2Animation-video-generation
• rinongal/StyleGAN-nada
• Avatar Artist Using GAN
• Generating Cartoon Style Facial Expressions with StackGAN

Automatic Line Art Colorization

• style2paints
• PaintsChainer
• Ugness/Line-Art-Colorization-SPADE
• sanjay235/Sketch2Color-anime-translation
• Pengxiao-Wang/Style2Paints_V3
• GANime: Generating Anime and Manga Character Drawings from Sketches
• Line Drawing Colorization

Character Animating

• Talking Head Anime from a Single Image
• Talking Head Anime from a Single Image 2: More Expressive

Super Resolution

• waifu2x
• Anime4K
• goldhuang/SRGAN-PyTorch

Segmentation

• jerryli27/AniSeg
• zymk9/Yet-Another-Anime-Segmenter

Landmark Detection

• Anime face landmark detection by deep cascaded regression