Awesome-Image-Colorization

A collection of Deep Learning based Image Colorization papers and corresponding source code/demo program, including Automatic and User Guided (i.e. with User Interaction) colorization, as well as video colorization.

Feel free to create a PR or an issue. (Pull Request is preferred)

Outline

• Automatic Image Colorization
• User Guided Image Colorization
  • Based on color strokes
  • Based on reference color image
  • Based on color palette
  • Based on language(text)
  • Multi-modal (Joint inputs)
  • Interactive Colorization
• Techniques of Improving Image Colorization
• Video Colorization
  • Automatically
  • Based on reference

---

1. Automatic Image Colorization

• Software / Demo

Name	Author/Owner	Code/Project Link
DeOldify	Jason Antic	[link]
Palette.fm	Emil Wallner	[link]

• Papers

Paper	Source	Code/Project Link
Learning Large-Scale Automatic Image Colorization	ICCV 2015	[project] [code]
Deep Colorization	ICCV 2015
Learning Representations for Automatic Colorization	ECCV 2016	[project] [code]
Colorful Image Colorization	ECCV 2016	[project] [code]
Let there be Color!: Joint End-to-end Learning of Global and Local Image Priors for Automatic Image Colorization with Simultaneous Classification	SIGGRAPH 2016	[project] [code]
Learning Diverse Image Colorization	CVPR 2017	[code]
Structural Consistency and Controllability for Diverse Colorization	ECCV 2018
Coloring With Limited Data: Few-Shot Colorization via Memory Augmented Networks	CVPR 2019	[project] Unofficial: [code (PyTorch)]
ChromaGAN: Adversarial Picture Colorization with Semantic Class Distribution	WACV 2020	[code]
Instance-aware Image Colorization	CVPR 2020	[code] [project]
Pixelated Semantic Colorization	IJCV 2020
Colorization Transformer	ICLR 2021	[code]
Focusing on Persons: Colorizing Old Images Learning from Modern Historical Movies	ACM MM 2021	[code]
Towards Vivid and Diverse Image Colorization with Generative Color Prior	ICCV 2021	[code]
SCSNet: An Efficient Paradigm for Learning Simultaneously Image Colorization and Super-Resolution (also reference based)	AAAI 2022
Bridging the Domain Gap towards Generalization in Automatic Colorization	ECCV 2022	[code]
ColorFormer: Image Colorization via Color Memory assisted Hybrid-attention Transformer	ECCV 2022
BigColor: Colorization using a Generative Color Prior for Natural Images	ECCV 2022	[project] [code]
CT2: Colorization Transformer via Color Tokens	ECCV 2022	[code]
Disentangled Image Colorization via Global Anchors	SIGGRAPH Asia 2022	[code] [project]
UniColor: A Unified Framework for Multi-Modal Colorization with Transformer (also multi-modal)	SIGGRAPH Asia 2022	[project]

2. User Guided Image Colorization

2.1 Based on color strokes

• Software / Demo

Image Type	Name	Author/Owner	Code/Project Link
Line art	PaintsChainer (Petalica Paint)	Preferred Networks, Inc.	[V3 Demo] [V1 code]
Line art	Style2Paints (SEPA)	Style2Paints Research	[link]

• Papers

Image Type	Paper	Source	Code/Project Link
Manga	Manga colorization	SIGGRAPH 2006	[Project Home] Unofficial: [code]
Line art / Sketch / Manga	LazyBrush: Flexible Painting Tool for Hand-drawn Cartoons	Eurographics 2009	[Project Home & Software] Unofficial: [code1 (Matlab)] [code2 (C#)] [code3]
Line art / Sketch	Outline Colorization through Tandem Adversarial Networks	1704.08834	[code]
Natural Gray-Scale	Real-Time User-Guided Image Colorization with Learned Deep Priors	SIGGRAPH 2017	[project] [code1] [code2]
Sketch	Scribbler: Controlling Deep Image Synthesis with Sketch and Color	CVPR 2017
Line art / Sketch	Auto-painter: Cartoon Image Generation from Sketch by Using Conditional Generative Adversarial Networks	Neurocomputing 2018	[code]
Line art	User-Guided Deep Anime Line Art Colorization with Conditional Adversarial Networks	ACM MM 2018	[code]
Line art	Style2paints V3 : Two-stage Sketch Colorization	SIGGRAPH Asia 2018	[Project] [Paper's code(V3)] Unofficial: [code1] [code2] [PaintsTensorFlow]
Line art	User-Guided Line Art Flat Filling with Split Filling Mechanism	CVPR 2021	[Project] [code]
Line art	Dual Color Space Guided Sketch Colorization	TIP 2021
Natural Gray-Scale	iColoriT: Towards Propagating Local Hint to the Right Region in Interactive Colorization by Leveraging Vision Transformer	WACV 2023	[project] [code]

2.2 Based on reference color image

Image Type	Paper	Source	Code/Project Link
Line art	Style2paints V1 : Style Transfer for Anime Sketches with Enhanced Residual U-net and Auxiliary Classifier GAN	ACPR 2017	[Code] Unofficial: [code]
Manga	Comicolorization: Semi-Automatic Manga Colorization	SIGGRAPH Asia 2017	[code]
Sketch	TextureGAN: Controlling Deep Image Synthesis with Texture Patches	CVPR 2018	[code]
Natural Gray-Scale	Deep Exemplar-based Colorization	SIGGRAPH 2018	[code]
Natural Gray-Scale	Example-Based Colourization Via Dense Encoding Pyramids (also palette based)	Pacific Graphics 2018	[code]
Natural Gray-Scale	A Superpixel-based Variational Model for Image Colorization	TVCG 2019
Natural Gray-Scale	Automatic Example-based Image Colourisation using Location-Aware Cross-Scale Matching	TIP 2019
Line art / Sketch	Adversarial Colorization Of Icons Based On Structure And Color Conditions	ACM MM 2019	[Code]
Line art / Sketch	Reference-Based Sketch Image Colorization using Augmented-Self Reference and Dense Semantic Correspondence	CVPR 2020	[project] Unofficial: [code]
Natural Gray-Scale	Stylization-Based Architecture for Fast Deep Exemplar Colorization	CVPR 2020	[code]
Manga	Manga Filling Style Conversion with Screentone Variational Autoencoder	SIGGRAPH Asia 2020	[project]
Line art / Sketch	Colorization of Line Drawings with Empty Pupils	PG 2020
Line art / Sketch	Active Colorization for Cartoon Line Drawings	TVCG 2020
Natural Gray-Scale	Gray2ColorNet: Transfer More Colors from Reference Image	ACM MM 2020
Line art / Sketch	Line Art Correlation Matching Feature Transfer Network for Automatic Animation Colorization	WACV 2021
Natural Gray-Scale	Globally and Locally Semantic Colorization via Exemplar-Based Broad-GAN	TIP 2021
Natural Gray-Scale	Yes, "Attention Is All You Need", for Exemplar based Colorization	ACM MM 2021
Natural Gray-Scale	SCSNet: An Efficient Paradigm for Learning Simultaneously Image Colorization and Super-Resolution (also automatic)	AAAI 2022
Line art / Sketch	Style-Structure Disentangled Features and Normalizing Flows for Diverse Icon Colorization	CVPR 2022
Line art / Sketch	Eliminating Gradient Conflict in Reference-based Line-Art Colorization	ECCV 2022	[code]
Natural Gray-Scale	Semantic-Sparse Colorization Network for Deep Exemplar-based Colorization	ECCV 2022

2.3 Based on color palette

Image Type	Paper	Source	Code/Project Link
Natural Image	Palette-based Photo Recoloring	SIGGRAPH 2015	[project]
Natural Gray-Scale	Coloring with Words: Guiding Image Colorization Through Text-based Palette Generation (also text based)	ECCV 2018	[code]
Natural Gray-Scale	Example-Based Colourization Via Dense Encoding Pyramids (also reference based)	Pacific Graphics 2018	[code]
Natural Gray-Scale	PalGAN: Image Colorization with Palette Generative Adversarial Networks	ECCV 2022	[code]

2.4 Based on language or text

Image Type	Paper	Source	Code/Project Link
Natural Gray-Scale / Sketch	Language-Based Image Editing with Recurrent Attentive Models	CVPR 2018	[code]
Natural Gray-Scale	Coloring with Words: Guiding Image Colorization Through Text-based Palette Generation (also palette based)	ECCV 2018	[code]
Natural Gray-Scale	Learning to Color from Language	NAACL 2018	[code]
Line art	Tag2Pix: Line Art Colorization Using Text Tag With SECat and Changing Loss	ICCV 2019	[code] [code2 (GUI)]
Scene Sketch	Language-based Colorization of Scene Sketches	SIGGRAPH Asia 2019	[code] [project]
Natural Gray-Scale	L-CoDe: Language-based Colorization using Color-object Decoupled Conditions	AAAI 2022	[code]
Natural Gray-Scale	L-CoDer: Language-based Colorization with Color-object Decoupling Transformer	ECCV 2022	[code]

2.5 Multi-modal (Joint inputs)

Image Type	Paper	Input	Source	Code/Project Link
Natural Gray-Scale	Interactive Deep Colorization Using Simultaneous Global and Local Inputs	stroke + palette	ICASSP 2019
Natural Gray-Scale	UniColor: A Unified Framework for Multi-Modal Colorization with Transformer	stroke + exemplar + text	SIGGRAPH Asia 2022	[project]

2.6 Interactive Colorization

Image Type	Paper	Source	Code/Project Link
Line art	Guiding Users to Where to Give Color Hints for Efficient Interactive Sketch Colorization via Unsupervised Region Prioritization	WACV 2023

3. Techniques of Improving Image Colorization

Paper	Source	Code/Project Link
Deep Edge-Aware Interactive Colorization against Color-Bleeding Effects	ICCV 2021	[project] [code(metric)]
Line Art Colorization Based on Explicit Region Segmentation	Pacific Graphics 2021	[code]

4. Video Colorization

4.1 Automatically

Image Type	Paper	Source	Code/Project Link
Natural Gray-Scale	Fully Automatic Video Colorization with Self-Regularization and Diversity	CVPR 2019	[code]

4.2 Based on reference

Image Type	Paper	Source	Code/Project Link
Natural Gray-Scale	Switchable Temporal Propagation Network	ECCV 2018
Natural Gray-Scale	Tracking Emerges by Colorizing Videos	ECCV 2018	[code]
Natural Gray-Scale	Deep Exemplar-based Video Colorization	CVPR 2019	[code]
Natural Gray-Scale	DeepRemaster: Temporal Source-Reference Attention Networks for Comprehensive Video Enhancement	SIGGRAPH Asia 2019	[code] [project]
Natural Gray-Scale	Reference-Based Video Colorization with Spatiotemporal Correspondence	2011.12528
Line art	The Animation Transformer: Visual Correspondence via Segment Matching	ICCV 2021	[App]
Line art	Line Art Correlation Matching Feature Transfer Network for Automatic Animation Colorization	WACV 2021
Line art	Reference-Based Deep Line Art Video Colorization	TVCG 2022
Natural Gray-Scale	BiSTNet: Semantic Image Prior Guided Bidirectional Temporal Feature Fusion for Deep Exemplar-based Video Colorization	arxiv 22.12	[code] [project]