Awesome-Sketch-Based-Applications

A collection of sketch based applications.

Feel free to create a PR or an issue.

Outline

• 1. Sketch Based Image Synthesis
  • 1.1 Automatic Synthesis
  • 1.2 Style-based Synthesis
• 2. Sketch Based Image Editing
• 3. Sketch Based Image Retrieval (SBIR)
• 4. Sketch Based 3D Shape Retrieval
• 5. Sketch Based 3D Shape Modeling
• 6. Sketch Completion
• 7. Sketch Segmentation and Perceptual Grouping
• 8. Sketch Based Transfer Learning
• 9. Sketch and Shading

---

1. Sketch Based Image Synthesis

1.1 Automatic Synthesis

Paper	Source	Code/Project Link
SketchyGAN: Towards Diverse and Realistic Sketch to Image Synthesis	CVPR 2018	[code]
Image Generation from Sketch Constraint Using Contextual GAN	ECCV 2018	[code] [project]
Multi-Instance Sketch to Image Synthesis With Progressive Generative Adversarial Networks	IEEE Access 2019
Interactive Sketch & Fill: Multiclass Sketch-to-Image Translation	ICCV 2019	[code] [project]
LinesToFacePhoto: Face Photo Generation from Lines with Conditional Self-Attention Generative Adversarial Network	ACM MM 2019
SketchyCOCO: Image Generation from Freehand Scene Sketches	CVPR 2020	[code]
DeepFaceDrawing: Deep Generation of Face Images from Sketches	SIGGRAPH 2020	[project] [code]
Unsupervised Sketch-to-Photo Synthesis	ECCV 2020	[code]

1.2 Style-based Synthesis

Paper	Source	Code/Project Link
TextureGAN: Controlling Deep Image Synthesis with Texture Patches	CVPR 2018	[code]
Multimodal Unsupervised Image-to-Image Translation	ECCV 2018	[code]

2. Sketch Based Image Editing

Paper	Source	Code/Project Link
FaceShop: Deep Sketch-based Face Image Editing	SIGGRAPH 2018	[project]
Sparse, Smart Contours to Represent and Edit Images	CVPR 2018	[project]
Example-Guided Style-Consistent Image Synthesis from Semantic Labeling	CVPR 2019	[code]
Free-Form Image Inpainting with Gated Convolution (DeepFill v2)	ICCV 2019	[project] [code]
SC-FEGAN: Face Editing Generative Adversarial Network With User's Sketch and Color	ICCV 2019	[code]
Fashion Editing with Multi-scale Attention Normalization	1906.00884
Intuitive, Interactive Beard and Hair Synthesis with Generative Models	CVPR 2020
Deep Plastic Surgery: Robust and Controllable Image Editing with Human-Drawn Sketches	ECCV 2020

3. Sketch Based Image Retrieval (SBIR)

• Object-level

Paper	Source	Code/Project Link
Sketch Me That Shoe	CVPR 2016	[code-caffe] [code-tf] [project]
Deep Sketch Hashing: Fast Free-hand Sketch-Based Image Retrieval	CVPR 2017	[code]
Deep Spatial-Semantic Attention for Fine-Grained Sketch-Based Image Retrieval	ICCV 2017	[project]
Zero-Shot Sketch-Image Hashing	CVPR 2018
SketchMate: Deep Hashing for Million-Scale Human Sketch Retrieval	CVPR 2018
Generative Domain-Migration Hashing for Sketch-to-Image Retrieval	ECCV 2018	[code]
Generalising Fine-Grained Sketch-Based Image Retrieval	CVPR 2019
Doodle to Search: Practical Zero-Shot Sketch-Based Image Retrieval	CVPR 2019	[project] [code]
LiveSketch: Query Perturbations for Guided Sketch-Based Visual Search	CVPR 2019
Semantically Tied Paired Cycle Consistency for Zero-Shot Sketch-Based Image Retrieval	CVPR 2019	[code]
Generative Model for Zero-Shot Sketch-Based Image Retrieval	CVPRW 2019
Learning Structural Representations via Dynamic Object Landmarks Discovery for Sketch Recognition and Retrieval	TIP 2019
Stacked Semantic-Guided Network for Zero-Shot Sketch-Based Image Retrieval	1904.01971
Semantic-Aware Knowledge Preservation for Zero-Shot Sketch-Based Image Retrieval	ICCV 2019
Semantic Adversarial Network for Zero-Shot Sketch-Based Image Retrieval	1905.02327
TC-Net for iSBIR: Triplet Classification Network for Instance-level Sketch Based Image Retrieval	ACM MM 2019
Sketch Less for More: On-the-Fly Fine-Grained Sketch Based Image Retrieval	CVPR 2020	[code]
Solving Mixed-modal Jigsaw Puzzle for Fine-Grained Sketch-Based Image Retrieval	CVPR 2020

• Scene-level

Paper	Source	Code/Project Link
SceneSketcher: Fine-Grained Image Retrieval with Scene Sketches	ECCV 2020

4. Sketch Based 3D Shape Retrieval

Paper	Source	Code/Project Link
Sketch-based 3D Shape Retrieval using Convolutional Neural Networks	CVPR 2015
Learning Cross-Domain Neural Networks for Sketch-Based 3D Shape Retrieval	AAAI 2016
Deep Correlated Metric Learning for Sketch-based 3D Shape Retrieval	AAAI 2017
Learning Barycentric Representations of 3D Shapes for Sketch-based 3D Shape Retrieval	CVPR 2017
Deep Cross-modality Adaptation via Semantics Preserving Adversarial Learning for Sketch-based 3D Shape Retrieval	ECCV 2018
Unsupervised Learning of 3D Model Reconstruction from Hand-Drawn Sketches	ACMMM 2018

5. Sketch Based 3D Shape Modeling

Paper	Source	Code/Project Link
3D Shape Reconstruction from Sketches via Multi-view Convolutional Networks	3DV 2017	[code] [project]
Shape Synthesis from Sketches via Procedural Models and Convolutional Networks	TVCG 2017
DeepSketch2Face: A Deep Learning Based Sketching System for 3D Face and Caricature Modeling	SIGGRAPH 2017	[project]
BendSketch: Modeling Freeform Surfaces Through 2D Sketching	SIGGRAPH 2017	[project]
Robust Flow-Guided Neural Prediction for Sketch-Based Freeform Surface Modeling	SIGGRAPH Asia 2018	[code] [project]
Learning a Shared Shape Space for Multimodal Garment Design	SIGGRAPH Asia 2018	[project]
DeepSketchHair: Deep Sketch-based 3D Hair Modeling	TVCG 2019

6. Sketch Completion

Paper	Source	Code/Project Link
Joint Gap Detection and Inpainting of Line Drawings	CVPR 2017	[project] [code]
SketchGAN: Joint Sketch Completion and Recognition with Generative Adversarial Network	CVPR 2019
Sketch-BERT: Learning Sketch Bidirectional Encoder Representation from Transformers by Self-supervised Learning of Sketch Gestalt	CVPR 2020

7. Sketch Segmentation and Perceptual Grouping

• Sketch Segmentation

Type	Paper	Source	Code/Project Link
Pixelwise	Fast Sketch Segmentation and Labeling With Deep Learning	CGA 2019
SPFusionNet: Sketch Segmentation Using Multi-modal Data Fusion	ICME 2019
SketchyScene: Richly-Annotated Scene Sketches (scene-level)	ECCV 2018	[code]
Stroke-level	Free Hand-Drawn Sketch Segmentation	ECCV 2012
	Data-driven Segmentation and Labeling of Freehand Sketches	SIGGRAPH Asia 2014	[code] [project] [dataset]
	Example-Based Sketch Segmentation and Labeling Using CRFs	TOG 2016
	SketchSegNet+: An End-to-End Learning of RNN for Multi-Class Sketch Semantic Segmentation	IEEE Access 2019
	Stroke-based sketched symbol reconstruction and segmentation	1901.03427

• Sketch Perceptual Grouping

Type	Paper	Source	Code/Project Link
Stroke-level	Sketching by perceptual grouping	ICIP 2013
	Making Better Use of Edges via Perceptual Grouping	CVPR 2015
	Universal Sketch Perceptual Grouping / Toward Deep Universal Sketch Perceptual Grouper	ECCV 2018 / TIP 2019	[code]

8. Sketch Based Transfer Learning

Paper	Source	Code/Project Link
SketchTransfer: A Challenging New Task for Exploring Detail-Invariance and the Abstractions Learned by Deep Networks	WACV 2020

9. Sketch and Shading

Paper	Source	Code/Project Link
Learning to Shadow Hand-drawn Sketches	CVPR 2020	[project] [code]