Skip to content
A list of synthetic dataset and tools for computer vision
Python Shell
Branch: master
Clone or download
qiuwch Merge pull request #20 from breezykermo/topic/fix-link
provide alternative link for SunCG
Latest commit a296778 May 8, 2019
Type Name Latest commit message Commit time
Failed to load latest commit information.
src Remove scholarpy which is not very reliable. Nov 26, 2016
.gitignore Update gitignore. Sep 13, 2016
LICENSE Create LICENSE Nov 30, 2016 provide alternative link for SunCG May 8, 2019 Update Jun 5, 2017

Synthetic for Computer Vision

This is a repo for tracking the progress of using synthetic images for computer vision research. If you found any important work is missing or information is not up-to-date, please edit this file directly and make a pull request. Each publication is tagged with a keyword to make it easier to search.

If you find anything missing from this page, please edit this file to add it. When adding a new item, you can simply follow the format of existing items. How this document is structured is documented in

How to use: Click publication to jump to the paper title, detailed information such as code and project page will be provided together with pdf file.**

Synthetic image dataset

3D Model Repository

Realistic 3D models are critical for creating realistic and diverse virtual worlds. Here are research efforts for creating 3D model repositories.



ECCV 2016 Workshop Virtual/Augmented Reality for Visual Artificial Intelligence (VARVAI) workshop

ICCV 2017 Workshop Role of Simulation in Computer Vision

Virtual Reality Meets Physical Reality: Modelling and Simulating Virtual Humans and Environments Siggraph Asia 2016 workshop

CVPR 2017 Workshop THOR Challenge

See also:





  • Adversarially Tuned Scene Generation (pdf)

  • UE4Sim: A Photo-Realistic Simulator for Computer Vision Applications (pdf) (project)

  • Playing for Benchmarks (pdf)
  • A Self-supervised Learning System for Object Detection using Physics Simulation and Multi-view Pose Estimation (:octocat:code) (pdf) (project)
  • Procedural Generation of Videos to Train Deep Action Recognition Networks (pdf) (project) (citation:8)
  • Learning from Synthetic Humans (:octocat:code) (pdf) (project) tag: synthetic human

  • Nvidia Issac

  • Configurable, Photorealistic Image Rendering and Ground Truth Synthesis by Sampling Stochastic Grammars Representing Indoor Scenes

  • Aerial Informatics and Robotics Platform (:octocat:code) (pdf) (project) tag: tool
  • Tobin, Josh, et al. "Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World." arXiv preprint arXiv:1703.06907 (2017). tag: domain (pdf)
  • M. Johnson-Roberson, C. Barto, R. Mehta, S. N. Sridhar, Karl Rosaen,and R. Vasudevan, “Driving in the matrix: Can virtual worlds replace human-generated annotations for real world tasks?,” in IEEE International Conference on Robotics and Automation, pp. 1–8, 2017. (:octocat:code) (pdf) (project) (citation:3)



  • Sadeghi, Fereshteh, and Sergey Levine. "rl: Real single-image flight without a single real image. arXiv preprint." arXiv preprint arXiv:1611.04201 12 (2016). tag: rl

  • Johnson, Justin, et al. "CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoning." arXiv preprint arXiv:1612.06890 (2016). (pdf)

  • McCormac, John, et al. "SceneNet RGB-D: 5M Photorealistic Images of Synthetic Indoor Trajectories with Ground Truth." arXiv preprint arXiv:1612.05079 (2016).

  • de Souza, César Roberto, et al. "Procedural Generation of Videos to Train Deep Action Recognition Networks." arXiv preprint arXiv:1612.00881 (2016). (pdf) (project) tag: synthetic human

  • Synnaeve, Gabriel, et al. "TorchCraft: a Library for Machine Learning Research on Real-Time Strategy Games." arXiv preprint arXiv:1611.00625 (2016). (pdf) (code)

  • Lin, Jenny, et al. "A virtual reality platform for dynamic human-scene interaction." SIGGRAPH ASIA 2016 Virtual Reality meets Physical Reality: Modelling and Simulating Virtual Humans and Environments. ACM, 2016. (pdf) (project)

  • Mahendran, A., et al. "ResearchDoom and CocoDoom: Learning Computer Vision with Games." arXiv preprint arXiv:1610.02431 (2016). (pdf) (project)

  • The SYNTHIA dataset: A large collection of synthetic images for semantic segmentation of urban scenes. 2016 (pdf) (project) (citation:4)
  • Virtual Worlds as Proxy for Multi-Object Tracking Analysis. 2016
    (pdf) (project) (citation:5)

  • Playing for data: Ground truth from computer games. 2016
    (pdf) (citation:1)

  • Play and Learn: Using Video Games to Train Computer Vision Models. 2016
    (pdf) (citation:1)

  • ViZDoom: A Doom-based AI Research Platform for Visual Reinforcement Learning. 2016
    (:octocat:code) (pdf) (project) (citation:4)

  • UnrealCV: Connecting Computer Vision to Unreal Engine 2016
    (:octocat:code) (project) (pdf)
  • Learning Physical Intuition of Block Towers by Example 2016
    (:octocat:code) (pdf) (citation:12)

  • Target-driven Visual Navigation in Indoor Scenes using Deep Reinforcement Learning 2016

  • A Dataset and Evaluation Methodology for Depth Estimation on 4D Light Fields. ACCV 2016
    (:octocat:code) (pdf) (project) (citation)



  • A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimation. 2015
    (pdf) (citation:9)
  • Render for cnn: Viewpoint estimation in images using cnns trained with rendered 3d model views. 2015
    (:octocat:code) (pdf) (citation:33)



  • Virtual and real world adaptation for pedestrian detection. 2014
    (pdf) (citation:46)
  • Seeing 3d chairs: exemplar part-based 2d-3d alignment using a large dataset of cad models. 2014
    (:octocat:code) (pdf) (project) (citation:110)
  • Handa, Ankur, Thomas Whelan, John McDonald, and Andrew J. Davison. "A benchmark for RGB-D visual odometry, 3D reconstruction and SLAM." In Robotics and automation (ICRA), 2014 IEEE international conference on, pp. 1524-1531. IEEE, 2014. (project)



  • Detailed 3d representations for object recognition and modeling. 2013
    (pdf) (citation:67)





  • Learning appearance in virtual scenarios for pedestrian detection. 2010
    (pdf) (citation:79)



  • Ovvv: Using virtual worlds to design and evaluate surveillance systems. 2007
    (pdf) (citation:58)
You can’t perform that action at this time.