PUG: Photorealistic and Semantically Controllable Synthetic Data for Representation Learning

PUG: Photorealistic Unreal Graphics

Website | Research Paper | Datasheet

PUG_Video.mp4

This codebase contains:

• download links for the PUG-datasets
• dataloaders
• scripts that are needed to samples images from a running interactive environment made with the Unreal Engine.
• script to evaluate VLMs models with PUG: SPAR
• list of the assets used to create the PUG datasets (which are listed in each PUG folders)

Downloading the PUG datasets

Here are the links to download the PUG datasets:

• PUG: Animals (78GB)
• PUG: ImageNet (27GB)
• PUG: SPAR (16GB)
• PUG: AR4T (97GB)

Dataset loaders

Please look at each PUG subfolder to get information on how to load the datasets.

PUG Animals

PUG ImageNet

PUG SPAR

PUG AR4T

How to create a PUG environment ?

The instruction are availables in the torchmultiverse folder.

LICENSE

The datasets are distributed under the CC-BY-NC, with the addenda that they should not be used to train Generative AI models, as found in the LICENSE file.

Citing PUG

If you use the PUG datasets, please cite:

@misc{bordes2023pug,
      title={PUG: Photorealistic and Semantically Controllable Synthetic Data for Representation Learning}, 
      author={Florian Bordes and Shashank Shekhar and Mark Ibrahim and Diane Bouchacourt and Pascal Vincent and Ari S. Morcos},
      year={2023},
      eprint={2308.03977},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}