Skip to content

kennymckormick/DrawBench-Glance

Repository files navigation

DrawBench at a Glance

This repo compares images generated by awesome generative models on the DrawBench provided in Imagen. Currently, it includes images generated with some OpenSource image generative models (stable diffusion 1-4, 1-5, 2-1). One can browse the generated images and get a rough idea about the capability of these models.

Features

  1. The repo includes results generated by 3 stable diffusion models (1-4, 1-5, 2-1) with 200 prompts from DrawBench, organized by prompt categories.
  2. The provided scripts can be used to generate images with additional prompts and private models and provide organized results (the README files in each sub-directory display the results).

Limitations

  1. Currently, this repo only includes 3 OpenSource stable diffusion models.
  2. Currently, this repo only includes images generated with one random seed (0), which makes the comparison not comprehensive.

Generate the images on your own

  1. Create a conda environment: conda env create -f bench.yaml.
  2. Activate the conda environment: conda activate bench.
  3. Run the notebook GenImages.ipynb. It takes ~30 minutes to generate a single image for 200 prompts with 3 different diffusion models on a single RTX3090.

Preview

Each sub-directory includes the generation results of a single prompt category. Some examples are provided below as a preview.

Category: Color
1:A red colored car.
CompVis-stable-diffusion-v1-4 runwayml-stable-diffusion-v1-5 stabilityai-stable-diffusion-2-1
Category: Conflicting
26:A horse riding an astronaut.
CompVis-stable-diffusion-v1-4 runwayml-stable-diffusion-v1-5 stabilityai-stable-diffusion-2-1
Category: Counting
36:One car on the street.
CompVis-stable-diffusion-v1-4 runwayml-stable-diffusion-v1-5 stabilityai-stable-diffusion-2-1
Category: DALL-E
55:A triangular purple flower pot. A purple flower pot in the shape of a triangle.
CompVis-stable-diffusion-v1-4 runwayml-stable-diffusion-v1-5 stabilityai-stable-diffusion-2-1
Category: Descriptions
75:A vehicle composed of two wheels held in a frame one behind the other, propelled by pedals and steered with handlebars attached to the front wheel.
CompVis-stable-diffusion-v1-4 runwayml-stable-diffusion-v1-5 stabilityai-stable-diffusion-2-1
Category: Gary Marcus et al.
95:Paying for a quarter-sized pizza with a pizza-sized quarter.
CompVis-stable-diffusion-v1-4 runwayml-stable-diffusion-v1-5 stabilityai-stable-diffusion-2-1
Category: Misspellings
105:Tcennis rpacket.
CompVis-stable-diffusion-v1-4 runwayml-stable-diffusion-v1-5 stabilityai-stable-diffusion-2-1
Category: Positional
115:A train on top of a surfboard.
CompVis-stable-diffusion-v1-4 runwayml-stable-diffusion-v1-5 stabilityai-stable-diffusion-2-1
Category: Rare Words
135:Acersecomicke.
CompVis-stable-diffusion-v1-4 runwayml-stable-diffusion-v1-5 stabilityai-stable-diffusion-2-1
Category: Reddit
142:A church with stained glass windows depicting a hamburger and french fries.
CompVis-stable-diffusion-v1-4 runwayml-stable-diffusion-v1-5 stabilityai-stable-diffusion-2-1
Category: Text
180:A storefront with 'Hello World' written on it.
CompVis-stable-diffusion-v1-4 runwayml-stable-diffusion-v1-5 stabilityai-stable-diffusion-2-1