Skip to content

Latest commit

 

History

History
72 lines (52 loc) · 5.92 KB

README.md

File metadata and controls

72 lines (52 loc) · 5.92 KB
Raw

Mosaic Silhouettes

NB: code was originally written in August 2018, and first published here in December 2018.

This project uses Google's pre-trained Inception-v3 model to generate vectors for an arbitrary directory of images (be it emojis, holiday happy-snaps, or even photos of your family). With these vectors, it applies a dimensionality reduction (either UMAP or t-SNE), and then maps those coordinates onto a silhouette mask while preserving the structure. Finally, a mosaic is generated using square thumbnails of the original images.

The net result is a gridded image, in the shape of a masked silhouette, where the image placement is clustered by image similarity (as according to the Inception-v3 vectors):

iOS Emojis Christmas time! Various coloured frogs

This is really just a combination & extension of a few existing projects:

Usage

  1. Place the images (JPG, PNG, or GIF) into a subfolder named img
  2. Find a mask to use (ideally a black [pixel=0] & white [pixel=255] PNG) and place it in the root directory
  3. Run python mosaic.py --mask mask_image.png

At this point the script will run through a variety of sub-routines, and save the progress as it goes (into output/vectors.json) incase something goes wrong. The broad steps are:

  1. Vectorise images
  2. Generate thumbnails
  3. Reduce dimensionality
  4. Load mask & determine scale
  5. Transform coordinates
  6. Produce image

Configuration

There are a variety of arguments available.

Argument Type Default Description
mask string frog_mask.png Filepath of the mask image to use (relative to the root directory)
mask_channel int 0 Which channel to use for the mask
mask_value int 0 Which integer value of the mask channel to identify as the silhouette
mask_output bool true Whether or not to save output/mask_debug.png to help visual identification of channel & values
vector_file str None Filepath (relative to the root directory) of pre-generated vectors.json. Use this if you want to skip this step in future runs to save time (for any given image the vector will never change)
thumb_dir str None Directory path (relative to the root directory) of thumbnail images. Use this if you want to skip this step in future runs to save time
thumb_size int 20 Size (in pixels) of the thumbnail square which is produced. Increase this if you want a higher resolution image
dim_red_skip bool False Skip the dimension reduction if it has already been done to save time. If this is selected it will read the reduced x and y coordinates from the output/vectors.json file
dim_red_type str umap Which dimensionality algorithm to use. Should be either umap or tsne
dim_red_output bool True Whether or not to output a scatter plot of the t-SNE and UMAP output
coord_output bool True Whether or not to output an image of the coordinate transforms

Output

For a successful run of the script, the following files are saved (in output/):

File Description Example
mosaic.png The actual mosaic silhouette PNG
vectors.json A JSON file containing the filename, image vector, dimension reduced coordinates (both UMAP and t-SNE), and gridded coordinates.
vectors.txt A tab-separated text flie containing all the features above except the vector. This can be used for easy post-analysis (e.g., df = pd.read_txt('vectors.txt', sep='\t').
mask_debug.png An image showing the various mask channels and an idea of their colour values. For further debugging, the top 5 values are further printed out in the stdout of the script.
mask_grid.png The mask channel, overlaid with the prospective gridded locations of the images
dimension_reduction.png A scatterplot showing the UMAP and t-SNE reducations, to aid with algorithm choice
coordinate_transform.png Shows the direction of movement from dimension reduced coordinates to the gridded mosaic coordinates

Download Google Images by keyword

The script download_images.py is provided as an example of how to easily download a bunch of images from Google Images, using the google_images_download (see https://github.com/hardikvasa/google-images-download for more details and installation instructions).

The main function as it stands runs download_example_frog_images() which just downloads 100 images of various coloured frogs. The other function download_single_keyword() (commented out) allows you to download any arbitrary keyword. It is recommended to see the official documentation for the full suite of options available.

For example, simply running python download_images.py will download 900 frog images into img/ as a starting example.