Tiny Models

Small generative models built and trained from scratch to run locally in the browser.

Current Models

• Semantic model: DiT that generates a landscape image from a drawing (from a semantic palette). More info in the blog post

semantic_gen_demo.mp4

• Edit model: DiT for text-based image editing. For the text encoder, I used a pretrained sentence-transformers/all-MiniLM-L6-v2

Project Layout

• frontend/ contains the code for the web app (React + TS) and deployed assets (e.g., models). Models of size > 100 MB must be split into parts in order to get around GitHub's file size limit (the path to each part must be included in frontend/constants.ts)
• pipelines/ contains the code for modeling, training, sampling, and exporting to produce the ONNX checkpoints