Project ID: pLxS4aFn
Abstract: A Latent Diffusion Engine (LDE) is a machine learning model that generates images by gradually refining random noise into a coherent output. It operates in a latent space, where it reverses a diffusion process (turning noise back into data) to create high-quality results.
git clone https://github.com/epochlab/PLAYHOUSE
Navigate into the PLAYHOUSE directory and clone ComfyUI then install the required packages.
git clone https://github.com/comfyanonymous/ComfyUI.git
pip install -r requirements.txt
Update model directory paths: arch/extra_model_paths.yaml
Run:
python arch/execute.py
Define the parameters by editing the HyperConfig class found in the execute script.
| Parameter | Description |
|---|---|
| Model | The diffusion model choosen for inferance (eg. SDXL). |
| LoRA | Low-Rank Adaptation; a fine-tuning technique for reducing the number of parameters trained. |
| ControlNet | A network extension which controls image synthesis by adding extra conditions. |
| Upscale | Generate high-resolution images from lower-resolution inputs by predicting and adding finer details to the image. |
| Sampler | Direct the steps and processes by which noise is gradually refined into a coherent image. |
| Scheduler | Control the timing and magnitude of updates applied to the model, impacting how noise is reduced across iterations. |
| Denoise | The amount of denoise to apply to the latent source vector. |
| CFG | Classifier-free guidance; scale how much the image generation process follows the text prompt. |
Stable Diffusion XL
EDSR Upscaling
ControlNet | Depth & Canny
Use Houdini to dictate scene layout and generate the source imagery and required AOV's (Albedo, ZDepth, Normals and Curvature)
To ensure the generated images exhibit optical characteristics, use the Nuke framework to composite lens artifacts such as lens distortion, colour grading, noise/grain and chromatic aberration.
SDXL | 1024x1024, 1152x896, 896x1152, 1216x832, 832x1216, 1344x768, 768x1344, 1536x640, 640x1536
2.35 | 1280x545, 1920x816, 2048x871