Skip to content

S2D2: Simple Stable Diffusion based on Diffusers (Hires.fix on jupyter notebook, ipython or python interactive shell)

License

Notifications You must be signed in to change notification settings

keisuke-okb/S2D2

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

25 Commits
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

S2D2: Simple Stable Diffusion based on Diffusers

Diffusers-based simple generating image module with upscaling features for jupyter notebook, ipython or python interactive shell

Features

  • ☑ Just prepare safetensors files to go
  • ☑ Run Hires.fix without AUTOMATIC1111/stable-diffusion-webui
    • ☑ Latent Upscaler
    • ☒ GAN models
  • ☑ Multi-stage upscaling (extension of Hires.fix)
  • ☑ LoRA
  • ☒ Controlnet
  • ☒ Multi-batch generation (Only single generation is supported)

Schedule

  • Add ControlNet (Convert SD safetensors file to diffusers model files)

Getting Started

1. Install libraries

pip install -r requirements.txt

2. Prepare safetensors files of SD models and LoRA(option)

  • Only safetensors file is supported.
  • Place the files in the directory of your choice.

Ex.

3. Run jupyter notebook

cd s2d2
jupyter notebook

4. Import main class and load LoRA(option)

from s2d2 import StableDiffusionImageGenerator
generator = StableDiffusionImageGenerator(
    r"C:\xxx\Counterfeit-V30.safetensors",
)
# Load LoRA (multi files)
generator.load_lora(r"C:\xxx\lora_1.safetensors", alpha=0.2)
generator.load_lora(r"C:\xxx\lora_2.safetensors", alpha=0.15)

5. Generate image using enhance features(Hires.fix and its extended upscaling)

image = generator.diffusion_enhance(
          prompt,
          negative_prompt,
          scheduler_name="dpm++_2m_karras", # [1]
          num_inference_steps=20, # [2]
          num_inference_steps_enhance=20, # [3]
          guidance_scale=10,  # [4]
          width=700, # [5]
          height=500, # [6]
          seed=-1, # [7]
          upscale_target="latent", # [8] "latent" or "pil". pil mode is temporary implemented.
          interpolate_mode="bicubic", # [9]
          antialias=True, # [10]
          upscale_by=1.8, # [11]
          enhance_steps=2, # [12] 2=Hires.fix
          denoising_strength=0.60, # [13]
          output_type="pil", # [14] "latent" or "pil"
          decode_factor=0.15, # [15] Denominator when decoding latents. Used to adjust the saturation of the image during decoding.
          decode_factor_final=0.18215, # [16] Denominator when decoding final latents.
          )
image.save("generated_image.jpg) # or just "image" to display image in jupyter

Correspondence of web ui and parameters

image

Parameters

🚧🚧🚧🚧🚧Under construction🚧🚧🚧🚧🚧

  • Available schedulers are:
SCHEDULERS = {
    "unipc": diffusers.schedulers.UniPCMultistepScheduler,
    "euler_a": diffusers.schedulers.EulerAncestralDiscreteScheduler,
    "euler": diffusers.schedulers.EulerDiscreteScheduler,
    "ddim": diffusers.schedulers.DDIMScheduler,
    "ddpm": diffusers.schedulers.DDPMScheduler,
    "deis": diffusers.schedulers.DEISMultistepScheduler,
    "dpm2": diffusers.schedulers.KDPM2DiscreteScheduler,
    "dpm2-a": diffusers.schedulers.KDPM2AncestralDiscreteScheduler,
    "dpm++_2s": diffusers.schedulers.DPMSolverSinglestepScheduler,
    "dpm++_2m": diffusers.schedulers.DPMSolverMultistepScheduler,
    "dpm++_2m_karras": diffusers.schedulers.DPMSolverMultistepScheduler,
    "dpm++_sde": diffusers.schedulers.DPMSolverSDEScheduler,
    "dpm++_sde_karras": diffusers.schedulers.DPMSolverSDEScheduler,
    "heun": diffusers.schedulers.HeunDiscreteScheduler,
    "heun_karras": diffusers.schedulers.HeunDiscreteScheduler,
    "lms": diffusers.schedulers.LMSDiscreteScheduler,
    "lms_karras": diffusers.schedulers.LMSDiscreteScheduler,
    "pndm": diffusers.schedulers.PNDMScheduler,
}

Generated sample images

  • Used Counterfeit-V30.safetensors
  • Initial resolution: 696x496
  • Upscale factor: 1.8
  • Target resolution: 696x496(x1.8, nearest multiple of 8) = 1248x888

2-stage upscaling(Hires.fix)

image

N-stage upscaling(Ex.4)

Stepwise upscaling between the initial resolution and the target resolution. image

Comparison of generating images without or with latent upscaling

  • Without latent upscaling: Single generation@696x496

  • With latent upcscaling: 2-stage generation(like Hires.fix, 696x496 to 1248x888)

  • Prompt: "1girl, solo, full body, blue eyes, looking at viewer, hairband, bangs, brown hair, long hair, smile, blue eyes, wine-red dress, outdoor, night, moonlight, castle, flowers, garden"

  • Negative prompt: "EasyNegative, extra fingers, fewer fingers, bad hands"

image image image image image image image

References

About

S2D2: Simple Stable Diffusion based on Diffusers (Hires.fix on jupyter notebook, ipython or python interactive shell)

Topics

Resources

License

Stars

Watchers

Forks

Languages