Where should I obtain the lora-sdxl-dreambooth-id in Inference

[Zarato2122]

Describe the bug

I tried to upload the download link from the README file generated during training, but an error indicated it was incorrect. Where should I obtain the lora-id for Inference?

Reproduction

README.md:

base_model: /data/ziqiang/czc/diffusers/examples/dreambooth/model
library_name: diffusers
license: openrail++
instance_prompt: a photo of sks dog
widget: []
tags:

• text-to-image
• text-to-image
• diffusers-training
• diffusers
• lora
• template:sd-lora
• stable-diffusion-xl
• stable-diffusion-xl-diffusers

---

SDXL LoRA DreamBooth - daniu111/output

Model description

These are daniu111/output LoRA adaption weights for /data/ziqiang/czc/diffusers/examples/dreambooth/model.

The weights were trained using DreamBooth.

LoRA for the text encoder was enabled: False.

Special VAE used for training: /data/ziqiang/czc/diffusers/examples/dreambooth/model/vae.

Trigger words

You should use a photo of sks dog to trigger the image generation.

Download model

Weights for this model are available in Safetensors format.

Download them in the Files & versions tab.

Intended uses & limitations

How to use

# TODO: add an example code snippet for running this diffusion pipeline

Limitations and bias

[TODO: provide examples of latent issues and potential remediations]

Training details

[TODO: describe the data used to train the model]

Inference:
from huggingface_hub.repocard import RepoCard
from diffusers import DiffusionPipeline
import torch

lora_model_id = <"lora-sdxl-dreambooth-id">
card = RepoCard.load(lora_model_id)
base_model_id = card.data.to_dict()["base_model"]

pipe = DiffusionPipeline.from_pretrained(base_model_id, torch_dtype=torch.float16)
pipe = pipe.to("cuda")
pipe.load_lora_weights(lora_model_id)
image = pipe("A picture of a sks dog in a bucket", num_inference_steps=25).images[0]
image.save("sks_dog.png")

"The lora-dreambooth-sdxl-id seems to need to be uploaded, but I don't know where to obtain this ID."

Logs

No response

System Info

• 🤗 Diffusers version: 0.32.0.dev0
• Platform: Linux-5.4.0-198-generic-x86_64-with-glibc2.31
• Running on Google Colab?: No
• Python version: 3.12.4
• PyTorch version (GPU?): 2.4.0 (True)
• Flax version (CPU?/GPU?/TPU?): not installed (NA)
• Jax version: not installed
• JaxLib version: not installed
• Huggingface_hub version: 0.26.2
• Transformers version: 4.46.3
• Accelerate version: 1.1.1
• PEFT version: 0.7.0
• Bitsandbytes version: not installed
• Safetensors version: 0.4.5
• xFormers version: 0.0.27.post2
• Accelerator: NVIDIA RTX A6000, 49140 MiB
  NVIDIA RTX A6000, 49140 MiB
  NVIDIA RTX A6000, 49140 MiB
• Using GPU in script?:
• Using distributed or parallel set-up in script?:

Who can help?

@hlky

[hlky]

lora_model_id = <"lora-sdxl-dreambooth-id">

Hub path or local path e.g.

lora_model_id = "/my/local/path"

or

lora_model_id = "username/repo_id"

To use it from the Hub, upload the folder, there's probably an option like --push-to-hub on the training script which will upload it for you at the end of the training run.

cc @linoytsaban @sayakpaul for training.

[Zarato2122]

I set the training output to be saved in the "output" folder, but I didn't find the model file in the "output" folder. Directly providing the path to "output" results in an IsADirectoryError. So, where should I fill in the path for lora-sdxl-dreambooth-id in Inference if I only want to use a local model without uploading it to Hugging Face?
@sayakpaul

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

[hlky]

@Zarato2122

pipe.load_lora_weights("output/pytorch_lora_weights.safetensors")

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.