[Bug]: Safety checker is enabled for LoRA training with no way to disable it

[levicki]

Is there an existing issue for this?

• I have searched the existing issues and checked the recent builds/commits of both this extension and the webui

What happened?

Yesterday I updated to the latest version of the extension and I tried training a LoRA on some NSFW content.

The class images generated were all black, and most of them weren't even being saved to the classifiers output folder.

I checked, and under the model\dreambooth\MODEL_NAME\working there was a safety_checker folder with 1.13 GB model.safetensors file and config.json with commit hash cb41f3a270d63d454d385fc2e4f571c487c253c5 and name/path CompVis/stable-diffusion-safety-checker.

Also, I noticed that model_index.json in the working folder now contains:

{
  "_class_name": "StableDiffusionPipeline",
  "_diffusers_version": "0.19.3",
  "feature_extractor": [
    "transformers",
    "CLIPFeatureExtractor"
  ],
  "requires_safety_checker": true,
  "safety_checker": [
    "stable_diffusion",
    "StableDiffusionSafetyChecker"
  ],
# cut for brevity

Before in my older LoRA trainings (from the same base model) it looked like this:

    "_class_name": "StableDiffusionPipeline",
    "_diffusers_version": "0.16.1",
    "feature_extractor": [
        null,
        null
    ],
    "requires_safety_checker": null,
    "safety_checker": [
        null,
        null
    ],
# cut for brevity

Steps to reproduce the problem

1. Try training a NSFW LoRA with the latest version of AUTOMATIC1111 and this extension

Commit and libraries

Initializing Dreambooth
Dreambooth revision: cf086c5
Successfully installed fastapi-0.94.1

[+] xformers version 0.0.20 installed.
[+] torch version 2.0.1+cu118 installed.
[+] torchvision version 0.15.2+cu118 installed.
[+] accelerate version 0.21.0 installed.
[+] diffusers version 0.19.3 installed.
[+] transformers version 4.30.2 installed.
[+] bitsandbytes version 0.35.4 installed.

Command Line Arguments

--xformers --api

Console logs

Not relevant because there is no error, it's just undesirable behavior.

Additional information

No response

[saunderez]

Are you using txt2img or diffusers for image generation? If its diffusers it might be us but if it's txt2img its very likely automatic1111. Let me know either way.

[levicki]

@saunderez

It makes absolutely no difference which generator I use for classifier images -- as soon I click Create Model button in Dreambooth tab, folders feature_extractor and safety_checker are created in the models\dreambooth\MODEL_NAME\working folder which weren't being created before.

They are both referenced in model_index.json and safety_checker is enabled by default.

I can generate NSFW images in AUTOMATIC1111 with the same [filewords] from input images used for classifier images just fine, they are safety checked only in Dreambooth tab and nowhere else.

EDIT: It seems that's coming from the diffusers package:

https://github.com/search?q=repo%3Ahuggingface%2Fdiffusers+requires_safety_checker&type=code

[saunderez]

Got you. I don't think this should be happening unless the safety checker is actually specified. It seems to be fixed in dev already coz my model folders don't have it. Let me take a look.

[saunderez]

I replicated your problem and have a workaround but need to find a proper fix.

We aren't enabling it in code so I suspect its from the default config. That config gets loaded with the model which enables it anywhere the pipeline is used that doesn't have it explicitly disabled. Rather than add a bunch of lines to disable it everywhere a pipeline could load it we can just remove it from the config completely. Ideally this should be at model creation but we can fix it in post for now.

All you have to do is either delete the safety_checker section in model_index.json located in the working_dir of the model or set it's parameters to null.

This is the section to delete, and how it appears nulled out. Yours will have the set to true and valid network names for it.

"requires_safety_checker": null,
"safety_checker": [
null,
null
],

Once it's been neutered you can safely delete the entire safety_checker folder.

[levicki]

@saunderez

Thanks for looking into it.

Yes, I figured out that workaround before creating an issue, or otherwise I wouldn't have been able to provide as much details as I did.

That said, I am looking forward to the proper fix so I don't have to edit the file for every model I create because that's not the only problem — it also wastes bandwidth and makes useless disk writes (1.13 GB model.safetensors for safety_checker is added to every created model).

On a side note, I think you should also properly disable feature_extractor since it seems to be only used for safety_checker.

Finally, it would be nice if someone looked at #1334 and tried to reproduce, I can't train extended LoRA anymore and I'd like to know what the cause might be. The only thing different from before is update to A1111 and Dreambooth.

[paboum]

All I can say, even if you really need NSFW filters to opinionate morality of your code, it should at least yield a warning in a console! Or abort generation! Generating a BLACK rectangle is racist btw.

[morphinapg]

Instead of manually changing this every time, is there a simple change to the code we can make to get it to work automatically?

[levicki]

@saunderez Any ETA on a proper fix for this one?

[paboum]

I just run:

sed -i 's/requires_safety_checker": true/requires_safety_checker": null/' */working/model_index.json
sed -i 's/"stable_diffusion"/null/' */working/model_index.json
sed -i 's/"StableDiffusionSafetyChecker"/null/' */working/model_index.json

in my stable-diffusion-webui/models/dreambooth dir.

Besides, I haven't been able to train one model ever with this tool, using it just to generate class images. In https://github.com/bmaltais/kohya_ss the safety checks are simply ignored by default and nobody cares.

[levicki]

@paboum I don't have sed on Windows and even if I did (which would require installing WSL which in turn requires Hyper-V which messes up VirtualBox), I wouldn't want to have to run it for every model I create.

[paboum]

https://learn.microsoft.com/en-us/answers/questions/656069/sed-alternative-in-windows

[levicki]

https://learn.microsoft.com/en-us/answers/questions/656069/sed-alternative-in-windows

As I said, I am not interested in manual workarounds. This shouldn't be difficult to fix.

[github-actions]

This issue is stale because it has been open 5 days with no activity. Remove stale label or comment or this will be closed in 5 days

[levicki]

This issue is stale because it has been open 5 days with no activity. Remove stale label or comment or this will be closed in 5 days

Bump.

[github-actions]

This issue is stale because it has been open 5 days with no activity. Remove stale label or comment or this will be closed in 5 days

[levicki]

@saunderez This is coming from StableDiffusionPipeline — it is instanatiated with default parameters and defaults are to have both safety checker and feature extractor, not to mention that the requires_safety_checker bool is by default true in StableDiffusionPipeline constructor.

So yes, it's coming from you using the defaults. It affects both classification image and sample image generation during training.