[Bug]: After update, UI no longer detecting models in models/Stable-diffusion symlink subdirectory

[tdtrumble]

Is there an existing issue for this?

• I have searched the existing issues and checked the recent builds/commits

What happened?

I have a symlink subdirectory in models/Stable-diffusion with models organized into multiple subfolders. Prior to [b08500c], the UI detected all of these models without a problem, after [b08500c], it won't. I created a normal subfolder and put a model in there, no problems, so this issue is limited to symlinks.

Steps to reproduce the problem

1. create symlink subdirectory in models/Stable-diffusion and place model in there
   1.a EXAMPLE: ln -s 'something/SD_MODELS' 'something/stable-diffusion-webui/models/Stable-diffusion/SD_MODELS'
2. load webUI, notice model in symlink not detected

What should have happened?

webui should have detected the models in the symlink subfolder

Commit where the problem happens

[b08500c]

What platforms do you use to access the UI ?

Windows, Linux

What browsers do you use to access the UI ?

Microsoft Edge

Command Line Arguments

--listen --api --xformers

List of extensions

openOutpaint-webUI-extension	https://github.com/zero01101/openOutpaint-webUI-extension.git	5e84d6d5 (Mon Apr 10 23:01:41 2023)	unknown
sd-dynamic-prompts	https://github.com/adieyal/sd-dynamic-prompts.git	a53adac9 (Sun Apr 23 13:28:49 2023)	unknown
sd-webui-controlnet	https://github.com/Mikubill/sd-webui-controlnet.git	93b0f9e (Wed Apr 26 03:16:41 2023)	unknown
sd-webui-infinite-image-browsing	https://github.com/zanllp/sd-webui-infinite-image-browsing.git	d67fcf93 (Thu May 11 18:16:40 2023)	unknown
sd_dreambooth_extension	https://github.com/d8ahazard/sd_dreambooth_extension.git	926ae204 (Fri Mar 31 15:12:45 2023)	unknown
stable-diffusion-webui-Prompt_Generator	https://github.com/imrayya/stable-diffusion-webui-Prompt_Generator.git	0d4de20f (Fri Mar 17 11:22:15 2023)	unknown

Console logs

################################################################
Install script for stable-diffusion + Web UI
Tested on Debian 11 (Bullseye)
################################################################

################################################################
Running on tdtru user
################################################################

################################################################
Repo already cloned, using it as install directory
################################################################

################################################################
Create and activate python venv
################################################################

################################################################
Launching launch.py...
################################################################
Cannot locate TCMalloc (improves CPU memory usage)
Python 3.10.11 (main, Apr 20 2023, 19:02:41) [GCC 11.2.0]
Version: v1.2.0
Commit hash: b08500cec8a791ef20082628b49b17df833f5dda
Installing requirements
Installing sd-dynamic-prompts requirements.txt



If submitting an issue on github, please provide the full startup log for debugging purposes.

Initializing Dreambooth
Dreambooth revision: 926ae204ef5de17efca2059c334b6098492a0641
Successfully installed fastapi-0.94.1 gitpython-3.1.31 google-auth-oauthlib-0.4.6 transformers-4.26.1

Does your project take forever to startup?
Repetitive dependency installation may be the reason.
Automatic1111's base project sets strict requirements on outdated dependencies.
If an extension is using a newer version, the dependency is uninstalled and reinstalled twice every startup.

[+] xformers version 0.0.19.dev524 installed.
[+] torch version 1.13.1+cu117 installed.
[+] torchvision version 0.14.1+cu117 installed.
[+] accelerate version 0.18.0 installed.
[+] diffusers version 0.14.0 installed.
[+] transformers version 4.26.1 installed.
[+] bitsandbytes version 0.35.4 installed.


Launching Web UI with arguments: --listen --api --xformers
2023-05-13 12:52:52.605741: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2023-05-13 12:52:53.117996: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
==============================================================================
You are running torch 1.13.1+cu117.
The program is tested to work with torch 2.0.0.
To reinstall the desired version, run with commandline flag --reinstall-torch.
Beware that this will cause a lot of large files to be downloaded, as well as
there are reports of issues with training tab on the latest version.

Use --skip-version-check commandline argument to disable this check.
==============================================================================
ControlNet v1.1.112
ControlNet v1.1.112
Loading weights [6ce0161689] from stable-diffusion-webui/models/Stable-diffusion/v1-5-pruned-emaonly.safetensors
Creating model from config: stable-diffusion-webui/configs/v1-inference.yaml
LatentDiffusion: Running in eps-prediction mode
DiffusionWrapper has 859.52 M params.
Applying xformers cross attention optimization.
Textual inversion embeddings loaded(0): 
Model loaded in 2.8s (load weights from disk: 0.2s, create model: 0.3s, apply weights to model: 0.5s, apply half(): 0.3s, load VAE: 1.2s, move model to device: 0.3s).
Running on local URL:  http://0.0.0.0:7860

To create a public link, set `share=True` in `launch()`.
Startup time: 9.0s (import torch: 1.9s, import gradio: 0.3s, import ldm: 0.5s, other imports: 0.6s, load scripts: 4.3s, refresh VAE: 0.1s, create ui: 1.1s).
Closing server running on port: 7860
Restarting UI...
ControlNet v1.1.112
Running on local URL:  http://0.0.0.0:7860

To create a public link, set `share=True` in `launch()`.
Startup time: 1.7s (load scripts: 0.4s, create ui: 1.1s).
Closing server running on port: 7860
Restarting UI...
ControlNet v1.1.112
Running on local URL:  http://0.0.0.0:7860

To create a public link, set `share=True` in `launch()`.
Startup time: 1.8s (load scripts: 0.4s, create ui: 0.9s, gradio launch: 0.3s).
Closing server running on port: 7860
Restarting UI...
ControlNet v1.1.112
Running on local URL:  http://0.0.0.0:7860

To create a public link, set `share=True` in `launch()`.
Startup time: 1.4s (load scripts: 0.4s, create ui: 0.9s).
Calculating sha256 for stable-diffusion-webui/models/Stable-diffusion/test/v1-5-pruned-emaonly.safetensors: 6ce0161689b3853acaa03779ec93eafe75a02f4ced659bee03f50797806fa2fa
Loading weights [6ce0161689] from stable-diffusion-webui/models/Stable-diffusion/test/v1-5-pruned-emaonly.safetensors
Applying xformers cross attention optimization.
Weights loaded in 3.3s (calculate hash: 2.4s, load weights from disk: 0.2s, apply weights to model: 0.3s, move model to device: 0.4s).

Additional information

No response

[wallish77]

I can confirm this is happening on linux, but only for checkpoints. I also use symlinks for embeddings and LoRas (LoRas are also broken), but those are still detected without issue.

[Kadah]

Symlinks should be fixed on dev branch after cb3f8ff. That change did not get cherrypicked for the hotfix it seems.

Was struggling all night with various issues this was causing. LoRA loading/referencing had a few changes as well that complicated matters.

Edit: related to #5092

[tdtrumble]

Symlinks should be fixed on dev branch after cb3f8ff. That change did not get cherrypicked for the hotfix it seems.

Was struggling all night with various issues this was causing. LoRA loading/referencing had a few changes as well that complicated matters.

Edit: related to #5092

Yes thanks, made those two updates in my local repo and it worked perfectly!

[acncagua]

- for root, dirs, files in os.walk(path):
+ for root, dirs, files in os.walk(path, followlinks=True):
I made these two fixes in the local repository and confirmed that it works properly.
I hope you will submit an official PR.

[akx]

#10286 is in master (version 1.3.0), so this should be fixed now.