[Bug]: Latest release breaks symlinks in checkpoint directory

[powerfulparadox]

Is there an existing issue for this?

• I have searched the existing issues and checked the recent builds/commits

What happened?

I store my checkpoints on another drive and have the folder symlinked as a subfolder in the "models/Stable-diffusion" directory. I did this after first run, so I have the automatically downloaded 1.5 checkpoint in the normal place. After updating today, I only see the 1.5 checlpoint; none of the others on the other drive show up anymore.

Only the checkpoints are affected. I also have similar setups with hypernetworks, Loras, and Lycoris, but all of them still work as expected.

Steps to reproduce the problem

1. Have checkpoints in a symlinked subdirectory of the models folder
2. attempt to select a different model
3. None of the models in the symlinked directory appear

What should have happened?

All models should appear for selection

Commit where the problem happens

94a759b

What platforms do you use to access the UI ?

Linux

What browsers do you use to access the UI ?

Google Chrome

Command Line Arguments

--opt-sub-quad-attention --sub-quad-q-chunk-size 512 --sub-quad-kv-chunk-size 512 --sub-quad-chunk-threshold 80 --listen --api --enable-insecure-extension-access

List of extensions

PBRemTools
SD-latent-mirroring
Stable-Diffusion-Webui-Civitai-Helper
StylePile
a1111-sd-webui-lycoris
clip-interrogator-ext
sd-dynamic-prompts
sd-webui-ar
sd-webui-aspect-ratio-helper
sd-webui-controlnet
sd-webui-infinite-image-browsing
sd-webui-regional-prompter
sd-webui-tunnels

Console logs

Python 3.11.3 (main, Apr  5 2023, 15:52:25) [GCC 12.2.1 20230201]
Version: <none>
Commit hash: 94a759bd9f3d48be54bf4da54fa97ec637088c3f
Installing requirements
Error running install.py for extension /home/benjamin/stable-diffusion-webui-ux/extensions/PBRemTools.
Command: "/home/benjamin/stable-diffusion-webui/venv/bin/python" "/home/benjamin/stable-diffusion-webui-ux/extensions/PBRemTools/install.py"
Error code: 1
stdout: Installing onnxruntime-gpu for PBRemTools

stderr: Traceback (most recent call last):
  File "/home/benjamin/stable-diffusion-webui-ux/extensions/PBRemTools/install.py", line 16, in <module>
    launch.run_pip(f'install {package}', desc=f'{package} for PBRemTools')
  File "/home/benjamin/stable-diffusion-webui-ux/launch.py", line 143, in run_pip
    return run(f'"{python}" -m pip {command} --prefer-binary{index_url_line}', desc=f"Installing {desc}", errdesc=f"Couldn't install {desc}", live=live)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/benjamin/stable-diffusion-webui-ux/launch.py", line 111, in run
    raise RuntimeError(message)
RuntimeError: Couldn't install onnxruntime-gpu for PBRemTools.
Command: "/home/benjamin/stable-diffusion-webui/venv/bin/python" -m pip install onnxruntime-gpu --prefer-binary
Error code: 1
stdout: <empty>
stderr: ERROR: Could not find a version that satisfies the requirement onnxruntime-gpu (from versions: none)
ERROR: No matching distribution found for onnxruntime-gpu






Installing sd-dynamic-prompts requirements.txt


#######################################################################################################
Initializing Civitai Link
If submitting an issue on github, please provide the below text for debugging purposes:

Python revision: 3.11.3 (main, Apr  5 2023, 15:52:25) [GCC 12.2.1 20230201]
Civitai Link revision: c6251a12120ac9f946fcaf8d9cb0a0f19176d9e6
SD-WebUI revision: 94a759bd9f3d48be54bf4da54fa97ec637088c3f

Checking Civitai Link requirements...
[+] python-socketio[client] version 5.7.2 installed.

#######################################################################################################


Launching Web UI with arguments: --opt-sub-quad-attention --sub-quad-q-chunk-size 512 --sub-quad-kv-chunk-size 512 --sub-quad-chunk-threshold 80 --listen --api --enable-insecure-extension-access
/home/benjamin/stable-diffusion-webui/venv/lib/python3.11/site-packages/torch/cuda/__init__.py:546: UserWarning: Can't initialize NVML
  warnings.warn("Can't initialize NVML")
No module 'xformers'. Proceeding without it.
Error loading script: api.py
Traceback (most recent call last):
  File "/home/benjamin/stable-diffusion-webui-ux/modules/scripts.py", line 257, in load_scripts
    script_module = script_loading.load_module(scriptfile.path)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/benjamin/stable-diffusion-webui-ux/modules/script_loading.py", line 11, in load_module
    module_spec.loader.exec_module(module)
  File "<frozen importlib._bootstrap_external>", line 940, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/home/benjamin/stable-diffusion-webui-ux/extensions/PBRemTools/scripts/api.py", line 9, in <module>
    from scripts.main import process_image, model_list
  File "/home/benjamin/stable-diffusion-webui-ux/extensions/PBRemTools/scripts/main.py", line 14, in <module>
    from scripts.td_abg import get_foreground
  File "/home/benjamin/stable-diffusion-webui-ux/extensions/PBRemTools/scripts/td_abg.py", line 5, in <module>
    from sklearn.cluster import KMeans, MiniBatchKMeans
ModuleNotFoundError: No module named 'sklearn'

Error loading script: main.py
Traceback (most recent call last):
  File "/home/benjamin/stable-diffusion-webui-ux/modules/scripts.py", line 257, in load_scripts
    script_module = script_loading.load_module(scriptfile.path)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/benjamin/stable-diffusion-webui-ux/modules/script_loading.py", line 11, in load_module
    module_spec.loader.exec_module(module)
  File "<frozen importlib._bootstrap_external>", line 940, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/home/benjamin/stable-diffusion-webui-ux/extensions/PBRemTools/scripts/main.py", line 14, in <module>
    from scripts.td_abg import get_foreground
  File "/home/benjamin/stable-diffusion-webui-ux/extensions/PBRemTools/scripts/td_abg.py", line 5, in <module>
    from sklearn.cluster import KMeans, MiniBatchKMeans
ModuleNotFoundError: No module named 'sklearn'

Error loading script: sa_mask.py
Traceback (most recent call last):
  File "/home/benjamin/stable-diffusion-webui-ux/modules/scripts.py", line 257, in load_scripts
    script_module = script_loading.load_module(scriptfile.path)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/benjamin/stable-diffusion-webui-ux/modules/script_loading.py", line 11, in load_module
    module_spec.loader.exec_module(module)
  File "<frozen importlib._bootstrap_external>", line 940, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/home/benjamin/stable-diffusion-webui-ux/extensions/PBRemTools/scripts/sa_mask.py", line 13, in <module>
    from segment_anything import SamAutomaticMaskGenerator, sam_model_registry
ModuleNotFoundError: No module named 'segment_anything'

Error loading script: td_abg.py
Traceback (most recent call last):
  File "/home/benjamin/stable-diffusion-webui-ux/modules/scripts.py", line 257, in load_scripts
    script_module = script_loading.load_module(scriptfile.path)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/benjamin/stable-diffusion-webui-ux/modules/script_loading.py", line 11, in load_module
    module_spec.loader.exec_module(module)
  File "<frozen importlib._bootstrap_external>", line 940, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/home/benjamin/stable-diffusion-webui-ux/extensions/PBRemTools/scripts/td_abg.py", line 5, in <module>
    from sklearn.cluster import KMeans, MiniBatchKMeans
ModuleNotFoundError: No module named 'sklearn'

Civitai Helper: Get Custom Model Folder
Civitai Helper: Load setting from: /home/benjamin/stable-diffusion-webui-ux/extensions/Stable-Diffusion-Webui-Civitai-Helper/setting.json
Civitai Helper: No setting file, use default
ControlNet v1.1.170
ControlNet v1.1.170
Civitai: API loaded
Loading weights [6ce0161689] from /home/benjamin/stable-diffusion-webui-ux/models/Stable-diffusion/v1-5-pruned-emaonly.safetensors
Creating model from config: /home/benjamin/stable-diffusion-webui-ux/configs/v1-inference.yaml
LatentDiffusion: Running in eps-prediction mode
DiffusionWrapper has 859.52 M params.
Applying sub-quadratic cross attention optimization.
Textual inversion embeddings loaded(483): 1111r31nhart, 1auraba1137, 1M4NV3LL4N122, 1ucy1iu, 3113fann1ng3, 3mmar0b3rts, 3mwatson2010, 3N1DS1NCL41R , 3r1nm0r1art72, 4lineabboud, 4lizeej, 4lyd3bc4r, 4lyssadiaz, 4m0ur4nth, 4nn4k3ndrick, 4utumnf4lls, 512paintstyle1, 512paintstyle3, 512paintstyle3-light, 512speedpainter, 7dirtywords, 80sdarkfantasysd15b, abstract-concepts, adele, advntr, aid28, aidv1-neg, akira5, al3ssagigioli, alanahpearce1, alexahad-114500, alizjacot, alizjacot-1850, alyhan, alysonhannigan44, am0uranth, amazonmuscless, amberheard1, amgutierrez, AnaD, anastasiyakvitko, anders-zorn, ang0ur13r1c3, ann3hathawa72, ann3l3s3milton, antonialastnamesd15, anyachalotra, arcan3, arcan3v2, AriGra3V-383, art by EMB_skstest3, art by mc-mimana10-2300-3500, art by mc-mimana11c-13100, art by mc-mimana25-500, art by mc-minimana3, art by mc_mimana2, art by Smoose2, AS-Young, AS-Young-neg, AS-Younger, AS-Youngest, ashr3sch, atj1, atompunkstylesd15, awaitingtongue, b3bahan-775, B3THH4RM0N, B4R0N, B4R0N22, bad-artist, bad-hands-5, bad_pictures, bad_picturesv3, bad_prompt_version2, bad_prompt_version2-neg, badhandv4, badv3, badv4, badv5, bebahan, bfold3, bilphaniem, black-and-white-design, blakelively1, br1tsp, brandi love, brapit63, brielastnamdsd15, brila_noexist, bs_indiasummer, bukkakAI, By bad artist -anime-neg, By bad artist -neg, c0b13smu1d3rs2, c0div0re, c0urt400, cam11am3nd3s, caoimhelastnamesd15, cardstyle15, carpodlewski, centerfold512, charturnerv2, chefk1ki, cherrynobodysd15, chlmtz2, chloenobodysd15, chr1sch0ng, chrllo, cny1, concept-art, corneo_pov_oral, corychase-940, cutecreature, dak0tafann1ng3, danahamm, dangercrow, dangerdonkey, dangerhawk, deformityv6, deniselastnamesd15, DFCnegativeembed-1000, DFCnegativeembed-500, dilleotta, dollienobodysd15, dpthrt, DreamArtistBADHAND, DreamArtistBADHAND-neg, durer-style, EasyNegative, easynegative, edgerunners-style-v2, elaine_noexist, elegant-flower, eliselastnamesd15, eliselastnamesd151, emb-daisy, emb_tupac, emilyvancamp1, emrata91, emw100, evaelfie00, evalyn_noexist, evelynnobodysd15, evreka15, f311c1ada7, fairy-tale-painting-style, faithlastnamesd15, fantasycharacter, female-kpop-singer, Florenc3, follynobodysd15, foshposh, fr3yaa11an, fs2023, fukicommune, gavidp, GITSEllie, GITSKrystenR, glorianobodysd15, GnosticOutsider, gothbuilding, gracevandien1, gregnobodysd15, gustavedore, gustavedore(1), gwypaltrow72, hades-65800, Hail33, handmade2d, harstyles94, hbech, hitokomoru-style, hobbithouse, hoppagames, hotfantasyface, hotify, hrbkbc1sg, hrbkbc1sgv2i, izzycka, j0rd7nj0n3s, j3nn1f3rc0n-1800, j3nna0rt3ga, j3ssbush, jane_noexist, JeAni69, jencoleman86, jencon90s, jenn1f1850, jennachew2, jennifer_aniston, jennifer_connelly_v2.1, jg_4mouranth, jg_ashleyresch, jg_n4t4lee, jillmorph, jokermovie, jubbslineart_v2, jujunaught-66600, juliatica, k31rakn1ght1e7, k34nur33v3, k3rrikendall, k4l3ycu0c0, ka11m3kr1s, kaitlyndever1, kaleycuoco26, kargillan87, katclov, katd3nn1ngs2, kateesackhoff2, kateupton1, katewinslet1, kath3r1n3langf0rd2, kathr7nn3wt0n, kathrynnewton4, khyle-28200, kiaram00n, KieShi3v, killer95, KimberlyNobodySD15, KirDun69, kkw-ph1, kkw-ph1-neg, knsgmrtrnr, kodakvision500t, koh_abelladanger, Koh_AjApplegate, koh_amberheard, koh_angelawhite, Koh_AnnaHopkins, koh_daisyridley, koh_doutzenkroes, koh_emiliaclarke, Koh_FelicityJones, Koh_KatherineBarrell, koh_kimkardashian, Koh_KristannaLoken, koh_kristinarose, koh_lanarhoades, koh_lauralux, koh_lauralux_v1_1, koh_lisaann, koh_meghanm, koh_miakhalifa, Koh_MillaJovovich, koh_pokimane, koh_sashagrey, kristennobodysd15, l1vvydunne, laurencohan1, lesanbrandt, letitiawright1, liquid-light, lndswgnr_v1.0, logo-with-face-on-shield, LoonyLL-1400, LoonyLL-2100, LoonyLL-2700, LoonyLL-2800, LoonyLL-3000, LoonyLL-700, LOP, LPB2k1, lr, lynaritaa, m00nb, M11anaVa7ntrub2, m1chelley3oh, M1lly4c0k2222, m1ngnawen-600, m3ganb00n3245, m3kiac0x-500, m4dison1vy, mad31a1n3p3tsch, madclyne97, madisongx, mads65, MadZie3v, magicalinterior, MakeItPopVA, MakeItPopVB, MakeItPopVC, MakeItPopVD, MakeItPopVE, MakeItPopVF, mar1shara7, maranobodysd15, marigarn, marijameglaj, mayca1amaw7, mbglee, mbsuper, mc-mimana5-4000, melissabenoist2, melissafumero1, memphisstyle-6750, metropolis_e, mew4, michellemylett2, michellerabit, microkini, milajovovichyounger, milazul97, MIlBob3V, MilBob3vLR, millajovovichmodern, mirellapapciakk, misslexi, mmmggg-5820, moebius, morb4cc4r1n, morfyddclark1, mortnobody15, mrmd, mylvestre, n1c0l4c4, n1nad0br3v, n4t3mmanu3l, n4t4lee, nata113n031, natashalyonne1, nbdy-celeste-sd15, nbdy-isabella-sd15, nbdy-julia-sd15, nbdy-scarlet-sd15, nc4v4n1s, neg_grapefruit, negative_hand-neg, NG_DeepNegative_V1_75T, nnlnnx_1985-1987_v1.4g, norreedus69, NovaFrogStyle, novakdjokovic, np_simple_negatives, oldsiemens, olfn, optional_styleAI, orientalist-art, osisaac, p3cruz, p3yt0n11st, p4m2000s, paige2023, pam90s4nderson, paradoxical-back-view-bj, pastel_style, pen-ink-portraits-bennorthen, pervpulp15, phoebe-wulf, pinup512, piotr-jablonski, pjensen, PlanIt, poly-hd, povblowjob3v3, pppascal, prettify, prt, pureerosface_v1, pwool3n, r0samund1ik3, r0sd4wn-1325, ralph-mcquarrie, ravlyn95, rayne-weynolds, rdbj-9500, realcumAI, rebromijn, reverseSuspended_analVersion, RFKTR_bontrex-150, rfktr_bwmnga, rfktr_circrex, RFKTR_fshnsta, RFKTR_plastic, RFKTR_rootrex, rfktr_sinetempore, rhlib-3400, riae84, rinellis93, robdowney65, rosalinenobodysd15, roseleslie, RSC, ruby0fee, rubyof33, rurinobodysd15, rz-neg-15-foranalog, rzvice15, s0f1av3rgara, s0fiag0mez, s0ph1ad1mart1n0, S4BR1N422, S4D1S1NK22 , s4r4hsh4hi, s4runderwood, s4u700, SadSin2VH, salhayek66, SamDoesArt1, SamDoesArt2 , samdoesarts, sarah_noexist, sashanobodysd15, scajohansson84, schnucks, sd15_journalSketch, shantae_styleAI, shtw41n, shuimo , sierra2024, skullgirls-48600, sling_bikini, slotgames, SopTurN3v005, SouthOfTheBorderSD15, southofthebordersd15, space_style, star-tours-posters, steampunkgranny, StefaniaS_V1, StefaniaS_V2, SterreKV1, str4h0vski, str4h0vski2, strah0vsky, Style-BladeRunner2049-8v, Style-Glass, Style-Glorious, Style-Glorious-neg, Style-Info, Style-Italy, Style-Japan, Style-Kilt, Style-Kitchen, Style-Kitchen-neg, Style-Moana, Style-Moana-neg, Style-NebMagic, Style-NebMagic-neg, Style-Petal, Style-Petal-neg, Style-Renaissance, Style-Renaissance-neg, Style-TronLegacy-12v-v2, Style-TronLegacy-8v-A, Style-TronLegacy-8v-B, Style-Witcher3, sumgl4u, sumgl4u1, sydsweeney97, t3r4patrick, t3r4patrick-1050, t3ssaf0wler, t4ny4m4vr1n4, tarot512, tat1anamas1an7, taylorswift2, tcr3ws, ter4p4trick, tfftgirlv05, tffthairv07illu, tffthairv07nike, tffthairv07phto, theSacs, trinkdber, ulzzang-6500, Unspeakable-Horrors-Composition-4v, valorantstyle, vellalovell1, verybadimagenegative_v1.3, vintagemap_f, vladstudio, vz3m4n0v4, W33333DDDDD, W3DDDN3SD4Y, WAS-CBB, wassnonam-39400, xmsorn, youngk1m, yu-jinnobodysd15, yv0nn3, Z3NNND4Y42222, ZoeSal2VH
Textual inversion embeddings skipped(90): Apocofy, paintstyle8, ParchArt, kthrinelngfrdwhy, laxpeintV2, dangergoose, dangerhorse, dangerowl, DaveSpaceFour, DaveSpaceOne, rz-neg-foranalogportrait, ThisHonor, retrofutur, inksketchcolour1subtle, Inksplat768, IpScifi, IpScifiHelper, card-style, cardstylev3, CarHelper, cartoonish_doll, karsh, Neg_Facelift768, nfixer, nfixernext, ActionHelper, SCG768-Illustrate, SCG768-Nebula, nrealfixer, painted_landscape, PaintStyle1, PaintStyle3, PaintStyle4, paintstyle5, paintstyle6-neg, SocialistRealism, cgart, Chadeisson, CinemaHelper, GigaChad, HeartArt, illuminutty_nrealfixer, inksketchcolour1, kc16-v1-4000, crpht-4300, trInkdBer, UlukInkSketch2, UrbanJungle, VintageHelper, style-bridal_sd2, PhotoHelper, pinup768, pixelart-1, PlanIt2, PMondrian, HyperFluid, HyperNuke, HyperSmoke, illuminutty_nartfixer, illuminutty_nfixer, HorrorByDave, InkPunk768, InkPunkHeavy768, InkPunkLandscapes768, InkPunkLite768, DrD_PNTE768, FloralMarble, glitch, dblx, mdjrny-ppc, mj-gs, nartfixer, style-widow_sd2, style-princess_sd2, paintstyle6, PortraitHelper, 0xaisticker, 100memories, 21charturnerv2, Art by Smoose-22, ScaryMonsters, ScaryMonstersV2, SCG768-Euphoria, KittyPic, rz-neg-forbettercontrast, rz-neg-general, rzminjourney, rzviceb10, rzviceb6, tarot768
Model loaded in 4.3s (create model: 0.4s, apply weights to model: 0.7s, apply half(): 0.3s, load VAE: 0.1s, move model to device: 0.5s, load textual inversion embeddings: 2.1s).
Loading weights [6ce0161689] from /home/benjamin/stable-diffusion-webui-ux/models/Stable-diffusion/v1-5-pruned-emaonly.safetensors
Creating model from config: /home/benjamin/stable-diffusion-webui-ux/configs/v1-inference.yaml
LatentDiffusion: Running in eps-prediction mode
DiffusionWrapper has 859.52 M params.
Applying sub-quadratic cross attention optimization.
Model loaded in 1.9s (create model: 0.3s, apply weights to model: 0.5s, apply half(): 0.3s, move model to device: 0.5s, load textual inversion embeddings: 0.3s).
Running on local URL:  http://0.0.0.0:7860

To create a public link, set `share=True` in `launch()`.
Civitai: Check resources for missing preview images
Startup time: 14.2s (import torch: 1.0s, import gradio: 0.8s, import ldm: 3.4s, other imports: 0.4s, load scripts: 1.2s, create ui: 7.0s).
Civitai: Found 223 resources missing preview images
Civitai: Found 23 hash matches
Civitai: Updated 0 preview images

Additional information

I think that this issue on the Automatic1111 repo should be helpful: AUTOMATIC1111#10346

At least it looks pretty much identical to what I've had happen.

[anapnoe]

#141

[PaulCoddington]

This appears to be the most recent commit where Extra Networks still worked with symlinks:

Revision: f2b9c2c
Date: 12/05/2023 11:57:05 pm

Looks like symlinks were broken with the release candidate merge:

Revision: b08500c
Date: 13/05/2023 5:16:37 pm

Problem persists through to commit 3c31e25.

[powerfulparadox]

I saw #141, but I wasn't sure it was the same issue.

Also, can confirm that I still have the problem with 3c31e25.

[powerfulparadox]

Seems to be fixed in d6b4aeb.