MacOS Sonoma 14.4 update breaks GPU acceleration on Apple Silicon

[francisjervis]

Steps to reproduce: Install CUI on MacOS 14.3.x. Update to latest version, 14.4.
Expected behavior: Generation of usable images
Issue: After update, ComfyUI outputs Rothko-esque solid color images (usually cyan or white), unless a ControlNet is used, in which case it outputs a ghostly image of the input on a solid color.

This has been reported to affect both CUI and other SD UI's since the beta version. However, Draw Things appears to function normally.

I have tried updating pytorch to the latest nightly which did not resolve the issue.

Also of note: "generation" time seems much shorter than before the update, tho the output is garbage.

[l0stl0rd]

Webui Forge works fine, does not seem slower either. No difference either if I use Torch 2.1.2 or the latest 2.2.1. I am on the latest 14.4 and did not have issues in beta either.

So I decided to install Comfy with the help of
Pinokio as I am lazy and Comfy does not have a nice Mac install script.

You are right all it generates is blue solid pictures.

InvokeAi works fine too so not sure why Comfy does not.

[SharmaTushar]

I just did a clean install of everything related to ComfyUI and it's still broken. Didn't imagine MacOS update would be the culprit.

[francisjervis]

If you search for "sonoma 14.4 stable diffusion" you will find a number of different reports of issues with the beta. However none of them seems to line up with the blue square output behavior with the release version.

[SharmaTushar]

Seeing that InvokeAI works fine and it uses venv, I decided to use it for ComfyUI as well. I've used it with Python3.11 and only for installing ComfyUI requirements and torch.
It unfortunately didn't work.
Would Miniconda need to be included in this somehow? I'm not sure what steps would need to be followed for that.

[francisjervis]

Would Miniconda need to be included in this somehow? I'm not sure what steps would need to be followed for that.

Honestly can't see how that would be relevant, if that fixed anything I would assume it was a version-specific incompatibility between 14.4 and a pytorch nightly or something (which was specified in the requirements.txt you used to build the venv but not user-wide, or something like that). My best guess is that one of the modules used an undocumented approach to optimizing something which was (without release notes) broken by Apple - so this is likely in a dependency like torch, but which one is anyone's guess at this point.

This issue in Automatic1111 might be relevant, but the behavior isn't the same (glitchy, not solid cyan).

[SharmaTushar]

Honestly can't see how that would be relevant

That's all I could conjure from my very limited python knowledge.

This AUTOMATIC1111/stable-diffusion-webui#5461 (comment) might be relevant, but the behavior isn't the same (glitchy, not solid cyan).

Switching to different samplers has indeed fixed a part of the issue. The initial generation is glitchy but iterative upscale improves it quite a bit.

[francisjervis]

Switching to different samplers has indeed fixed a part of the issue.

Which checkpoint? Using SDXL-Lightning here which requires Euler sampler, the output with others is always garbage.

[SharmaTushar]

So far I've only tried using SDXL 1.0 (aamXLAnimeMix_v10) with dpmpp_2m_sde_gpu. The final output still has artefacts; I've been noticing them even before the OS update, though to a lesser extent.

[francisjervis]

I just tested with and without the --cpu flag and it's immediately clear this is an mps/GPU acceleration issue.


Workflow should be embedded for reproducibility but I expect this to work on any workflow. Inspired by fixing the same issue with local LLM generation the same way...

[criyle]

I did a fresh install of comfyUI without any modification or custom_node and confirms that the workflows worked before the macOS upgrade stops working. The command line argument --cpu helps but much slower...

[powerAmore]

confirms that --cpu works for MacOS Sonoma 14.4
Device: mps will generate a pure color image on MacOS Sonoma 14.4

[pphoto808]

After updating Sonoma to 14.4 and some testing, ComfyUI seems to work with only SDE samplers. Also ControlNet nodes not working.

[francisjervis]

After updating Sonoma to 14.4 and some testing, ComfyUI seems to work with only SDE samplers. Also ControlNet nodes not working.

Try with the --cpu flag and make sure --force-fp16 is not set. That fixed ControlNet for me.

[untitledfile01]

Confirming that some SDE samplers do work. dmpp_sde and dmpp_sde_gpu are outputting clean results. dmpp2_sde, dmpp2_sde_gpu, dmpp3_sde, and dmpp3_sde_gpu will output, but it's not pretty.

[cmgaston]

From a very handmade and casual test, it seems to me that it's not a matter of a single sampler or a single scheduler but the combination of both: some combinations work, other don't. My results (M1 8GB):

Work (as expected):

• dpmpp_sde / sgm_uniform
• dpmpp_sde / normal
• euler / ddim_uniform
• dpm_sde_gpu / sgm_uniform
• dpm_sde_gpu / karras
• dpmpp_2m_sde / karras

Don't work (blurry and weird):

• dpmpp_sde / karras
• dpmpp_2m / karras
• euler, euler_a / normal
• euler, euler_a / sgm_uniform
• euler / karras
• euler / simple
• ddim / normal
• ddpm / normal
• ddpm / karras
• heun / normal

[powerAmore]

From a very handmade and casual test, it seems to me that it's not a matter of a single sampler or a single scheduler but the combination of both: some combinations work, other don't. My results (M1 8GB):

Work (as expected):

• dpmpp_sde / sgm_uniform
• dpmpp_sde / normal
• euler / ddim_uniform
• dpm_sde_gpu / sgm_uniform
• dpm_sde_gpu / karras
• dpmpp_2m_sde / karras

Don't work (blurry and weird):

• dpmpp_sde / karras
• dpmpp_2m / karras
• euler, euler_a / normal
• euler, euler_a / sgm_uniform
• euler / karras
• euler / simple
• ddim / normal
• ddpm / normal
• ddpm / karras
• heun / normal

+1

[CharosTao]

I temporarily downgraded torchvision to version 0.16.2 to resolve the issue. Below is my conda env environment.yml, and I am using Python 3.10.

conda install torchvision=0.16.2 -c pytorch

Host:

MacOS M1 Sonoma 14.4

[cmgaston]

I temporarily downgraded torchvision to version 0.16.2 to resolve the issue. Below is my conda env environment.yml, and I am using Python 3.10.

This works for me too and solves the blurry images with SD 1.5 models (thanks!). Half of the problem is now gone.

I still have insanely high generation times (over 100s/it) with SDXL models though, as if – in this case too – there’s no GPU acceleration (speed is the same as with che --cpu flag, more or less 10-15 times higher than usual). Unfortunately this is not changed by the torchvision downgrade so something else must also be involved.

FWIW, with SD 1.5 models result was the issue, not speed – at least not in my case.

Edit: of course my issue with SDXL models might be related with my 8GB of ram, but still it didn't happen on macOS 14.3.x. If this is the issue I'll live, it makes sense, so I wonder if anyone with beefier Macs (16GB or more) is experiencing the same.

Edit 2: Forget about the SDXL part, I dared to upgrade my main Mac (M2 Pro 16GB) and speed is fine. I guess it's just a matter of memory after all.

[francisjervis]

I temporarily downgraded torchvision to version 0.16.2 to resolve the issue. Below is my conda env environment.yml, and I am using Python 3.10.

This works for me too and solves the blurry images with SD 1.5 models (thanks!). Half of the problem is now gone.

Might be useful to add which sampler/scheduler pairs this "fix" worked for, looking at @cmgaston's post.

[cmgaston]

Might be useful to add which sampler/scheduler pairs this "fix" worked for, looking at @cmgaston's post.

In my case, it works for all the combos I tried

[chikovani97]

is there any way to get it?

[untitledfile01]

I'm using an M1 Ultra 64gb and using dmpp_sde_gpu/normal works fine with SDXL and other models. No change in speed or anything.

[chikovani97]

its already giving me dark solid color results, so I think its already affected

[QueryType]

I'm using an M1 Ultra 64gb and using dmpp_sde_gpu/normal works fine with SDXL and other models. No change in speed or anything.

So you just upgraded? No downgrade of torchvision?

[bluevisor]

M1 Macbook Pro, macOS 14.4, with torchvision==0.16.2, most checkpoints work, with a few exceptions.

This took 100 seconds, seems normal for SDXL.

[QueryType]

M1 Macbook Pro, macOS 14.4, with torchvision==0.16.2, most checkpoints work, with a few exceptions. This took 100 seconds, seems normal for SDXL.

So we need to wait for a torchvision fix?

[criyle]

I tried to upgrade to macOS 14.4.1 and it is still not working properly.

Are you sure you have downgraded to torchvision==0.16.2?

I tried to install torchvision==0.16.2 with python3.11 and it resolves the problem. However, I think downgrading to torchvision==0.16.2 automatically downgraded torch to 2.1.2, so I did a quick search and the root cause might be: pytorch/pytorch#122016

[viking1304]

This issue in Automatic1111 might be relevant, but the behavior isn't the same (glitchy, not solid cyan).

It is exactly the same problem, just with a slightly different faulty resulting image. Some samplers (mostly ancestral ones) do not work properly on A1111 and Forge on 14.4 and 14.4.1 with torch 2.2.0 or newer. In CPU mode, they give a proper image, but it takes forever.

Torch 2.1.2 works without a problem, even on updated macOS.

Temporary workaround:

pip install torch==2.1.2 torchvision==0.16.2

Those do not work on macOS 14.4 or newer:

pip install torch==2.2.0 torchvision==0.17.0
pip install torch==2.2.1 torchvision==0.17.1
pip install torch torchvision # same as above
pip install --pre torch torchvision --index-url https://download.pytorch.org/whl/nightly/cpu # development version

So, we are currently stuck with Torch 2.1.2 on our Macs. I was on 2.2.1 before the update but had to revert all my installs to 2.1.2.

[francisjervis]

So, we are currently stuck with Torch 2.1.2 on our Macs. I was on 2.2.1 before the update but had to revert all my installs to 2.1.2.

Looking at this SO answer, it should be possible to add a Mac-specific line to the requirements.txt file which will temporarily fix this by specifying 2.1.2. Will try this and submit a PR if it works.

[sagebru5h]

I'm running into this same issue using ComfyUI on a M3 MAX running Sonoma 14.4.1.
Running ComfyUI in CPU-only mode is unbearably slow.

Any tips on how to get Torch 2.1.2 installed?

pip is returning a No matching distribution found error :

and conda is returning a bunch of incompatibilities...

I'm running Python 3.12.2

[axel-backlund]

@sagebru5h You need Python 3.11 for torch 2.1.2

[sagebru5h]

@sagebru5h You need Python 3.11 for torch 2.1.2

Thank you! That solved it.

[hughescr]

Looks like the fix got committed today upstream in pytorch (I built pytorch/pytorch@05289a2 from source and comfy works again) but not yet merged to the 2.3 release branch -- nightlies should start working again soon hopefully, and with luck the fix will get into the 2.3 final release which is imminent - since Mac is completely broken without it, I imagine they'll get it in...

[viking1304]

torch 2.4.0.dev20240404 on macOS 14.4.1

🎉🎉🎉

[denisvanw]

Good news !
But does it mean that the fix won't be implemented in the 2.3 release ?

[criyle]

Good news ! But does it mean that the fix won't be implemented in the 2.3 release ?

I believe they will have backports for older versions and the README suggests nightly version of PyTorch for apple silicon anyway.

[martintomov]

This issue in Automatic1111 might be relevant, but the behavior isn't the same (glitchy, not solid cyan).

It is exactly the same problem, just with a slightly different faulty resulting image. Some samplers (mostly ancestral ones) do not work properly on A1111 and Forge on 14.4 and 14.4.1 with torch 2.2.0 or newer. In CPU mode, they give a proper image, but it takes forever.

Torch 2.1.2 works without a problem, even on updated macOS.

Temporary workaround:

pip install torch==2.1.2 torchvision==0.16.2

Those do not work on macOS 14.4 or newer:

pip install torch==2.2.0 torchvision==0.17.0
pip install torch==2.2.1 torchvision==0.17.1
pip install torch torchvision # same as above
pip install --pre torch torchvision --index-url https://download.pytorch.org/whl/nightly/cpu # development version

So, we are currently stuck with Torch 2.1.2 on our Macs. I was on 2.2.1 before the update but had to revert all my installs to 2.1.2.

Kudos! Fixed my issue

[hughescr]

Patch got merged for the 2.3 release now, so will be in 2.3

[denisvanw]

Ah, ok, great !
Does that mean that we have to wait for the official release date ?
Or is there a way to access non final versions before that ?
(sorry I'm new to this, I'm on the "artist" side, not a developer !

[viking1304]

Ah, ok, great ! Does that mean that we have to wait for the official release date ? Or is there a way to access non final versions before that ? (sorry I'm new to this, I'm on the "artist" side, not a developer !

You can stay on 2.1.2 for now until the release of 2.3.0. Then, you can upgrade to 2.3.0.

If you want to use the development version, uninstall the current torch and install nightly bild like this:

pip uninstall torch torchvision
pip install --pre torch torchvision --index-url https://download.pytorch.org/whl/nightly/cpu

[denisvanw]

Hello !

It seems that in my case pip does not want to install torch 2.1.2 !?
It get this error message :

ERROR: Could not find a version that satisfies the requirement torch==2.1.2 (from versions: 2.2.0, 2.2.1, 2.2.2)
ERROR: No matching distribution found for torch==2.1.2

... and I get it even after uninstalling torch and torchvision !
If someone can help a struggling non-developer lost amidst all those pips gits and pulls...??
That would be greatly appreciated !
(I'm on Mac and I installed Comfy in a venv)

denis

This issue in Automatic1111 might be relevant, but the behavior isn't the same (glitchy, not solid cyan).

It is exactly the same problem, just with a slightly different faulty resulting image. Some samplers (mostly ancestral ones) do not work properly on A1111 and Forge on 14.4 and 14.4.1 with torch 2.2.0 or newer. In CPU mode, they give a proper image, but it takes forever.
Torch 2.1.2 works without a problem, even on updated macOS.
Temporary workaround:

pip install torch==2.1.2 torchvision==0.16.2
``
  
Those do not work on macOS 14.4 or newer:

pip install torch==2.2.0 torchvision==0.17.0
pip install torch==2.2.1 torchvision==0.17.1
pip install torch torchvision # same as above
pip install --pre torch torchvision --index-url https://download.pytorch.org/whl/nightly/cpu # development version

So, we are currently stuck with Torch 2.1.2 on our Macs. I was on 2.2.1 before the update but had to revert all my installs to 2.1.2.

Kudos! Fixed my issue

[criyle]

Hello !

It seems that in my case pip does not want to install torch 2.1.2 !? It get this error message :

ERROR: Could not find a version that satisfies the requirement torch==2.1.2 (from versions: 2.2.0, 2.2.1, 2.2.2) ERROR: No matching distribution found for torch==2.1.2

... and I get it even after uninstalling torch and torchvision ! If someone can help a struggling non-developer lost amidst all those pips gits and pulls...?? That would be greatly appreciated ! (I'm on Mac and I installed Comfy in a venv)

denis

This issue in Automatic1111 might be relevant, but the behavior isn't the same (glitchy, not solid cyan).

It is exactly the same problem, just with a slightly different faulty resulting image. Some samplers (mostly ancestral ones) do not work properly on A1111 and Forge on 14.4 and 14.4.1 with torch 2.2.0 or newer. In CPU mode, they give a proper image, but it takes forever.
Torch 2.1.2 works without a problem, even on updated macOS.
Temporary workaround:

pip install torch==2.1.2 torchvision==0.16.2
``
  
Those do not work on macOS 14.4 or newer:

pip install torch==2.2.0 torchvision==0.17.0
pip install torch==2.2.1 torchvision==0.17.1
pip install torch torchvision # same as above
pip install --pre torch torchvision --index-url https://download.pytorch.org/whl/nightly/cpu # development version

So, we are currently stuck with Torch 2.1.2 on our Macs. I was on 2.2.1 before the update but had to revert all my installs to 2.1.2.

Kudos! Fixed my issue

You need python3.11 rather than python3.12.

[denisvanw]

Thanx a lot for this quick answer !
The source I found about downgrading python suggested a simple "pip install python==3.11"
which gets the answer :

ERROR: Could not find a version that satisfies the requirement python==3.11 (from versions: none)
ERROR: No matching distribution found for python==3.11

??

[denisvanw]

pip uninstall torch torchvision
pip install --pre torch torchvision --index-url https://download.pytorch.org/whl/nightly/cpu

Hourra !
This fixed the problem : I'm getting my "beautiful scenery nature glass bottle landscape" again : )))

[lionwill]

i'm downgrade python for 3.1.10 and reinstall environment torch==2.1.2 torchvision==0.16.2，fixed some problem but only some k-samples like dpmpp-2m-sde-gpu , others still not work。maybe some add-on's problem?

and pic must in 10241024, if 512512 will not work currect

[denisvanw]

In my case, it was UPgrading pytorch to 2.3 that solved the problem.
All the K sampler are finctionnal.
No need to downgrade anymore, if i understand correctly !

[viking1304]

i'm downgrade python for 3.1.10 and reinstall environment torch==2.1.2 torchvision==0.16.2，fixed some problem but only some k-samples like dpmpp-2m-sde-gpu , others still not work。maybe some add-on's problem?

Try to use the dev version:
#2992 (comment)

In my case, it was UPgrading pytorch to 2.3 that solved the problem.

No, you are using dev (which is 2.4) 🙃 2.3 is not released yet, but I hope it will be released soon.

[lionwill]

i'm downgrade python for 3.1.10 and reinstall environment torch==2.1.2 torchvision==0.16.2，fixed some problem but only some k-samples like dpmpp-2m-sde-gpu , others still not work。maybe some add-on's problem?

Try to use the dev version: #2992 (comment)

In my case, it was UPgrading pytorch to 2.3 that solved the problem.

No, you are using dev (which is 2.4) 🙃 2.3 is not released yet, but I hope it will be released soon.

thx very much, it's seems solve the problem. i install the pre version2.4. i will test some samples later

[morics95]

Hi guys, im very new to all of comfy ui. I installed it on my M1 Mac with Sonoma 14.4.1.
In the beginning i didnt event recognize, that i have the blurry images bug caused by sonoma 14.

Now after a while i got the temporary/half fix done with using torchvision 16.2.

But i still have problems when using certain KSampler Settings.

Is there still NO official fix for this? Did anyone maybe upgrade his Mac OS to Squoia 15.0.1? Or is there any other fix - i realy would like to use lcm and sgm_uniform but it gives me blurry images...