[Bug]: M1 Mac: Performance degrages severely after 1 generation #5488
Comments
|
Attempted a third generation. It is currently reporting step 5/20 with 3:46 elapsed and 10:50 estimated, so I expect similar results. EDIT: took 13:54. |
|
Additional information: This doesn't appear to be limited to the 2.0 model. Experimentally I tried cancelling the Terminal process, adding the 1.4 model via a symlink, and relaunching. With identical settings aside from selecting the 1.4 model, I'm getting identical performance: first batch took 2:34, second batch is currently at 5/20 after 3:45 with an ETA of 17:41. |
|
Using this software on an ARM-based mac is going to give you trouble, guaranteed. Not only is the newer Macs ARM-based (RISC which is most often used in network attached storage devices, cars and coffe makers) but also these CPUs use Apple proprietary GPUs which isn't really supported by this software (basically you need CUDA to make it work without issues). So get yourself an older Intel mac, put in a Geforce card, or wait until someone magically makes this work on a computer that looks best on a fancy desk. (Sorry, biased, but Apple decided to screw their fans over once more). |
And yet...I didn't have this problem even with this software as recently as 11 days ago. Nor do I have this problem with literally any other implementation I've tried on this machine. So shit on Apple all you want, but I don't think this is an Apple problem, per se. |
|
It is: trouble with the available compiled modules for Python. AI specific modules that are made to work on nVidia hardware which Apple no longer uses af they were thrown in the garbage bin by nVidia (for whatever reason, but likely because Apple wanted them to make proprietary stuff for them).
One of them is torch which is compiled specifically for CUDA which is an nVidia thing but there are a number of modules giving Mac M1/M2 users issues. Ready easy out, use compatible hardware or ask Apple to make their own (proprietary of course) Apple Diffusion.
Seriously though.
A number of Mac users have reported issues while it worked for them earlier. I have no idea why that is but it might be memory or I don't know. But I imagine that it had to do with the Apple GPU which cannot handle CUDA instructions or tries to, but fails.
Did you try to set up the ui to render cpu only to see if that works?
Only other suggestion I have (seriously) is as mentioned: you need supported hardware and basically torch only supports nVidia.
Other than that, try to do a render with the stable diffusion repo (the base one) and check if that works. If so, it is this UI that fails, but I am pretty sure they dont really aim to support the ARM-based Macs as the architecture is completely different and many Py modules are likely not ready to support them.
|
|
then why didn't I have this issue with this software before it integrated support for SD 2.0? why don't I have this problem with any other implementation (now up to 4 and counting) that I've tried on this machine? Your firey rhetoric neither fits the facts nor gets any closer to a solution to a problem that is unique to this implementation, so kindly shut up if you can't contribute something useful. |
|
I am the same problem, 2nd generation is 4 times slower, same settings different batch. |
Working on Mac requires a lot of memory, it seems. Perhaps more on the later release. But torch is only built for cuda and (it seems) AMD GPUs do I guess you need the memory to run both Python, the ui and it's modules AND then you need memory for generation. When I run the UI on Windows with 32 GB ram, I have maybe 35% left when it has started. Loading a model, hypernet, dream etc will take more. It transfers to the GPU as needed, freeing to the ram on the PC. Since torch is not really built for the Mac gpu, perhaps that's the issue. Slowing down due to high memory usage. Did you try to check ram and cpu usage while doing first and second generation? |
|
Open activity viewer and look at your memory pressure. I have a 32GB RAM Studio here and it's pretty peppy (for a Mac running SD) but once the memory usage gets beyond the physical RAM, it will start swapping to the relatively fast onboard SSD, but you get a serious performance hit in the meantime. I can get up to 40-some odd GB in use before that happens. Close all other applications and let it run. You can free up GPU RAM by turning off GPU acceleration in your browser. The unified memory is great as in it will let you continue to run SD even when running out of memory, but you will get dinged for it performance wise. |
|
I have read that's how it is on Mac. |
I have done all this. I've also done all this while using other implementations. Consistently Auto1111 is the only implementation with this problem. I'm not disputing that the unified memory on the Mac causes issues, I'm stating that from all available information, something Auto1111 specifically is doing is causing this lack of optimization. It doesn't happen in InvokeAI. It doesn't happen in DiffusionBee. It doesn't even happen in Draw Things, which is an iPad app sort of retrofit to run on the computer. It's a problem that is unique to Auto1111. |
|
That may be true, but the way Mac uses resources is the same for all. |
|
I don't see this issue running any checkpoint with the default 512x512 size. I've had this run for 12+ hours generating images with no performance hits since a bit over two weeks ago, when the big M1 update in Automatic came out. Prior to that I had issues with it crapping out with a semaphore issue or some sort of fault. Here is the log from what I have had it do this evening: Here it is with 768x1152, hiresfix: This is just big enough to get it to swap slightly, but no slowdown (last two are slightly longer due to my opening and using Safari). If I have this run all night, it may eventually give me an issue. I'll let it got for a few hours and see what it does. I'll update if I see anything. EDIT: |
|
I'm on M1 Pro, 16 gb ram. Here's my results running images today. Euler A, CFG 7, highres fix, 768x1024, 20 steps. When the "fast" ones in the results below finish, there's no image. The image only shows up from the super slow ones. Now I realize it takes longer to make bigger images, but this seems ridiculous. Please correct me if I'm wrong. Granted, being able to produce amazing works of art that fast is wonderful, but it does make it take a long time to cherry pick from hundreds of results. |
|
Those speeds (the slow ones) correspond with the time it takes to generate on cpu only on Windows - with a CPU that is about four generations old. Knowing it's unlikely that the Apple GPU supports torch, I guess this means that Apple users should try it on Linux, or get capable hardware. |
|
Here's my results for the same. I have 32GB of RAM: What I noticed on my machine the python process is using slightly more than 23GB of RAM with those settings. EDIT:
As you can see with these settings, the performance went into the toilet because it's swapping like mad. My swapfile fluctuated between 12 and 20GB with 100% RAM utilization while rendering this image. In fact, the 768x1024 size you mentioned earlier is very near the largest size I can do on this 32GB machine without it swapping. cooperdk: |
|
Thanks for running tests and sharing for comparison. Edit: After running another batch at 640x960, instead of averaging in the 120s, I was somewhere more around the 20s. Slightly smaller, but significantly faster. |
|
I bought my Studio a month before SD came out. If I knew then what I knew now, I'd have gotten a 64GB unit. I'm not majorly upset about it, as I usually make images as experimentation so 512x512 and 512x768 (and thereabouts) is perfectly fine for me. There's an upper limit bug in the M1s too, as I have tried and let it make larger images, but once you get above 1024x1024, you hit another bug that crashes the program - see #5278 for that one. SD on M1 came a long way since it was first released. I'm really hoping there's some good optimizations that can be wrung out of it over the next year. |
|
I have the earliest model M1(16g), I tried the v2.1 512 model and remove the argument --medvram, it seems working 'normally' again (like before using 1.5 models). |
My command line settings are: It ignores the skip CUDA test, but does the rest of them. |
You mentioned earlier you don't see the interim image. If you go to settings and set the value for "Show image creation progress every N sampling steps" to 1 (most frequent) or some other number, it will show the image after each N iteration steps. There is a performance penalty for this, but I find it minimal (maybe a few sec per image). I keep mine at 1 so when I generate that lovely supermodel and it turns out to be a horrid lobstrosity, I can interrupt the image generation and fix the prompt without waiting for it to finish first. This can be a huge time saver. |
cooperdk, do you seriously have nothing better to do with your life than dunk on hardware in the specific thread created to troubleshoot for it? I'm truly sorry your soul is that empty, but please shut up if you have nothing useful to contribute. |
|
Same issue here, several months after this thread was created. I didn't have any issues for weeks, 14 or so days up until 2-3 years ago, when A1111 suddenly began crawling. My machine got hot, for the first time, so I figured perhaps my machine was throttling, so I stopped generating for a while. This was 3 days ago, but ever since, I can no longer do more than 2-3 generations before everything grinds to a halt. I've tried different browsers, no difference, same issue, fast at first and then grinds to a halt. I've tried restarting (of course), I've tried reinstalling, no change. I don't know what it is but it is making SD unusable for me now, which is sad, because I really want to use it like I have for the past 14 days. Here is an example of the times I get: https://i.imgur.com/34IZISm.png. The message close to the start is me activating ControlNet openpose, but I've tried without activating it, it doesn't matter. After a few generations my instance grinds to a halt and I can no longer generate properly. I've also tried different instances and this only seems to happen on A1111. @twopiearr did you guys find a solution to this? It's been a few months now after all. Thank you! |
|
@ptppan see an incredibly helpful thread here: #5461 (reply in thread) and no anti-apple bigots, either! |
Haha awesome, thank you! Did you find anything in particular that helped you with the issue you described in this thread? Because I am having a similar issue. Looking through that thread, I am not sure what could actually solve this issue, and I'm not seeing any comments by you there? Did you do something recommended in that thread in particular that helped you with your issue? I am actually unsure if I actually have an issue now or if it's just the usage of LoRAs that slow things down (naturally), or if it perhaps is ControlNet that runs slow on Mac, or whatever it could be. Really confused about this issue, as it seems so vague. But love to know if you followed a particular advice in that thread? Thank you again for providing it! Edit: Nvm, stupid me, just saw your posts in the thread. Thank you so much, I'll proceed from there. |
|
Same issue here, Since last Sunday morning, it was able to generate a picture in 7 minutes, but it suddenly slowed down by four times. However, there is one detail I noticed. A few days ago, when everything was normal, I exited the webui with control + z , closing the terminal is directly closed, but after this happens, and then close the terminal, it will prompt me that python is still running, are you sure to close it |
|
|
|
after contrl + z, the Python still occupies 30g of memory and has not released it |
|
the Python version is Python 3.10.10 (main, Feb 8 2023, 05:34:50) [Clang 14.0.0 (clang-1400.0.29.202)] |
|
Python does not shut down if you close the terminal window. You have to break it from running. But the web ui was not made to run on Mac. You really should get proper gear to use it. |
|
Seriously, @cooperdk - get a hobby or something. |
Aye, on that note, Draw Things runs nicely on a Mac. |
You didn't stop it (says so in the screen 'suspended'), it's still running in background. to stop press CTRL-C |
|
Experiencing the same on my M1 air 8GB. |
|
This is really not written to execute on a Mac, but for Nvidia hardware. You're bound to get issues. |
Try this. Works great on Mac mini M2 8GB. Had similar problems with the default args.
It's been working great for several days, many sessions, hundreds of images generated averaging about 50 seconds per image. Let us know how it goes. Thanks. |
I gave it a shot on my M1 Mac Studio. The "opt-sub-quad-attention" argument causes "modules.devices.NansException" which with your other flag, generates blank (black) images. No idea why this is happening. |
Is there an existing issue for this?
What happened?
Did a fresh install this morning to make sure I was running the latest and greatest. Generating a 768x768 image at 20 steps with Euler_a using the 2.0 model. Launched with the --medvram argument but got similar results without it. Batch count and batch size both at 1. The first batch generated in 1:56; the second batch, with identical settings, took 17:04. Cancelling and relaunching the Terminal command does seem to get it back to the initial performance, but only for one batch, thus necessitating quitting and relaunching every time.
Steps to reproduce the problem
What should have happened?
The generation time for the second batch should be more or less the same as the time for the first batch.
Commit where the problem happens
44c46f0
What platforms do you use to access UI ?
MacOS
What browsers do you use to access the UI ?
Mozilla Firefox
Command Line Arguments
Additional information, context and logs
For the purposes of this bug report I launched with --medvram, mostly to see if it would improve performance as suggested in the Apple Installation article on the wiki. With this argument,the first generation happened in approx half the time, but similar results overall were observed in the second generation. (I don't actually know how long the second generation will take without the --medvram argument; the UI was estimating 30 mins, and I cancelled it after 20.)
The text was updated successfully, but these errors were encountered: