M3 Ultra 256GB - macOS Sequoia, serious memory leak

[djtuBIG-MaliceX]

I've got this working on both Windows and MacOS installing it identically with the instructions on the main README.md.
Everything works, however on macOS after 1 or 2 generations it would cause the entire macOS to crash and restart.

On closer inspection, noticed during genning, RAM usage would shoot up from a normal 10-20GB all the way to 249GB, and more often than not exceeding the 256GB capacity then hitting swap which freezes and crashes.

I don't think that's supposed to happen. On Windows, only notice the VRAM usage go up to 14.9GB / 16GB (NVIDIA RTX5070Ti) and anything over I've only ever seen it hit 2-3GB extra on 'shared memory', either way much smaller than whatever is going on for the mac.

Might need some further investigation but I don't think this would be stable for macOS for now until the memory consumption issue can be mitigated.

[pk-1999]

I also experienced problems with memory overflow and system crashes/restarts at the beginning. However, I found a closed issue #308 that offered a solution. After applying it, the memory usage has consistently remained around 60% over multiple runs. Maybe you can try it.

[djtuBIG-MaliceX]

Thanks, can confirm the torch MPS watermark threshold env var workaround works. 60% RAM usage over multiple runs is more than plenty headroom with the 256GB Mac Studio.

Probably worth documenting as it doesn't appear to be a problem for non-Mac users.

For those that don't quite understand:

1. Set the following env vars in your terminal session BEFORE starting the web UI:

export PYTORCH_ENABLE_MPS_FALLBACK=1
export PYTORCH_MPS_LOW_WATERMARK_RATIO=0.6
export PYTORCH_MPS_HIGH_WATERMARK_RATIO=0.8 

(for me, low of 0.2 and high of 0.3 is sufficient for M3 Ultra 256GB)

2. Run the webserver as normal.