M4 Max 36GB: Facet memory usage reaches 50GB+ on MPS

[maisun]

I'm testing Facet on an M4 Max with 36GB unified memory and the auto 16GB profile.
With a batch size of only 10 photos, memory starts around 8–16GB but keeps increasing to around 50GB, then drops back below 8GB and repeats for the next batch. Swap can reach ~20GB.
Is this expected on Apple Silicon/MPS, or is there a recommended setting to keep memory usage within the 36GB physical RAM?

[ncoevoet]

Hello

Thanks for the detailed report and the screenshot.

Facet's memory checks don't see Mac memory use, and i do not have an macos to do experiments and fixings things:

What drives it:

1. Mac GPU memory is released only between groups of models.
2. Some models use far more memory per step than necessary.
3. Facet's memory checks don't see Mac memory use. the The auto-tuner reads free RAM without counting swap or GPU memory. The VLM tagger only ever clears NVIDIA memory, and its out-of-memory fallback only catches NVIDIA errors.

What you can do:

This isn't expected behaviour. Facet loads several models together on a Mac and doesn't release their memory between them, so a batch of photos can use much more memory than your Mac has. these settings should keep memory usage down. Try them in order, and stop once memory stays under your 36 GB.

1. Limit the memory PyTorch may use. Start the scan like this:
   PYTORCH_MPS_HIGH_WATERMARK_RATIO=1.0 python facet.py /path/to/photos
   This tells PyTorch not to reserve more than your Mac's recommended memory. If a scan then stops with an "out of memory" error, go to step 2.

2. Turn off the two heaviest optional models. Open scoring_config.json in your Facet folder (create it if it doesn't exist) and add:
   {
   "models": {
   "profiles": {
   "16gb": { "saliency_enabled": false }
   }
   },
   "iqa_extended": { "qrealign": false }
   }

• The first line turns off subject detection, which works out what the main subject is and whether it's sharp and well placed. It is the single biggest memory user. Your photos are still scored for aesthetics, composition, faces and technical quality.
• The second turns off Q-ReAlign, an extra image-quality model. The other quality models still run.

If your scoring_config.json already has content, merge these keys into it rather than replacing the file.

3. Process fewer photos at a time. Add this to the same file:
   {
   "processing": {
   "ram_chunk_size": 4,
   "auto_tuning": { "min_ram_chunk_size": 4 }
   }
   }
   You need both lines. Otherwise Facet raises the batch back to 10 on its own.

Scans get a little slower with step 3. Steps 1 and 2 have almost no effect on speed.

It would help a lot if you could send:

• the output of python facet.py --doctor;
• the output of python -c "import torch; print(torch.mps.recommended_max_memory() / 2**30)";
• whether step 1 on its own was enough.

In order to start working on a workaround.

[ncoevoet]

Staying under 25 GB with every model on is exactly what the branch was meant to do.

Everything below is on fix/mps-memory-release now, so a git pull on the branch is all you need:

The crash while loading: Facet now sets HF_DEACTIVATE_ASYNC_LOAD=1 by itself on macOS, so you can drop the variable from the command line. Models load a few seconds slower, one at a time. The crash is a random race between threads in transformers, which is why a reboot or a second run seemed to fix it. Please tell me if it ever comes back after the pull.

The bos_token_id / eos_token_id warnings: harmless. transformers checks those IDs against SigLIP's text vocabulary and the config fails that check, but the IDs only matter when a model generates text. Facet only uses SigLIP to turn images and search text into embeddings.

Merged HDR / panorama DNGs: the RAW decoder Facet uses can't read Lightroom's merged DNGs, so Facet now scores them from the preview Lightroom embeds in the file, extracted with exiftool. The preview must be at least 1024 px on its long side. If DSC03469-Pano.dng and the others are still skipped after the pull, please paste the output of:

exiftool -a -G1 -PreviewImage -JpgFromRaw -OtherImage -ImageSize DSC05485-HDR.dng

Bigger faces in the burst comparison: the face panel now has a Size slider next to Eyes and Smile (64 to 256 px, 112 px by default, it was fixed at 64 px). Your browser remembers it. Two related changes while I was there:

• after dragging any of those sliders, the arrow keys go back to moving between frames instead of nudging the slider;
• a hover loupe: move the mouse over the photo (or a compare pane, or a face) and it magnifies the full-resolution pixels under the cursor. The magnifier button in the toolbar turns it off.

Rebuild the web client after the pull and restart the viewer, or those UI changes won't show:

cd client && npm install && npx ng build && cd ..

If you can share the DNG that can't be loaded i would appreciate, in order to properly load it

[maisun]

Hi @ncoevoet :

Thank you for the quick fix! I pulled the latest fix/mps-memory-release branch and did a fresh scan.

The MPS memory fix looks great: with all models enabled, including Saliency and Q-ReAlign, peak memory stays under 25 GB on my 36 GB Mac. This is a big improvement compared with the previous version.
The startup crash also seems fixed. I no longer need the HF_DEACTIVATE_ASYNC_LOAD=1 workaround.
The face UI improvement is also very useful. Being able to increase the face size all the way to 256 px makes the burst comparison much better, especially for checking eyes and expressions. The hover loupe is useful as well.
The only remaining issue I found is the merged HDR/panorama DNG fallback. These files are still failing with:
Unsupported file format or not RAW file
For example:
DSC05485-HDR.dng
The exiftool output is:
[IFD0] Preview Image : (Binary data 10065 bytes, use -b option to extract)
[Composite] Image Size : 4672x7008

So the DNG definitely contains a PreviewImage, and the reported image size is well above the 1024 px requirement.
The other merged HDR/panorama DNGs show the same RAW decoder error.
I'll continue testing the rest, but the memory/startup fixes look very good.

[ncoevoet]

great some improvements ! could you share with me non working raw file ? per email ? nicolas.coevoet at gmail

thanks

[ncoevoet]

Thanks for testing everything so thoroughly. Your exiftool output shows exactly what went wrong with the merged DNGs, and my last reply was wrong about them.

The 10 KB PreviewImage is a 256 px thumbnail. The 4672×7008 Image Size is the merged image itself, not the preview. Since version 13, Lightroom Classic compresses every image inside a merged HDR/Panorama DNG as JPEG XL, previews included. The only ordinary JPEG left is that thumbnail, which is below the 1024 px Facet needs, so the exiftool fallback still skipped DSC05485-HDR.dng and the others. I reproduced this with a real Lightroom Classic 13.1 merge from an α7 IV.

This is fixed in 1.17.0 "Fusion", which also contains everything from the fix/mps-memory-release branch. Facet now decodes the 1024 px JPEG XL preview stored in the file. It needs two new packages, so update with:

git checkout master && git pull
pip install -r requirements.txt
cd client && npm install && npx ng build && cd ..

Then rescan the folder that holds the merged DNGs. No need to email me the file. If one is still skipped, please paste the output of:

exiftool -a -G1 -Compression -ImageWidth -ImageHeight DSC05485-HDR.dng