[bug]: crash while restoring faces on M1

[whosawhatsis]

Is there an existing issue for this?

• I have searched the existing issues

OS

macOS

GPU

mps

VRAM

32GB

What happened?

The backend errors out and becomes unresponsive after trying to run face restoration (either GFPGAN or CodeFormer). This is with the current version installed using this fork (because mac installation is currently broken on this repo): https://github.com/damian0815/InvokeAI/tree/fix_macos_installer

Note, this does not seem to be the same error as #1451

Screenshots

Traceback (most recent call last):
File "/Users/who/Downloads/InvokeAI/ldm/generate.py", line 467, in prompt2image
results = generator.generate(
File "/Users/who/Downloads/InvokeAI/ldm/invoke/generator/base.py", line 98, in generate
image_callback(image, seed, first_seed=first_seed)
File "/Users/who/Downloads/InvokeAI/backend/invoke_ai_web_server.py", line 761, in image_done
image = self.codeformer.process(
File "/Users/who/Downloads/InvokeAI/ldm/invoke/restoration/codeformer.py", line 50, in process
face_helper.get_face_landmarks_5(resize=640, eye_dist_threshold=5)
File "/Users/who/Downloads/InvokeAI/.venv/lib/python3.10/site-packages/facexlib/utils/face_restoration_helper.py", line 139, in get_face_landmarks_5
bboxes = self.face_det.detect_faces(input_img, 0.97) * scale
File "/Users/who/Downloads/InvokeAI/.venv/lib/python3.10/site-packages/facexlib/detection/retinaface.py", line 227, in detect_faces
keep = py_cpu_nms(bounding_boxes, nms_threshold)
File "/Users/who/Downloads/InvokeAI/.venv/lib/python3.10/site-packages/facexlib/detection/retinaface_utils.py", line 41, in py_cpu_nms
keep = torchvision.ops.nms(
File "/Users/who/Downloads/InvokeAI/.venv/lib/python3.10/site-packages/torchvision/ops/boxes.py", line 40, in nms
_assert_has_ops()
File "/Users/who/Downloads/InvokeAI/.venv/lib/python3.10/site-packages/torchvision/extension.py", line 33, in _assert_has_ops
raise RuntimeError(
RuntimeError: Couldn't load custom C++ ops. This can happen if your PyTorch and torchvision versions are incompatible, or if you had errors while compiling torchvision from source. For further information on the compatible versions, check https://github.com/pytorch/vision#installation for the compatibility matrix. Please check your PyTorch version with torch.version and your torchvision version with torchvision.version and verify if they are compatible, and if not please reinstall torchvision so that it matches your PyTorch install.

Additional context

No response

Contact Details

No response

[damian0815]

when invoke starts do you get an error like:

/Users/damian/1.actions/invokeai installer test3 with spaces/InvokeAI/.venv/lib/python3.10/site-packages/torchvision/io/image.py:13: 
UserWarning: Failed to load image Python extension: 
dlopen(/Users/damian/1.actions/invokeai installer test3 with spaces/InvokeAI/.venv/lib/python3.10/site-packages/torchvision/image.so, 0x0006): 
Symbol not found: 
(__ZN2at4_ops19empty_memory_format4callEN3c108ArrayRefIxEENS2_8optionalINS2_10ScalarTypeEEENS5_INS2_6LayoutEEENS5_INS2_6DeviceEEENS5_IbEENS5_INS2_12MemoryFormatEEE)

?

[whosawhatsis]

Starting the InvokeAI browser-based UI..
* Initializing, be patient...
>> Initialization file /Users/who/.invokeai not found. Applying default settings...
/Users/who/Downloads/InvokeAI/.venv/lib/python3.10/site-packages/torchvision/io/image.py:13: UserWarning: Failed to load image Python extension: dlopen(/Users/who/Downloads/InvokeAI/.venv/lib/python3.10/site-packages/torchvision/image.so, 0x0006): Symbol not found: __ZN2at4_ops19empty_memory_format4callEN3c108ArrayRefIxEENS2_8optionalINS2_10ScalarTypeEEENS5_INS2_6LayoutEEENS5_INS2_6DeviceEEENS5_IbEENS5_INS2_12MemoryFormatEEE
  Referenced from: <06D2C3BD-26E5-3AB9-A866-63839BE393A7> /Users/who/Downloads/InvokeAI/.venv/lib/python3.10/site-packages/torchvision/image.so
  Expected in:     <0BB917BA-0D8F-3429-B798-476DA4A619BD> /Users/who/Downloads/InvokeAI/.venv/lib/python3.10/site-packages/torch/lib/libtorch_cpu.dylib
  warn(f"Failed to load image Python extension: {e}")
NOTE: Redirects are currently not supported in Windows or MacOs.

That's the only thing that looks like an error that I'm seeing during startup.

[damian0815]

ok, thanks. @tildebyte any idea? same issue as i saw, OP’s error message is more informative than mine. note how torchvision is loading a .so (=standard linux dll extension) while libtorch is a .dylib (=standard macos dll extension) (this seems correct). is something messed up in the source where pip is downloading these from?

[damian0815]

fwiw the sha256sum of image.so (ff2f67c1f072dc7234642f2613bfc73ce3bd8cb1c30c39c7e0cc0fc87b8edcb0) is identical in the .venv folder after installing to what i get if i use homebrew's pip (i.e. non-conda) to install torchvision==0.13.1. conda's image.so has a different sha256sum though.

[tildebyte]

I have no clue 🤷
It is, at the very least, bizarre that something is trying to load a .so...

[whosawhatsis]

I just tried the install using invokeAI-src-installer-mac.zip, and face restoration seems to be working.