[Feature] Optional on-device ML processing in the mobile app

[chris3713]

I have searched the existing feature requests, both open and closed, to make sure this is not a duplicate request.

• Yes

The feature

It would be useful if the Immich mobile app could optionally run ML processing locally for assets that are already present on the device.

A fairly common setup is:

• Immich runs on a low-power NAS or home server
• photos are uploaded from a modern phone
• the phone has dedicated ML hardware that is significantly more capable than the server CPU

Right now, the phone uploads the original image and the server then runs face detection, smart search, etc.

In some setups this is backwards from a hardware point of view: the device that already has the image may have a powerful NPU / Neural Engine, while the Immich server is running on something like an N100, Raspberry Pi or low-end NAS CPU.

And this isn't just about phones having faster CPUs. Modern mobile SoCs have dedicated hardware specifically designed for running ML workloads efficiently. Apple devices, for example, have the Neural Engine, and Apple Photos already makes extensive use of on-device ML for things like scene classification and identifying people and pets.

So we already carry around devices that are specifically optimized for exactly this kind of workload, often while the server receiving the photos is intentionally optimized for low power consumption and storage rather than ML performance.

Immich already supports Remote Machine Learning, which helps with this, but it still requires running the machine-learning container with Docker on another machine.

There are also community solutions such as Immich Accelerator that can run ML natively on Apple Silicon, but that means installing and maintaining another service on the Mac, e.g. through Homebrew.

Both are useful options, but for newly uploaded photos the mobile app already has everything it needs: the original asset and capable ML hardware.

It would be nice if the app could optionally run supported inference locally and send the result back to the server.

A possible flow could be:

1. The app finds a new local asset.
2. The asset is uploaded normally.
3. The app runs supported ML jobs locally when appropriate.
4. The generated detections / embeddings are submitted to the server.
5. Server-side ML remains the fallback for anything that was not processed by a client.

This wouldn't need to replace the existing machine-learning service. A hybrid approach would probably make more sense.

For example, client-side processing could initially be limited to assets that are already local to the phone and only run while charging, on Wi-Fi, or while the app is active.

On iOS this could use Core ML and the Neural Engine. On Android, it could use the available on-device ML runtimes and hardware acceleration (e.g. LiteRT with GPU/NPU acceleration).

The main benefit would be that users with a low-power Immich server don't need to run another Docker instance or install a separate ML service on another computer just to make use of hardware they already have in their phone.

Related, but different from #25726: that request is about accelerating a remote machine-learning service on macOS using Core ML. This request is about running inference directly in the mobile client for assets that are already available on the device.

Platform

• Server
• Web
• Mobile

[bo0tzz]

#24369

[chris3713]

Just to clarify the distinction I was trying to make in #32005:

I don't really mean moving the existing backend processing stack (ffmpeg, Python ML service, etc.) to the mobile client.

I'm specifically thinking about ML inference for assets that are already local to the device. That could use the platform-native ML runtime (e.g. Core ML / Neural Engine on iOS or LiteRT with hardware acceleration on Android) and only send the resulting detections / embeddings back to Immich.

I agree that moving general processing jobs to the client would be a major rewrite and could hurt UX. But ML seems like a slightly different case because modern phones have dedicated hardware specifically intended for this workload.

It could also be completely optional and opportunistic.

So the idea isn't really "run the backend on the phone", but rather "allow the phone to act as an additional inference backend using its native ML stack".

[mmomjian]

The asset being local isn’t the hard part of this proposal.