Numpy2 support

[brontide]

Overview

I have been messing around with an upgrade to numpy2 which permits the upgrade of many key packages from their held-back status. Yes, AI was used in the process by which many of these patches were developed or documented. I am currently running this as a custom build for my personal use. I'm leaving this here as a snapshot of what works and what does not, thankfully I don't think anything is a blocker presuming MemryX wheels can be corrected.

Frigate's dependency tree was pinned to numpy 1.26.x. Several upstream packages have moved on to require or prefer numpy 2, making the pin increasingly difficult to maintain. This patchset u pgrades the stack to numpy >= 2 and addresses the breaking changes that follow.

What Changed

numpy >= 2 and loosened pins

requirements-wheels.txt moves from numpy == 1.26.* to numpy >= 2. Many other pins were loosened from exact (==) to minimum (>=) to resolve transitive dependency conflicts that arose from the numpy 2 floor. Key packages: pandas, scipy, opencv, aiohttp, fastapi, cryptography, tensorflow, onnxruntime (bumped to 1.24), shapely, degirum.

numpy is pre-installed in the Dockerfile (pip3 install "numpy>=2,<3") before the bulk wheel install to prevent other packages from pulling in a conflicting numpy 1.x pin during resolution. Given some pinballing of the version between the various stages I may re-pin this to 2.2.6 which is what it ends up as.

ai-edge-litert replaces tflite_runtime

tflite_runtime has no numpy 2 compatible build. Google's replacement is ai-edge-litert, which provides the same Interpreter and load_delegate API under a new package name. All seven import sites use a fallback chain:

try:
    from tflite_runtime.interpreter import Interpreter, load_delegate
except ModuleNotFoundError:
    from ai_edge_litert.interpreter import Interpreter, load_delegate

Realistically, this may not be needed as tflite_runtime should likely be replaced with ai_edge_litert when this work is complete. This preserves compatibility if someone still has tflite_runtime installed. Confirmed working with Coral EdgeTPU.

norfair tracker — numpy 2 fork

The norfair object tracking library is unmaintained and uses deprecated numpy APIs that fail under numpy 2 (e.g., np.float_, np.int_ aliases removed in numpy 2). A patched wheel is pul led from brontide/norfair-numpy2:

norfair[video] @ https://github.com/brontide/norfair-numpy2/releases/download/v2.3.0/norfair-2.3.0-py3-none-any.whl

This is a minimal patch that I did not write, I ressurected it from a deleted PR — fixes the removed aliases and nothing else. Long-term, norfair should be replaced or properly forked.

HailoRT wheel install disabled

HailoRT bumped to 4.23.0 (runtime libraries still installed), but the Python wheel install is commented out. The hailort 4.23 wheel has a hard numpy < 2 pin baked into its metadata, which forces pip to downgrade numpy during install and breaks everything else. The runtime .so libraries work fine — only the Python bindings are blocked until Hailo ships a numpy 2 compatible wheel. Upstream HailoRT requirements do not pin numpy so I expect that this will be easily fixed.

MemryX freeze install disabled

MemryX's pip3 install -r /opt/mx_accl_frigate/freeze is commented out. Their freeze file pins numpy to 1.x, which conflicts with the numpy 2 floor. Same pattern as HailoRT — the compiled runtime still loads, but the Python dependency install is skipped to avoid crushing the numpy 2 pin.

pandas 3.0 datetime resolution change ( PR will be submitted )

pandas 3.0 (pulled in by the >= 2.2 floor) changed DatetimeIndex internal storage from datetime64[ns] (nanoseconds) to datetime64[us] (microseconds). The motion activity endpoint converted timestamps using:

df.index = df.index.astype(int) // (10**9)  # assumed nanoseconds

With microsecond storage this produces timestamps ~1000x too small, silently breaking the motion indicator bars on the review screen. Fixed with resolution-independent arithmetic:

# Pre-computed constants for resolution-independent datetime-to-epoch conversion
# (pandas 3.0+ stores datetime64 as microseconds, not nanoseconds)
_EPOCH = pd.Timestamp("1970-01-01")
_ONE_SECOND = pd.Timedelta("1s")

...

df.index = (df.index - _EPOCH) // _ONE_SECOND

Current Status

Testing was done on x86_64 with OpenVINO on iGPU and Intel A380.

Component	Status	Notes
Core (numpy/pandas/scipy/opencv)	Working	All loosened pins resolve cleanly
Coral EdgeTPU (ai-edge-litert)	Working	Confirmed with live detection
OpenVINO	Working	Tested end-to-end
Classification	Working	Tested end-to-end
norfair tracker	Working	Via numpy2 fork wheel
Motion indicators	Working	Fixed pandas 3.0 timestamp conversion
HailoRT	Not working	Python wheel pins numpy < 2; runtime libs installed, Python bindings disabled
MemryX	Not working	Freeze file pins numpy 1.x; runtime installed, Python deps disabled
ONNX Runtime	Untested	Bumped to 1.24, should work but not verified with inference
TensorRT	Untested	No changes made, may need attention

[NickM-27]

Thanks for creating an updated summary. We're not in any hurry to do this, it likely will make sense to take another look for 0.19 when we'll probably look at upgrading to Trixie and need to move to litert for the coral

[brontide]

I tried to do a Trixie port but numpy2 is really required and there are a number of legacy Intel dependencies which complicated the attempt. You may want to steal the norfair patches since it seems to be a deal project.