Beyond the Vendor

Beyond the vendor stack

Audience: streamer developers. This page lists what OpenIMP and open-tx-isp do differently from, or in addition to, the vendor stack, and how a streamer uses it. Only items that are device-tested are listed as supported; opt-in or pending items are marked.

Integration rule: everything is reachable through the standard IMP API with the vendor signatures. Where behaviour differs, it is in what the call returns or what the hardware now does. Vendor builds of the streamer must stay unchanged: wrap anything extra in a compile-time switch.

The USE_OPENIMP macro

timps has a compile-time switch USE_OPENIMP (Makefile USE_OPENIMP ?= 0, -DUSE_OPENIMP in CFLAGS). thingino sets it from the package selection:

USE_OPENIMP=$(if $(BR2_PACKAGE_OPENIMP),1,0)

Under it, capability headers (isp_caps.h, enc_caps.h) enable the extra per-SoC controls. There is deliberately no runtime capability query; a query call was prototyped and withdrawn. If you write your own streamer: copy the pattern, put every call that the vendor headers lack behind #if defined(USE_OPENIMP), and declare the OpenIMP prototype yourself. OpenIMP's own imp_isp.h declares all ISP setters on every platform; vendor headers lack some (for example SetDPC_Strength, SetDefog_Strength, SetDRC_Strength, SetBcshHue, SetBacklightComp, SetFrontCrop on T10/T20/T30).

Encoder and rate control

• Capped modes are real: CappedVBR/CappedQuality run a closed-loop regulator with the vendor PSNR cap (42 dB) on T31; plain VBR is closed loop. Vendor behaviour was silent CBR.
• T31 uses the ported Allegro rate-control core for VBR, CappedVBR, CappedQuality and CBR (OPENIMP_T31_RC_CORE=legacy restores the old controller).
• Deviations (less than vendor): T31 CBR writes no filler NAL, so a static scene stays below the target bitrate; CappedQuality behaves like CappedVBR. Do not rely on CBR output being exactly at the target.
• T10/T20/T21 accept mode, bitrate, QP range, GOP and fps. Some knobs (QP steps, staticTime, changePos, qualityLvl, I/P delta, bias) are accepted but ignored on paths that do not run the vendor controller; do not expose them as live controls without checking the effective rate-control log line.
• A log line per channel states the effective rate control. Out-of-range QP or fps is clamped with a warning naming the field.
• CreateChn(PT_H265) returns -1 on SoCs without HEVC hardware (T10/T20/T21/T23); the vendor returns success and creates a channel that never produces frames. Check the return value and fall back to H.264.
• Real HEVC on T31.
• After 3 failed pictures the T20/T21 encoder is re-created; after 2 fruitless re-creates the channel stops instead of stalling for 20 s per picture. Expect GetStream to fail and restart the channel from the streamer.
• Encoder channel statistics use the vendor layout with a real average bitrate and can be polled for telemetry.

ISP tuning

• Scene mode and colour effects act on hardware (T21, T20 colour effects); getters return what was set.
• Sinter/Temper denoise strength takes effect on T10, T20, T21 where the vendor firmware ignores it. Value 128 equals the IQ table, so the default picture is identical to the vendor.
• AWB state is kept across stream restarts (T23); on-demand snapshots no longer show a colour cast.
• Maximum analog gain, maximum digital gain (an extra AE stage on T23), IT max and SetSensorFPS reach the AE. T21 ae_it_max_us acts (the vendor ignores it).
• Unknown tuning control IDs are rejected (T23). Check return codes: an error means the control does not exist.
• T23 front crop rejects windows outside the sensor, smaller than 64x64 or odd-sized with -EINVAL.
• T31 privacy mask: 4 rectangles per channel.
• Not yet exposed by timps and pending USE_OPENIMP enabling: DPC strength, defog/Iridix floor, DRC strength, scene and colour effects.

JPEG / snapshots

• Configured JPEG quality is applied; Helix SoCs adapt quality (-5 on truncation, +5 after 100 clean frames), so the configured value is only the starting point.
• The last JPEG is re-delivered when the encoder is busy or video memory is short. A snapshot can repeat the previous frame under load.
• Hardware JPEG on T31 (timps CPU about 70 % down to 17 % at one snapshot per second).
• JPEG shares the H.264 bitstream area on T23 and uses a 1 MiB bitstream buffer on T20/T10, saving video memory.

IVS / motion

• An IVS channel already in use returns EBUSY. Handle it, do not retry blindly.
• Real frame-difference motion detection on T20/T21/T30 (the vendor reported "no motion" always).
• Motion v2 (on by default since 2026-10-04): per-cell background model, hold-off after day/night or gain jumps, object grouping, bounding boxes and strength. OPENIMP_MOTION_V2=0 (or features = 0 through the API) restores the vendor algorithm; IMP_IVS_MoveOutput is then bit-identical to the vendor. Configure with OPENIMP_MOTION_V2* (see Environment Variables) or per channel through the extension API in include/imp/openimp_ivs_move_ex.h; the box/strength result is read with OpenIMP_IVS_MoveGetResultEx() after IMP_IVS_GetResult():

#include "openimp_ivs_move_ex.h"          /* copy the header, no other dependencies */
#pragma weak OpenIMP_IVS_MoveGetResultEx  /* weak: also links against the vendor libimp */
#pragma weak OpenIMP_IVS_MoveSetConfigEx

if (OpenIMP_IVS_MoveSetConfigEx) {
    OpenIMP_IVS_MoveConfigEx c = { .size = sizeof c,
                                   .version = OPENIMP_IVS_MOVE_EX_VERSION,
                                   .features = 0xf };
    OpenIMP_IVS_MoveSetConfigEx(chn, &c);
}
/* after IMP_IVS_GetResult(): */
OpenIMP_IVS_MoveOutputEx out = { .size = sizeof out, .version = OPENIMP_IVS_MOVE_EX_VERSION };
if (OpenIMP_IVS_MoveGetResultEx && !OpenIMP_IVS_MoveGetResultEx(chn, &out)) { /* objects, suppress reasons */ }

ABI rules: every struct starts with size and version; fields are only appended. Errors: -1 with EINVAL (bad channel/size) or ENOENT (no move interface on the channel). Status: device-tested on two cameras (T20, T23) over one night in shadow and override mode; fewer false alarms than the vendor algorithm (blinking lights, room light, IR switch); car headlights can still form an object.

Audio, OSD, rotation

• Real echo cancellation (WebRTC AECM) on T31: EnableAec returns an error when it cannot run instead of a fake success. Check the return value.
• T31 OSD drawn by the IPU (default on), also in hardware-JPEG snapshots; T20/T21 OSD through an IPU hook (the vendor path never drew).
• Software rotation 90/270 on T31 (the vendor-equivalent tiles) before OSD, IVS and encoder.

Reference buffer sharing

T21/T23: reference frames shared between streams, about 1.5 MB less video memory, on by default (OPENIMP_REF_SHARE=0 disables). Not applicable to T10/T20/T31.

Robustness

• Reload and stop robustness: 10 stop/start and 10 rmmod/insmod cycles with 0 oops on T20/T21/T23/T31 and T41; kill -9 of the streamer recovers.
• rmmod of a sensor module while streaming is refused (the sensor module is pinned while the ISP is open).
• QBUF buffers outside the rmem window are rejected (qbuf_guard).
• Device nodes are opened with O_CLOEXEC, so helper processes do not hold /dev/isp-m0.
• One startup warning when pools plus fixed buffers exceed video memory, see Memory.
• Smaller binaries: libimp about 0.47-0.73 MB depending on the SoC (T41 465,728 B, T23 726 KB) instead of 1.0-1.3 MB.
• Measured against the vendor stack on T20 (night scene, same streamer): streamer CPU 10.5 % instead of 25.2 %, snapshot latency 0.20 s instead of 0.45 s; free memory about 5 MB lower because the open module is larger.

Known deviations (less than vendor)

Real WDR on T23 is missing, T41 flip has no effect (the streamer does not call the live flip setter), T31 CBR writes no filler, CappedQuality equals CappedVBR. The feature matrix keeps the current list.