You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
obs-avip: an NDI plugin in Rust in the obs-rust org, and a path to native AV-over-IP support
#4439
OBS has no NDI support of its own. Users install two things from third parties: the DistroAV plugin and the NDI runtime. The plugin is built for upstream OBS. Nobody tests it against obs-rust, and when it fails, the user cannot tell whether the cause is OBS, the plugin or the network.
We use NDI every day, in and out of OBS. We measured three problems that a user notices:
Slow resume after a network cut. After a cut of 10 s, the picture came back after 17 to 20 s. After a cut of 45 s, it took 65 s. The time grows with the length of the cut (NDI camera feeds freeze and don't come back up after network interruptions #3745). The delay is in the NDI SDK's own reconnect, not in libobs: a plain SDK receiver without OBS shows the same times. DistroAV inherits it.
High latency by default. The default latency mode is "normal": 97 to 205 ms, different after each connect. "Lowest" gives 70 to 75 ms, but a user must know where to find it.
A prototype NDI plugin in Rust, about 5,300 lines, 57 unit tests:
ndi-core: no OBS types. The NDI function table, a mock of the NDI API (CI has no NDI runtime), the reconnect state machine, the source finder, tally, frame and audio conversion, and DistroAV's settings keys with their defaults as a test table.
ndi-obs: a thin OBS shell. The source, the output, the main and preview outputs, and a Tools menu item that opens the output settings in OBS's own properties window (no Qt code in the plugin).
The NDI runtime is loaded at run time. No SDK files are in the code, and the runtime is installed by the user, as for DistroAV.
What it does today:
Drop-in for DistroAV. It uses DistroAV's ids (ndi_source, ndi_output) and settings keys, so a scene collection and a profile move over with no change. It registers its ids in obs_module_post_load, and only when no other module registered them first. With DistroAV loaded, it writes one log line and stays out of the way.
Resume in 0.7 s after any cut. When no frame comes for 1.5 s, it connects again every 2 s. Measured: 0.7 s after a cut of 5, 10 or 45 s (DistroAV: 2.2, 17 to 20 and 65 s).
Tested on Linux in daily use (swapped for DistroAV with no change to the profile) and on macOS with the obs-rust nightly. Not tested on Windows.
What it does not do:
It is not faster. With the same settings, CPU and latency are the same as DistroAV. Most of the CPU is in the NDI runtime's own threads. Our output copies each frame once before it sends it, because the SDK wants the buffer to stay valid until the next send. That costs about 1 % of a core more on 1080p.
No PTZ, no NDI filters, no alpha output, no 16-bit (HDR) formats yet.
Proposal
Move the prototype into the obs-rust org as obs-avip ("audio and video over IP"), as a plugin repository next to obs-browser and obs-websocket, and ship it with obs-rust.
Why in the org and not as one more third-party plugin:
It is tested against obs-rust. The core runs its tests in CI with the mock. The shell builds against the obs-rust headers, so an ABI change shows in CI, not on a user's machine.
One install less. Users get NDI with obs-rust. The runtime is still a separate install in the first version (see the open questions).
It is Rust, with the core apart from OBS. That is the shape the port wants. The core can move into the tree later without a rewrite.
Its settings are OBS properties. Everything in obs_properties also works through obs-websocket (GetInputPropertiesListPropertyItems gives the NDI source list, SetInputSettings sets it). Remote control comes with no extra code.
The name is not tied to NDI. NDI is a closed protocol with one vendor's SDK. Open standards for the same job exist (IPMX, SMPTE ST 2110). "AVIP" leaves room for them, and it keeps a trademark out of the plugin's name. The UI says NDI only to describe what a source is.
Path to native support
Step by step, each step only when the one before has users:
Plugin in the org (this proposal). DistroAV's ids, so users can switch both ways.
Native module. The core moves to rust/obs-avip, the shell to plugins/obs-avip, under the naming and layout rules of the tree. The output settings become a page in OBS Settings, so no dialog in the plugin.
It steps aside for a third-party plugin. A native module loads before third-party plugins, and the first registration of an id wins (obs-module.c.h). So native NDI registers its ids in obs_module_post_load, only when no plugin took them, and then it does not load the NDI runtime at all. A user who wants DistroAV, or a plugin built on the NDI Advanced SDK, keeps it. A setting "Built-in NDI: auto / off" covers plugins with other ids.
More outputs. What vMix users ask for: scaling per output, a frame-rate divisor (a 1080p25 monitor feed next to a 4K main), audio track per output, more outputs with their own scene or source, and alpha for graphics into a vision mixer.
Open standards. IPMX or ST 2110 behind the same source and output types, with no closed SDK.
Open questions
Licence. obs-rust is GPL-2. The plugin loads a closed runtime that the user installs, as DistroAV does today. Shipping the runtime inside obs-rust is a separate question: the NDI SDK licence has its own terms for bundled products. This needs a maintainer decision, and probably a question to NDI, before any bundling. Until then: the user installs the runtime.
Ids. DistroAV's ids make the switch free, but the two cannot run at the same time (the plugin steps aside). Own ids would need a converter. We suggest DistroAV's ids.
Repository or tree. An org repository first (like obs-browser), or plugins/obs-avip in this repository from the start?
Testing. CI has the mock only. The real runtime and the network tests (cut, tally, receivers) run on hardware. Is a manual hardware check per release enough, or do we want a self-hosted runner with a test sender?
Upstream. The reconnect finding helps DistroAV users too. We plan to report it to DistroAV with the measurement, independent of this proposal.
The prototype is in a private repository now. We can push it to the org for review when there is interest.
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
Uh oh!
There was an error while loading. Please reload this page.
Problem
OBS has no NDI support of its own. Users install two things from third parties: the DistroAV plugin and the NDI runtime. The plugin is built for upstream OBS. Nobody tests it against obs-rust, and when it fails, the user cannot tell whether the cause is OBS, the plugin or the network.
We use NDI every day, in and out of OBS. We measured three problems that a user notices:
What we built
A prototype NDI plugin in Rust, about 5,300 lines, 57 unit tests:
ndi-core: no OBS types. The NDI function table, a mock of the NDI API (CI has no NDI runtime), the reconnect state machine, the source finder, tally, frame and audio conversion, and DistroAV's settings keys with their defaults as a test table.ndi-obs: a thin OBS shell. The source, the output, the main and preview outputs, and a Tools menu item that opens the output settings in OBS's own properties window (no Qt code in the plugin).What it does today:
ndi_source,ndi_output) and settings keys, so a scene collection and a profile move over with no change. It registers its ids inobs_module_post_load, and only when no other module registered them first. With DistroAV loaded, it writes one log line and stays out of the way.What it does not do:
Proposal
Move the prototype into the obs-rust org as obs-avip ("audio and video over IP"), as a plugin repository next to obs-browser and obs-websocket, and ship it with obs-rust.
Why in the org and not as one more third-party plugin:
obs_propertiesalso works through obs-websocket (GetInputPropertiesListPropertyItemsgives the NDI source list,SetInputSettingssets it). Remote control comes with no extra code.Path to native support
Step by step, each step only when the one before has users:
rust/obs-avip, the shell toplugins/obs-avip, under the naming and layout rules of the tree. The output settings become a page in OBS Settings, so no dialog in the plugin.obs-module.c.h). So native NDI registers its ids inobs_module_post_load, only when no plugin took them, and then it does not load the NDI runtime at all. A user who wants DistroAV, or a plugin built on the NDI Advanced SDK, keeps it. A setting "Built-in NDI: auto / off" covers plugins with other ids.Open questions
plugins/obs-avipin this repository from the start?The prototype is in a private repository now. We can push it to the org for review when there is interest.
All reactions