Development Guide

This document covers the architecture, build process, and patch management workflow for jetson-ffmpeg — a library that enables hardware-accelerated video encoding/decoding on NVIDIA Jetson platforms via FFmpeg's codec interface.

The goal is to ensure that anyone can continue development, update patches for new FFmpeg versions, and maintain the project independently.

Dev Container

A containerized development environment is available for working directly on Jetson hardware via VS Code Remote SSH. The container uses NVIDIA's L4T JetPack base image with GPU access through the NVIDIA container runtime.

See DEVCONTAINER.md for setup instructions covering host prerequisites, VS Code configuration, and JetPack version selection.

Repository Structure

jetson-ffmpeg/
├── CMakeLists.txt              # Build system for libnvmpi (shared + static)
├── nvmpi.pc.in                 # pkg-config template
├── scripts/                    # Operator scripts (run from any directory)
│   ├── build.sh                # Build/install libnvmpi (auto-detects stubs vs Jetson)
│   └── ffpatch.sh              # Patches a vanilla FFmpeg source tree with nvmpi support
├── include/
│   ├── nvmpi.h                 # Public C API (encoder/decoder create/put/get/close)
│   ├── NVMPI_bufPool.hpp       # Thread-safe buffer pool (template)
│   ├── nvmpi_frame_buffer.hpp      # DMA frame buffer wrapper
│   └── nvUtils2NvBuf.h         # Compatibility shim: NvUtils API ↔ legacy nvbuf_utils
├── src/
│   ├── nvmpi_dec_api.cpp       # Decoder public API (create/put/get/flush/close)
│   ├── nvmpi_dec_capture.cpp   # Decoder capture thread loop, resolution-change
│   ├── nvmpi_dec_planes.cpp    # Decoder V4L2 CAPTURE-plane alloc/teardown
│   ├── nvmpi_dec_internal.h    # Decoder context struct, macros, forward decls
│   ├── nvmpi_enc_api.cpp       # Encoder public API (create/put/get/close)
│   ├── nvmpi_enc_output.cpp    # Encoder DQ-thread callback, output-plane DMA setup
│   ├── nvmpi_enc_internal.h    # Encoder context struct, macros, forward decls
│   └── nvmpi_frame_buffer.cpp      # DMA buffer alloc/destroy
├── stubs/                      # Stub .so files for cross-compilation (aarch64)
├── ffmpeg/                     # FFmpeg integration layer
│   ├── dev/                    # Patch development environment
│   │   ├── common/             # FFmpeg codec source files (shared across versions)
│   │   │   └── libavcodec/
│   │   │       ├── nvmpi_dec.c # FFmpeg decoder codec (AVCodec/FFCodec wrapper)
│   │   │       └── nvmpi_enc.c # FFmpeg encoder codec (AVCodec/FFCodec wrapper)
│   │   ├── 6.0/                # Version-specific FFmpeg overlay files
│   │   │   ├── configure       # Patched configure (adds --enable-nvmpi)
│   │   │   ├── libavcodec/Makefile     # Patched Makefile (adds nvmpi_dec.o / nvmpi_enc.o)
│   │   │   └── libavcodec/allcodecs.c  # Patched codec registration
│   │   ├── .gitignore          # Ignores cloned ffmpeg dirs (ffmpeg4.2/, etc.)
│   │   ├── copy_files.sh       # Copies overlay files into cloned FFmpeg trees
│   │   ├── update_patch.sh     # Clones FFmpeg, patches, generates .patch files
│   │   └── try_build.sh        # Runs update_patch.sh then builds all versions
│   └── patches/                # Generated patch files (applied by users)
│       └── ffmpeg6.0_nvmpi.patch  # (one per supported version: 6.0–8.1)
├── test/
│   ├── hw-all.sh               # Auto-discovery runner for all hw-*.sh suites
│   ├── hw-*.sh                  # Per-feature hardware test suites
│   ├── smoke-all.sh             # Full cross-version matrix test
│   └── gen-samples.sh           # Input-sample generators (sourced by suites)
└── docs/                       # Documentation

Architecture Overview

The project has two distinct layers:

┌─────────────────────────────────────────────────────┐
│                    FFmpeg Process                     │
│                                                       │
│  ┌─────────────────────────────────────────────────┐ │
│  │ FFmpeg Integration Layer (patched into FFmpeg)   │ │
│  │   nvmpi_enc.c  ←→  AVCodec / FFCodec interface  │ │
│  │   nvmpi_dec.c  ←→  AVCodec / FFCodec interface  │ │
│  └──────────────────────┬──────────────────────────┘ │
│                         │ calls libnvmpi C API        │
│  ┌──────────────────────▼──────────────────────────┐ │
│  │ libnvmpi (shared library, installed on system)  │ │
│  │   nvmpi_dec_*.cpp (V4L2 decoder, modular)       │ │
│  │   nvmpi_enc_*.cpp (V4L2 encoder, modular)       │ │
│  │   nvmpi_frame_buffer  (DMA buffer management)       │ │
│  └──────────────────────┬──────────────────────────┘ │
│                         │ V4L2 / NvBuffer API         │
│  ┌──────────────────────▼──────────────────────────┐ │
│  │ NVIDIA Jetson Multimedia API (system libraries)  │ │
│  │   libnvv4l2, libnvbufsurface, libnvjpeg, etc.   │ │
│  └─────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────┘

Layer 1 — libnvmpi (src/, include/, CMakeLists.txt): A standalone shared library that wraps NVIDIA's V4L2-based multimedia API into a simple C interface. This is what gets installed on the system (libnvmpi.so).

Layer 2 — FFmpeg integration (ffmpeg/dev/common/, patches): Source files that implement FFmpeg's codec interface (AVCodec/FFCodec) by calling libnvmpi. These get patched into FFmpeg source trees and compiled as part of FFmpeg.

The nvmpi Library (libnvmpi)

Public API (`include/nvmpi.h`)

The library exposes a pure C API:

Decoder:

nvmpi_create_decoder(nvDecParam*) — Create decoder context
nvmpi_decoder_put_packet(ctx, nvPacket*) — Feed compressed packet
nvmpi_decoder_get_frame(ctx, nvFrame*, wait) — Retrieve decoded frame
nvmpi_decoder_flush(ctx) — Reset pipeline for seek / stream restart
nvmpi_decoder_close(ctx) — Destroy decoder

Encoder:

nvmpi_create_encoder(nvEncParam*) — Create encoder context
nvmpi_encoder_put_frame(ctx, nvFrame*) — Feed raw frame
nvmpi_encoder_get_packet(ctx, nvPacket**) — Retrieve encoded packet
nvmpi_encoder_dqEmptyPacket(ctx, nvPacket**) — Get empty packet buffer from pool
nvmpi_encoder_qEmptyPacket(ctx, nvPacket*) — Return empty packet buffer to pool
nvmpi_encoder_force_idr(ctx) — Force IDR frame on next input
nvmpi_encoder_set_bitrate(ctx, bitrate) — Change bitrate mid-stream
nvmpi_encoder_flush(ctx) — Mid-stream encoder reset (STREAMOFF/drain/STREAMON)
nvmpi_encoder_close(ctx) — Destroy encoder

JPEG Decoder:

nvmpi_create_jpeg_decoder() — Create JPEG decoder (NvJPEGDecoder, not V4L2)
nvmpi_jpeg_decoder_put_packet(ctx, packet) — Decode one JPEG frame
nvmpi_jpeg_decoder_get_frame(ctx, frame, wait) — Retrieve decoded frame
nvmpi_jpeg_decoder_close(ctx) — Destroy JPEG decoder

JPEG Encoder:

nvmpi_create_jpeg_encoder(quality) — Create JPEG encoder (quality 1-100)
nvmpi_jpeg_encoder_put_frame(ctx, frame) — Encode one YUV420 frame
nvmpi_jpeg_encoder_get_packet(ctx, packet) — Retrieve JPEG packet
nvmpi_jpeg_encoder_close(ctx) — Destroy JPEG encoder

Supported Codecs

Operation	H.264	H.265/HEVC	VP8	VP9	MPEG2	MPEG4	MJPEG
Decode	Yes	Yes	Yes	Yes	Yes	Yes	Yes
Encode	Yes	Yes	No	No	No	No	Yes

MJPEG Decoder Architecture

The MJPEG decoder (mjpeg_nvmpi) uses a fundamentally different path from V4L2 video decoders. Instead of V4L2 M2M, it uses NvJPEGDecoder::decodeToFd() — synchronous per-frame decode through the Tegra NVJPG engine.

Key differences from V4L2 decoders:

No capture thread — decode is synchronous in put_packet
No resolution events — dimensions detected from each decoded frame
Separate context — nvmpictx_jpeg (not nvmpictx) to avoid ODR collisions
VIC transform required — decodeToFd produces block-linear output; VIC converts to pitch-linear YUV420

Implementation files: src/nvmpi_jpegdec.cpp, src/nvmpi_jpegdec_internal.h

Progressive JPEG (SOF2) is detected via marker scan and rejected before the hardware call — the NVJPG engine supports baseline DCT (SOF0) only.

MJPEG Encoder Architecture

The MJPEG encoder (mjpeg_nvmpi) mirrors the decoder's non-V4L2 approach, using NvJPEGEncoder::encodeFromFd() — synchronous per-frame encode through the Tegra NVJPG engine.

Key differences from V4L2 encoders:

No DQ thread — encode is synchronous in put_frame
No rate control — quality-based (libjpeg-style 1-100)
Separate context — nvmpictx_jpeg_enc (not nvmpictx)
DMA buffer required — input frame is copied to a DMA buffer, encoded via hardware

Implementation files: src/nvmpi_enc_jpeg.cpp, src/nvmpi_enc_jpeg_internal.h

On modules without NVJPG encode capability (e.g. Orin Nano), createJPEGEncoder() returns NULL — the FFmpeg wrapper reports a clear error suggesting -c:v mjpeg software fallback.

Internal Components

NVMPI_bufPool<T> (include/NVMPI_bufPool.hpp): Thread-safe producer/consumer buffer pool using mutex-guarded queues. Used for both frame buffers (decoder) and packet buffers (encoder).
nvmpi_frame_buffer (include/nvmpi_frame_buffer.hpp, src/nvmpi_frame_buffer.cpp): Wraps DMA buffer allocation/destruction. Abstracts the difference between NvUtils API (JetPack 5+) and legacy nvbuf_utils API.
nvUtils2NvBuf.h (include/nvUtils2NvBuf.h): Compile-time compatibility layer. When WITH_NVUTILS is defined, maps legacy NvBuffer* names to NvBufSurf* equivalents. This is what enables support across JetPack versions without #ifdef in every function.

Modular File Structure

Source files follow nvmpi_{codec}_{concern}.cpp — see ARCHITECTURE.md for the full convention, include structure, and encoder migration path.

Decoder files:

File	Concern
`src/nvmpi_dec_api.cpp`	Public API (`create`, `put_packet`, `get_frame`, `flush`, `close`)
`src/nvmpi_dec_capture.cpp`	Capture thread loop, resolution-change handler
`src/nvmpi_dec_planes.cpp`	V4L2 CAPTURE-plane alloc/teardown, color format selection
`src/nvmpi_dec_internal.h`	Context struct, macros, forward declarations
`src/nvmpi_jpegdec.cpp`	JPEG decoder (NvJPEGDecoder-backed, synchronous)
`src/nvmpi_jpegdec_internal.h`	JPEG decoder context struct

Encoder files:

File	Concern
`src/nvmpi_enc_api.cpp`	Public API (`create`, `put_frame`, `get_packet`, `close`, packet pool ops)
`src/nvmpi_enc_output.cpp`	DQ-thread callback, output-plane DMA buffer setup
`src/nvmpi_enc_internal.h`	Context struct, macros (`TEST_ERROR`, `MAX_BUFFERS`, `OUTPLANE_MEMTYPE_*`)
`src/nvmpi_enc_jpeg.cpp`	JPEG encoder API (NvJPEGEncoder-backed, synchronous)
`src/nvmpi_enc_jpeg_internal.h`	JPEG encoder context struct

When adding a new feature:

Identify which concern it touches (API? capture/output loop? planes?).
Edit only that file. If it doesn't fit any existing concern, create a new nvmpi_{codec}_{concern}.cpp and add it to CMakeLists.txt.
Never put new public API signatures outside include/nvmpi.h.

NvUtils vs nvbuf_utils

The codebase supports two NVIDIA buffer management APIs:

Feature	nvbuf_utils (legacy)	NvUtils / NvBufSurface (JetPack 5+)
Header	`nvbuf_utils.h`	`nvbufsurface.h`, `NvBufSurface.h`
Detection	No `nvbufsurface.h`	`nvbufsurface.h` exists
CMake flag	(default)	`-DWITH_NVUTILS` (auto-detected)

CMakeLists.txt auto-detects this: if nvbufsurface.h exists in the Jetson Multimedia API headers, it enables WITH_NVUTILS and links the corresponding libraries.

Building libnvmpi

On a Jetson device (native build):

The quickest path is the helper script (auto-detects real Jetson libs vs stubs):

./scripts/build.sh --install     # alias in the dev container: build --install

Equivalent raw CMake build:

mkdir build && cd build
cmake ..
make -j$(nproc)
sudo make install

With custom paths:

cmake .. \
  -DJETSON_MULTIMEDIA_API_DIR=/path/to/jetson_multimedia_api \
  -DJETSON_MULTIMEDIA_LIB_DIR=/path/to/tegra/libs \
  -DCUDA_INCLUDE_DIR=/usr/local/cuda/include \
  -DCUDA_LIB_DIR=/usr/local/cuda/lib64

Cross-compilation with stubs:

cmake .. -DWITH_STUBS=ON

This links against stub libraries in stubs/ instead of real NVIDIA libraries. Useful for CI/CD or development on non-Jetson hosts. The stub libraries are minimal aarch64 ELF shared objects that satisfy the linker without actual implementations.

Build outputs

libnvmpi.so.1.0.0 (shared library)
libnvmpi.a (static library)
nvmpi.pc (pkg-config file)
Installed header: nvmpi.h

FFmpeg Integration Layer

How FFmpeg finds nvmpi

FFmpeg's build system needs three modifications to recognize nvmpi as a hardware acceleration library:

configure — Register nvmpi in HWACCEL_LIBRARY_LIST, add --enable-nvmpi flag, define codec dependencies (e.g., h264_nvmpi_encoder_deps="nvmpi")
libavcodec/Makefile — Add object file rules (e.g., OBJS-$(CONFIG_H264_NVMPI_ENCODER) += nvmpi_enc.o)
libavcodec/allcodecs.c — Register codec symbols (e.g., extern FFCodec ff_h264_nvmpi_decoder;)

FFmpeg codec source files

Located in ffmpeg/dev/common/libavcodec/:

nvmpi_enc.c (602 lines): Implements FFmpeg encoder codecs for H.264 and HEVC. Handles profile/level mapping, rate control, packet pool management. Contains version-aware macros for FFmpeg API changes across versions.
nvmpi_dec.c (258 lines): Implements FFmpeg decoder codecs for H.264, HEVC, VP8, VP9, MPEG2, MPEG4. Handles frame allocation, DRM prime output, resize options.

As of v3.0.0 (FFmpeg 6.0+ only), the wrapper code uses FFCodec and receive_packet unconditionally. The only remaining version guard is the FF_PROFILE_* → AV_PROFILE_* rename at libavcodec 62.11 (FFmpeg 8.0+):

// FF_PROFILE renamed to AV_PROFILE (FFmpeg 8.0+)
#if (LIBAVCODEC_VERSION_MAJOR >= 62 && LIBAVCODEC_VERSION_MINOR >= 11)
#define FF_PROFILE_H264_INTRA AV_PROFILE_H264_INTRA
#endif

Version-specific overlay files

Each supported FFmpeg version has its own directory under ffmpeg/dev/:

ffmpeg/dev/6.0/    → configure, libavcodec/Makefile, libavcodec/allcodecs.c

As of v3.0.0, all supported versions (6.0+) use the FFCodec interface, so the overlays differ only in minor configure/Makefile text. FFmpeg 4.2 and 4.4 overlays were retired in v3.0.0 — they used the legacy AVCodec / encode2 interface.

Patch System

Overview

The patch system converts the overlay files + common codec sources into standard git diff patches that end users apply to vanilla FFmpeg source trees.

How it works

scripts/ffpatch.sh: The runtime patching script. Takes a path to an FFmpeg source directory and:

Detects libavcodec version from headers (version.h or version_major.h)
Creates backup copies of configure, Makefile, allcodecs.c
Uses sed to insert nvmpi entries into each file:
- Adds --enable-nvmpi to configure help
- Adds nvmpi to HWACCEL_LIBRARY_LIST
- Adds dependency declarations for each codec
- Adds object file rules to Makefile
- Adds extern declarations to allcodecs.c
Copies nvmpi_dec.c and nvmpi_enc.c into libavcodec/
Each insertion is idempotent (checks grep before sed)

ffmpeg/dev/update_patch.sh: The patch generation script:

Shallow-clones FFmpeg release branches (4.2, 4.4, 6.0)
Runs scripts/ffpatch.sh on each clone
Copies overlay files from ffmpeg/dev/{version}/ and ffmpeg/dev/common/
Generates patches via git add -A && git diff --cached
Writes patches to ffmpeg/patches/

ffmpeg/dev/copy_files.sh: Copies overlay + common files into cloned trees (used during development iteration).

ffmpeg/dev/try_build.sh: Runs update_patch.sh then attempts ./configure --enable-nvmpi && make for each version. Build validation.

Patch file content

Each patch in ffmpeg/patches/ is a complete git diff that modifies:

configure (nvmpi registration)
libavcodec/Makefile (build rules)
libavcodec/allcodecs.c (codec symbol registration)
libavcodec/nvmpi_dec.c (new file — decoder implementation)
libavcodec/nvmpi_enc.c (new file — encoder implementation)
libavcodec/nvmpi_enc_jpeg.c (new file — MJPEG encoder implementation)

Development Workflow

Updating FFmpeg Patches

When you need to modify the FFmpeg codec implementation or fix a bug in the integration layer:

1. Edit the source files:

For changes common to all FFmpeg versions:

# Edit the shared codec files
vim ffmpeg/dev/common/libavcodec/nvmpi_enc.c
vim ffmpeg/dev/common/libavcodec/nvmpi_dec.c

For version-specific changes (configure, Makefile, allcodecs.c):

vim ffmpeg/dev/4.2/configure
vim ffmpeg/dev/4.4/configure
vim ffmpeg/dev/6.0/configure
# (same for Makefile and allcodecs.c)

2. Regenerate patches:

cd ffmpeg/dev
./update_patch.sh

This will:

Clone fresh FFmpeg source trees (shallow, into gitignored dirs)
Apply scripts/ffpatch.sh to each
Copy your edited overlay/common files
Generate new patch files in ffmpeg/patches/

3. Verify the patches build:

cd ffmpeg/dev
./try_build.sh

4. Commit both the source changes and regenerated patches.

Adding Support for a New FFmpeg Version

Example: adding FFmpeg 7.0 support.

Step 1 — Clone and inspect the new version:

cd ffmpeg/dev
git clone git://source.ffmpeg.org/ffmpeg.git -b release/7.0 --depth=1 ffmpeg7.0

Step 2 — Create the version overlay directory:

mkdir -p 7.0/libavcodec

Step 3 — Copy the closest existing version as starting point:

cp 6.0/configure 7.0/configure
cp 6.0/libavcodec/Makefile 7.0/libavcodec/Makefile
cp 6.0/libavcodec/allcodecs.c 7.0/libavcodec/allcodecs.c

Step 4 — Diff the new FFmpeg version against the overlay files:

Compare the vanilla FFmpeg 7.0 files against your overlay to find what changed:

diff ffmpeg7.0/configure 7.0/configure
diff ffmpeg7.0/libavcodec/Makefile 7.0/libavcodec/Makefile
diff ffmpeg7.0/libavcodec/allcodecs.c 7.0/libavcodec/allcodecs.c

Step 5 — Update overlay files to match new FFmpeg version:

The overlay files are essentially copies of the original FFmpeg files with nvmpi entries added. You need to:

Start from the vanilla FFmpeg 7.0 files
Add the same nvmpi entries that exist in the 6.0 overlay
Check if the codec interface changed (e.g., AVCodec → FFCodec transition happened at version 60)

Key things to check:

Has HWACCEL_LIBRARY_LIST moved or been renamed in configure?
Has the allcodecs.c extern format changed?
Has the Makefile section where codec objects are listed been restructured?
Does libavcodec/version.h or version_major.h report a new major version?

Step 6 — Check if common codec files need updates:

Look at ffmpeg/dev/common/libavcodec/nvmpi_enc.c and nvmpi_dec.c for version checks. If FFmpeg 7.0 introduces API changes (renamed functions, new required fields), add appropriate #if guards:

#if LIBAVCODEC_VERSION_MAJOR >= XX
    // new API
#else
    // old API
#endif

Step 7 — Update scripts/ffpatch.sh:

The runtime patching script (scripts/ffpatch.sh) uses sed with anchors based on existing FFmpeg text. If FFmpeg 7.0 changed the text around insertion points (e.g., the line --disable-videotoolbox that anchors --enable-nvmpi), update the corresponding sed command.

Check each sed anchor still exists in the new version:

grep -n 'disable-videotoolbox' ffmpeg7.0/configure
grep -n 'h264_nvenc_encoder_deps' ffmpeg7.0/configure
grep -n 'CONFIG_H264_CUVID_DECODER' ffmpeg7.0/libavcodec/Makefile

Step 8 — Register the new version with the dev scripts:

update_patch.sh, copy_files.sh, and try_build.sh each iterate over a single VERSIONS list. Add the new version there:

# In ffmpeg/dev/update_patch.sh, copy_files.sh, and try_build.sh:
VERSIONS="6.0 6.1 7.0 7.1 8.0 8.1"

The scripts handle cloning, patching (via scripts/ffpatch.sh), copying overlays, and writing ffmpeg/patches/ffmpeg7.0_nvmpi.patch automatically — no per-version code to add.

Step 9 — (covered by Step 8)

try_build.sh builds every version in the same VERSIONS list, so no extra change is needed.

Step 10 — Test the full pipeline:

cd ffmpeg/dev
./update_patch.sh   # generates patches
./try_build.sh      # builds all versions

Testing Patches Locally

To test a patch against a specific FFmpeg version without the dev scripts:

# Clone FFmpeg
git clone git://source.ffmpeg.org/ffmpeg.git -b release/6.0 --depth=1 ffmpeg-test
cd ffmpeg-test

# Apply patch
git apply /path/to/jetson-ffmpeg/ffmpeg/patches/ffmpeg6.0_nvmpi.patch

# Build (on Jetson)
./configure --enable-nvmpi
make -j$(nproc)

# Verify codecs registered
./ffmpeg -hide_banner -encoders 2>/dev/null | grep nvmpi
./ffmpeg -hide_banner -decoders 2>/dev/null | grep nvmpi

Hardware Test Suites

Hardware testing uses test/hw-all.sh, which auto-discovers and runs every per-feature suite (test/hw-*.sh), hw-smoke first. Each suite is self-contained and tests one specific behavior or regression guard.

JETSON_VARIANT=orin-nano ./test/hw-all.sh
HW_SUITES="decoder-chunk encoder-gop" ./test/hw-all.sh   # subset

The full cross-version matrix (build libnvmpi + patch + build + hw-all for every supported FFmpeg version) is run via test/smoke-all.sh. See test/README.md for the authoritative suite list, conventions, and how to add a new suite.

Cross-Building with Stubs

The stubs/ directory contains minimal aarch64 ELF shared objects that satisfy the linker:

Stub	Real library
`libnvv4l2.so`	NVIDIA V4L2 implementation
`libnvjpeg.so`	NVIDIA JPEG library
`libnvbufsurface.so`	NvBufSurface API (JetPack 5+)
`libnvbufsurftransform.so`	Surface transform (JetPack 5+)
`libnvbuf_utils.so`	Legacy buffer utils
`libv4l2.so.0`	Symlink to `libnvv4l2.so`

Use stubs for:

CI/CD pipelines that verify the library compiles
Cross-compilation on x86 hosts
IDE indexing / code completion

Stubs cannot be used for runtime testing — the actual NVIDIA hardware and drivers are required.

Codec Registration Reference

Each nvmpi codec requires entries in three FFmpeg files. This table shows what each codec needs:

configure entries

Codec	Dependency line	Bitstream filter select
`h264_nvmpi` encoder	`h264_nvmpi_encoder_deps="nvmpi"`	—
`h264_nvmpi` decoder	`h264_nvmpi_decoder_deps="nvmpi"`	`h264_nvmpi_decoder_select="h264_mp4toannexb_bsf"`
`hevc_nvmpi` encoder	`hevc_nvmpi_encoder_deps="nvmpi"`	—
`hevc_nvmpi` decoder	`hevc_nvmpi_decoder_deps="nvmpi"`	`hevc_nvmpi_decoder_select="hevc_mp4toannexb_bsf"`
`mpeg2_nvmpi` decoder	`mpeg2_nvmpi_decoder_deps="nvmpi"`	—
`mpeg4_nvmpi` decoder	`mpeg4_nvmpi_decoder_deps="nvmpi"`	—
`vp8_nvmpi` decoder	`vp8_nvmpi_decoder_deps="nvmpi"`	—
`vp9_nvmpi` decoder	`vp9_nvmpi_decoder_deps="nvmpi"`	—

Makefile entries

Each codec adds one object file rule:

OBJS-$(CONFIG_H264_NVMPI_ENCODER)      += nvmpi_enc.o
OBJS-$(CONFIG_H264_NVMPI_DECODER)      += nvmpi_dec.o
# ... etc for each codec

allcodecs.c entries

All supported versions (6.0+) use FFCodec:

extern const FFCodec ff_h264_nvmpi_decoder;
extern const FFCodec ff_h264_nvmpi_encoder;

Note: v2.x also supported FFmpeg < 60 (4.2, 4.4) which used extern AVCodec (non-const, non-FFCodec). This path was retired in v3.0.0.

Version Compatibility Notes

FFmpeg API breakpoints handled in the codebase

Change	Version	Where handled
`FF_PROFILE_` → `AV_PROFILE_`	libavcodec 62.11+	`nvmpi_enc.c`
`nvbuf_utils` → `NvBufSurface` API	JetPack 5+	`nvUtils2NvBuf.h`, `WITH_NVUTILS` define

Retired guards (v2.x only): ff_alloc_packet2 → ff_get_encode_buffer (libavcodec 60), AVCodec → FFCodec (libavcodec 60), old encode2 API → receive_packet (NVMPI_FF_NEW_API). All removed in v3.0.0 — these are now the only path.

Wrapper code paths by FFmpeg version

As of v3.0.0, all supported versions use FFCodec and receive_packet unconditionally. The only remaining compile-time fork is the AV_PROFILE rename at libavcodec 62.11.

FFmpeg	libavcodec	`AV_PROFILE`	Path
6.0	60.3	✗	A
6.1	60.31	✗	A
7.0	61.3	✗	A
7.1	61.19	✗	A
8.0	62.11	✓	B
8.1	62.xx	✓	B

CI builds and hw-tests all six versions. The path column documents why 6.0–7.1 compile wrapper-identical code — useful when triaging a failure (a path-B-only regression points at the AV_PROFILE guard).

Retired paths (v2.x): FFmpeg 4.2 (path A-legacy: no FFCodec, no receive_packet) and 4.4 (path B-legacy: receive_packet but no FFCodec) were removed in v3.0.0.

Adding support for future FFmpeg API changes

When FFmpeg deprecates or renames an API:

Check the new version's libavcodec/version.h or version_major.h for the major/minor version
Add a preprocessor guard in the common codec files (ffmpeg/dev/common/libavcodec/)
Test with both old and new versions to ensure the guards work
Regenerate patches

Troubleshooting

`scripts/ffpatch.sh` fails with "Patching ... failed!"

The sed commands in scripts/ffpatch.sh use anchor strings from FFmpeg source files. If FFmpeg reorganized the file, the anchor may no longer exist. Check which function failed (configure, Makefile, or allcodecs.c) and find the new location of the anchor text in the new FFmpeg version.

Build fails with undefined reference to `nvmpi_*`

libnvmpi must be installed before building FFmpeg. Ensure pkg-config --libs nvmpi returns -lnvmpi.

Codec not found at runtime

Verify FFmpeg was configured with --enable-nvmpi: ffmpeg -buildconf | grep nvmpi
Verify libnvmpi is in the library path: ldconfig -p | grep nvmpi
Verify on actual Jetson hardware (not stubs)

NvBufSurface errors on JetPack 5+

Ensure the build detected WITH_NVUTILS. Check CMake output for "Using NvUtils API." If it says "Using NvBufUtils API." on JetPack 5+, verify the nvbufsurface.h header path.

Encoder runs slowly without max performance mode

The encoder defaults to max_perf = true. If you see slow encoding, check that the Jetson power mode is set appropriately: sudo nvpmodel -m 0 (MAXN).

🏠 Home · 📦 Repository · 🐞 Issues · 🏷 Releases · 📋 README · 📝 CHANGELOG

Documentation lives in this wiki. To change a doc, edit the relevant wiki page (clone jetson-ffmpeg.wiki.git) — the docs/ folder in the repo has been retired.

jetson-ffmpeg Wiki

📦 Usage & Runtime

❓ FAQ

🛠 Development

⚙️ CI / Infrastructure

🔬 Project / Fork Analysis

Uh oh!

Development Guide

Development Guide

Table of Contents

Dev Container

Repository Structure

Architecture Overview

The nvmpi Library (libnvmpi)

Public API (include/nvmpi.h)

Supported Codecs

MJPEG Decoder Architecture

MJPEG Encoder Architecture

Internal Components

Modular File Structure

NvUtils vs nvbuf_utils

Building libnvmpi

Build outputs

FFmpeg Integration Layer

How FFmpeg finds nvmpi

FFmpeg codec source files

Version-specific overlay files

Patch System

Overview

How it works

Patch file content

Development Workflow

Updating FFmpeg Patches

Adding Support for a New FFmpeg Version

Step 9 — (covered by Step 8)

Testing Patches Locally

Hardware Test Suites

Cross-Building with Stubs

Codec Registration Reference

configure entries

Makefile entries

allcodecs.c entries

Version Compatibility Notes

FFmpeg API breakpoints handled in the codebase

Wrapper code paths by FFmpeg version

Adding support for future FFmpeg API changes

Troubleshooting

scripts/ffpatch.sh fails with "Patching ... failed!"

Build fails with undefined reference to nvmpi_*

Codec not found at runtime

NvBufSurface errors on JetPack 5+

Encoder runs slowly without max performance mode

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

jetson-ffmpeg Wiki

Clone this wiki locally

Public API (`include/nvmpi.h`)

`scripts/ffpatch.sh` fails with "Patching ... failed!"

Build fails with undefined reference to `nvmpi_*`