Add FFmpeg N-API bindings research and extract implementation tasks

research/README.md

@@ -24,11 +24,14 @@ Each runtime has unique characteristics and optimal implementation paths:
 
 - **[nodejs-linux-napi-ffmpeg.md](./nodejs-linux-napi-ffmpeg.md)** - Detailed technical research for implementing WebCodecs in Node.js on Linux using N-API and FFmpeg. Covers architecture, threading, memory management, codec mapping, and distribution.
 
+- **[ffmpeg-napi.md](./ffmpeg-napi.md)** - **NEW**: Research on existing FFmpeg N-API bindings (node-av, @mmomtchev/ffmpeg, etc.) that can serve as a foundation for WebCodecs implementation, avoiding the need to build raw N-API bindings from scratch.
+
 ## Quick Summary
 
 | Runtime | Approach | Key Advantage | Primary Challenge |
 |---------|----------|---------------|-------------------|
 | **Node.js** | N-API + FFmpeg | Mature addon ecosystem | Threading/memory complexity |
+| **Node.js (alt)** | Existing N-API bindings (node-av) | Skip raw N-API work | Adapter layer needed |
 | **Deno** | Rust + codec libs | Type-safe, clean architecture | Codec library availability |
 | **Bun** | WebKit leverage | WebCodecs may already exist | Media backend wiring |
 
@@ -42,7 +45,8 @@ For detailed implementation tasks extracted from research:
 
 1. Start with **[webcodecs-overview.md](./webcodecs-overview.md)** for context on the problem space
 2. Review **[options.md](./options.md)** for technical implementation strategies
-3. Dive into the runtime-specific document for your target platform
+3. **NEW**: Check **[ffmpeg-napi.md](./ffmpeg-napi.md)** for existing N-API bindings that can accelerate development
+4. Dive into the runtime-specific document for your target platform
 
 ## External Resources
 

research/ffmpeg-napi.md

@@ -0,0 +1,135 @@
+# Existing FFmpeg N-API Bindings Research
+
+> **Parent document**: [Node.js Implementation Tasks](./nodejs-implementation.md)  
+> **Related**: [Node.js Linux N-API + FFmpeg Research](./nodejs-linux-napi-ffmpeg.md) | [Implementation Options](./options.md)
+
+## Summary
+
+| Aspect | Finding |
+|--------|---------|
+| **Outcome** | Yes, there are existing N-API FFmpeg bindings you can stand on. |
+| **Obstacles** | None of them are WebCodecs-shaped; you still need an adapter layer. |
+| **Plan** | Reuse an FFmpeg N-API library as the "engine" and build WebCodecs on top. |
+
+---
+
+## Key Finding
+
+There are already serious FFmpeg bindings built on Node-API, so you don't have to start from raw C FFmpeg + N-API. This avoids the largest yak: raw FFmpeg + Node-API integration.
+
+---
+
+## 1. Strong Candidates
+
+### node-av (SeydX)
+
+- **Repository**: [seydx/node-av](https://github.com/seydx/node-av)
+- Native FFmpeg v8 bindings for Node.js.
+- Uses a compiled addon (Node-API) with:
+  - **Low-level API**: `FormatContext`, `CodecContext`, `Frame`, `Packet`, etc., very close to libav* C API.
+  - **High-level API**: `Demuxer`, `Decoder`, `Encoder`, `Muxer`, plus a "pipeline" API that already models decode/encode chains.
+- Supports hardware acceleration (CUDA, VAAPI, auto-detect) and exposes `HardwareContext`.
+- Ships prebuilt binaries for all major platforms; on Linux you can just `npm install node-av`.
+- Fully typed for TypeScript, modern async/await patterns.
+
+**This is probably the best "drop-in engine" for a Linux-only WebCodecs implementation right now.**
+
+---
+
+### @mmomtchev/ffmpeg / node-ffmpeg (avcpp + nobind17)
+
+- **Repository**: [mmomtchev/ffmpeg](https://github.com/mmomtchev/ffmpeg) (library name often referred to as node-ffmpeg).
+- Wraps avcpp (a C++ wrapper around FFmpeg's C API) and exposes it to Node via nobind17, which itself sits on Node-API / node-addon-api.
+- Provides:
+  - A fairly complete mapping of FFmpeg (demuxers, decoders, encoders, filters).
+  - A Node.js Streams API: `Demuxer`, `AudioDecoder`, `VideoDecoder`, `Filter`, `VideoTransform`, `AudioTransform`, `VideoEncoder`, `AudioEncoder`, `Muxer`, etc.
+- Designed to allow safe-ish async use and multi-threading, with some explicit caveats around misconfigured streams.
+
+**Good fit if you like the streams model and don't mind the extra avcpp layer.**
+
+---
+
+### libav (Astronaut Labs)
+
+- **Package**: `libav` / **Repository**: [astronautlabs/libav](https://github.com/AstronautLabs/libav)
+- Node.js bindings to FFmpeg's libav* libraries, adapted to be "comfortable and efficient" in TypeScript.
+- Uses Node-API as well, but:
+  - Marked **"Pre-Alpha Quality; incomplete, broken, heavy development"**.
+  - Requires system FFmpeg 5.x and dev headers installed on Linux.
+
+**Probably better as inspiration/reference than a solid base for a WebCodecs project right now.**
+
+---
+
+### ff-helper (napi-rs-based helper)
+
+- **Package**: `ff-helper`
+- A napi-rs binding wrapping a few FFmpeg features, like getting video info and generating screenshots.
+- Explicitly **not** a full FFmpeg binding; just a helper.
+
+**Useful as a tiny example of napi-rs + FFmpeg, but not enough for WebCodecs.**
+
+---
+
+## 2. How This Helps the WebCodecs Plan (Linux-only)
+
+You can absolutely avoid writing raw C++/Node-API around FFmpeg from scratch:
+
+1. **Pick one of the full bindings as your "FFmpegService"**:
+   - If you want modern TS + pipelines + prebuilt binaries → **node-av**.
+   - If you want streams + avcpp and don't mind extra C++ layer → **node-ffmpeg**.
+
+2. **Implement WebCodecs classes** (`VideoDecoder`, `VideoEncoder`, `VideoFrame`, etc.) **in TypeScript**, and internally:
+   - Use node-av's `Decoder` / `Encoder` or node-ffmpeg's `VideoDecoder` / `VideoEncoder` to do the actual work.
+   - Map WebCodecs `configure()` → underlying codec/stream setup.
+   - Map `decode()`/`encode()` → feed frames/packets into the N-API binding.
+   - Use their existing thread/memory/resource management instead of building your own.
+
+### What You'd Still Need to Build
+
+You'd still need to:
+- Reproduce **WebCodecs semantics** (state machine, reset, close, `isConfigSupported`, error types).
+- Possibly wrap/normalize timestamps, formats, and pixel layouts to match spec.
+
+…but you **avoid the largest yak: raw FFmpeg + Node-API integration**.
+
+---
+
+## 3. Concrete Recommendation
+
+For a Linux-first WebCodecs-in-Node prototype:
+
+1. **Start with node-av as the backend**:
+   - Use its high-level `Decoder`/`Encoder` for quick progress; drop to low-level `CodecContext`/`Frame` when you need more control.
+
+2. **Wrap it in TS classes that exactly match the WebCodecs IDL**.
+
+3. **Only reach for your own N-API code** if you discover a semantic gap you cannot bridge cleanly via node-av.
+
+---
+
+## 4. Next Steps
+
+See extracted tasks for actionable follow-ups:
+
+- [Evaluate FFmpeg N-API Bindings](../tasks/evaluate-ffmpeg-napi-bindings.md) — Compare node-av vs node-ffmpeg for WebCodecs use
+- [VideoDecoder Shim on node-av](../tasks/videodecoder-shim-node-av.md) — Sketch a VideoDecoder shim that sits on top of node-av.Decoder
+
+---
+
+## 5. Library Comparison Matrix
+
+| Library | API Style | Prebuilt Binaries | HW Accel | TypeScript | Maturity | Best For |
+|---------|-----------|-------------------|----------|------------|----------|----------|
+| **node-av** | High + Low level | ✅ All platforms | ✅ CUDA/VAAPI | ✅ Full | Production | Quick WebCodecs prototype |
+| **@mmomtchev/ffmpeg** | Streams | ✅ | ⚠️ Limited | ✅ | Stable | Stream-based pipelines |
+| **libav (Astronaut)** | Low-level | ❌ | ❌ | ✅ | Pre-alpha | Reference/learning |
+| **ff-helper** | Utilities | ✅ | ❌ | ✅ | Minimal | Quick video info |
+
+---
+
+## Related Documents
+
+- [Node.js Linux N-API + FFmpeg Research](./nodejs-linux-napi-ffmpeg.md) — Original from-scratch N-API design
+- [Node.js Implementation Tasks](./nodejs-implementation.md) — Higher-level task breakdown
+- [Implementation Options](./options.md) — Comparison of implementation approaches

research/nodejs-implementation.md

@@ -5,6 +5,8 @@ This document outlines the research tasks needed to implement WebCodecs in Node.
 > **Parent document**: [WebCodecs Overview](./webcodecs-overview.md)
 > 
 > **Detailed research**: [Node.js Linux N-API + FFmpeg Research](./nodejs-linux-napi-ffmpeg.md) — Deep-dive into architecture, threading, memory management, and FFmpeg integration for Linux.
+>
+> ⭐ **NEW — Faster path**: [FFmpeg N-API Bindings Research](./ffmpeg-napi.md) — Existing production-ready FFmpeg N-API bindings that can serve as the backend, avoiding raw N-API development.
 > 
 > **Implementation tasks**: [tasks/](../tasks/) — Extracted actionable tasks with YAML frontmatter for tracking.
 
@@ -14,6 +16,15 @@ This document outlines the research tasks needed to implement WebCodecs in Node.
 
 Node.js has the most mature ecosystem for native addons and the strongest community demand (evidenced by the $10k challenge). The implementation will likely involve C++ bindings via N-API with FFmpeg or OS-level codec APIs.
 
+### ⭐ New Finding: Existing N-API Bindings Available
+
+Recent research discovered that production-ready FFmpeg N-API bindings already exist (see [FFmpeg N-API Bindings Research](./ffmpeg-napi.md)):
+
+- **node-av**: Full-featured FFmpeg bindings with prebuilts and HW acceleration
+- **@mmomtchev/ffmpeg**: Streams-based FFmpeg API
+
+This means we can **skip raw N-API development** and instead build a WebCodecs-compliant adapter layer on top of these existing libraries. See [Evaluate FFmpeg N-API Bindings](../tasks/evaluate-ffmpeg-napi-bindings.md) and [VideoDecoder Shim on node-av](../tasks/videodecoder-shim-node-av.md) for actionable tasks.
+
 ---
 
 ## Research Tasks

research/nodejs-linux-napi-ffmpeg.md

@@ -2,6 +2,8 @@
 
 > **Parent document**: [WebCodecs Overview](./webcodecs-overview.md)  
 > **Related**: [Node.js Implementation Tasks](./nodejs-implementation.md) | [Implementation Options](./options.md)
+>
+> ⚠️ **Important Update**: See [FFmpeg N-API Bindings Research](./ffmpeg-napi.md) for an **alternative faster path** using existing production-ready FFmpeg N-API bindings (node-av, @mmomtchev/ffmpeg). This document describes the from-scratch approach, which may still be useful if the existing bindings have gaps, but the shim-based approach is now recommended.
 
 **Scope**
 

research/options.md

@@ -2,17 +2,19 @@
 
 This document explores various approaches to implementing the WebCodecs API in Node.js, based on research of existing implementations and related projects.
 
-> **See also**: [Node.js Linux N-API + FFmpeg Research](./nodejs-linux-napi-ffmpeg.md) for detailed architecture and implementation guidance for the FFmpeg approach on Linux.
+> **See also**: 
+> - [Node.js Linux N-API + FFmpeg Research](./nodejs-linux-napi-ffmpeg.md) for detailed architecture and implementation guidance for the FFmpeg approach on Linux.
+> - **NEW**: [FFmpeg N-API Bindings Research](./ffmpeg-napi.md) for existing N-API bindings that can accelerate development.
 
 ## Overview
 
 The WebCodecs API provides low-level access to media encoders and decoders. Implementing it in Node.js requires bridging JavaScript to native codec libraries.
 
 ## Implementation Options
 
-### Option 1: FFmpeg via N-API
+### Option 1: FFmpeg via N-API (From Scratch)
 
-**Description**: Create Node.js native bindings to FFmpeg's libavcodec, libavformat, and related libraries.
+**Description**: Create Node.js native bindings to FFmpeg's libavcodec, libavformat, and related libraries from scratch.
 
 > 📖 **Detailed research available**: [Node.js Linux N-API + FFmpeg Research](./nodejs-linux-napi-ffmpeg.md) covers architecture, threading, memory management, codec mapping, licensing, and distribution for this approach.
 
@@ -43,6 +45,44 @@ The WebCodecs API provides low-level access to media encoders and decoders. Impl
 
 ---
 
+### Option 1b: Existing FFmpeg N-API Bindings + WebCodecs Shim ⭐ NEW
+
+**Description**: Use an existing, production-ready FFmpeg N-API binding (such as node-av) and build a thin WebCodecs-compliant adapter layer on top.
+
+> 📖 **Detailed research available**: [FFmpeg N-API Bindings Research](./ffmpeg-napi.md) covers available libraries, comparison, and recommended approach.
+
+**Approach**:
+1. Select an existing FFmpeg N-API library (recommended: node-av)
+2. Build WebCodecs-compliant TypeScript classes (`VideoDecoder`, `VideoEncoder`, etc.)
+3. Map WebCodecs API calls to the underlying library's API
+4. Implement WebCodecs state machine and error semantics in the adapter layer
+
+**Pros**:
+- **Avoids the largest yak**: raw FFmpeg + N-API integration already done
+- Prebuilt binaries available for all major platforms
+- Hardware acceleration support (node-av supports CUDA, VAAPI)
+- Full TypeScript support with modern async patterns
+- Significant time savings (estimated 2-3 weeks)
+- Battle-tested FFmpeg integration
+
+**Cons**:
+- Dependent on third-party library maintenance
+- May have semantic gaps requiring workarounds
+- Less control over low-level behavior
+- Need to map between two different APIs
+
+**Effort Estimate**: **Low-Medium** (significantly lower than from-scratch)
+
+**Key Libraries**:
+- **node-av (recommended)**: [seydx/node-av](https://github.com/seydx/node-av) — Best combination of features, prebuilds, and HW accel
+- **@mmomtchev/ffmpeg**: [mmomtchev/ffmpeg](https://github.com/mmomtchev/ffmpeg) — Streams-based API, good if you prefer that model
+
+**Recommended Next Steps**:
+- [Evaluate FFmpeg N-API Bindings](../tasks/evaluate-ffmpeg-napi-bindings.md)
+- [VideoDecoder Shim on node-av](../tasks/videodecoder-shim-node-av.md)
+
+---
+
 ### Option 2: WebKit/Chromium Port
 
 **Description**: Extract the WebCodecs implementation from WebKit or Chromium and adapt it for standalone use.
@@ -188,7 +228,8 @@ The WebCodecs API provides low-level access to media encoders and decoders. Impl
 
 | Option | Effort | Performance | Portability | Maintenance |
 |--------|--------|-------------|-------------|-------------|
-| FFmpeg N-API | High | Excellent | Medium | Medium |
+| **FFmpeg N-API Bindings + Shim** ⭐ | **Low-Medium** | **Excellent** | **Good** | **Low** |
+| FFmpeg N-API (from scratch) | High | Excellent | Medium | Medium |
 | Browser Port | Very High | Excellent | Low | High |
 | Bun-Style | Medium | Good | Low | Medium |
 | WASM | Low | Medium | Excellent | Low |
@@ -199,12 +240,38 @@ The WebCodecs API provides low-level access to media encoders and decoders. Impl
 
 For the $10k challenge timeline (1 month), we recommend:
 
-### Phase 1: Quick Start with WASM (Week 1-2)
+### ⭐ NEW: Fastest Path — Existing N-API Bindings + WebCodecs Shim
+
+Based on the discovery of existing production-ready FFmpeg N-API bindings (see [FFmpeg N-API Bindings Research](./ffmpeg-napi.md)), the fastest path is now:
+
+**Week 1: Foundation**
+1. Evaluate node-av and @mmomtchev/ffmpeg ([Evaluate FFmpeg N-API Bindings](../tasks/evaluate-ffmpeg-napi-bindings.md))
+2. Select the best candidate (likely node-av)
+3. Build VideoDecoder shim ([VideoDecoder Shim on node-av](../tasks/videodecoder-shim-node-av.md))
+
+**Week 2: Core Classes**
+1. Complete VideoDecoder with state machine, error handling
+2. Build VideoEncoder shim
+3. Implement VideoFrame and EncodedVideoChunk wrappers
+
+**Week 3-4: Polish + Testing**
+1. Add AudioDecoder, AudioEncoder shims
+2. Run WPT test subset
+3. Implement isConfigSupported() accurately
+4. Documentation and examples
+
+This approach can save **2-3 weeks** compared to building raw N-API bindings from scratch.
+
+### Alternative: WASM + Native Hybrid (Original Recommendation)
+
+If the N-API binding approach hits blockers, fall back to:
+
+#### Phase 1: Quick Start with WASM (Week 1-2)
 1. Use libavjs-webcodecs-polyfill as a starting point
 2. Adapt for Node.js environment
 3. Get tests passing with basic functionality
 
-### Phase 2: Optimize with Native Bindings (Week 3-4)
+#### Phase 2: Optimize with Native Bindings (Week 3-4)
 1. Add FFmpeg N-API bindings for performance-critical codecs (VP8, H.264)
 2. Use WASM as fallback
 3. Implement hardware acceleration detection
@@ -221,14 +288,17 @@ For the $10k challenge timeline (1 month), we recommend:
 2. **Mediabunny** - High-level video editing on WebCodecs
 3. **Effect-TS** - Functional patterns for async operations
 4. **libavjs-webcodecs-polyfill** - Pure JS WebCodecs polyfill
+5. **node-av** ⭐ NEW - Production-ready FFmpeg N-API bindings with HW accel
+6. **@mmomtchev/ffmpeg** ⭐ NEW - FFmpeg bindings with Node.js Streams API
 
 ## Conclusion
 
-The most practical approach for rapid development is to start with WebAssembly-based codecs (Option 4) and progressively add native optimizations (Option 1). This provides:
+**Updated recommendation**: The most practical approach for rapid development is now to leverage existing FFmpeg N-API bindings (Option 1b) and build a WebCodecs shim layer on top:
 
-1. **Immediate functionality** - Get tests passing quickly
-2. **Broad compatibility** - Works on any platform
-3. **Performance path** - Clear upgrade route to native
-4. **Lower risk** - Incremental development
+1. **Fastest time to working code** — Skip raw N-API development
+2. **Production-ready FFmpeg integration** — Already battle-tested
+3. **Hardware acceleration included** — node-av supports CUDA, VAAPI
+4. **Excellent performance** — Native FFmpeg, not WASM
+5. **Lower risk** — Smaller codebase to maintain
 
-The WASM approach using libavjs-webcodecs-polyfill can likely get basic functionality working within days, providing a foundation for further optimization.
+If the N-API binding approach encounters blockers, the WASM approach (Option 4) remains a viable fallback that can get basic functionality working within days.

tasks/README.md

@@ -24,10 +24,14 @@ research: ../research/relevant-doc.md
 
 | Task | Priority | Effort | Status |
 |------|----------|--------|--------|
+| [Evaluate FFmpeg N-API Bindings](./evaluate-ffmpeg-napi-bindings.md) | critical | medium | todo |
+| [VideoDecoder Shim on node-av](./videodecoder-shim-node-av.md) | critical | medium | todo |
 | [N-API PoC Addon](./napi-poc-addon.md) | critical | large | todo |
 | [FFmpeg Static Build](./ffmpeg-static-build.md) | critical | medium | todo |
 | [Codec String Parser](./codec-string-parser.md) | high | small | todo |
 
+> **Note**: The first two tasks (Evaluate FFmpeg N-API Bindings and VideoDecoder Shim) represent an alternative faster path using existing N-API bindings. See [FFmpeg N-API Bindings Research](../research/ffmpeg-napi.md) for details.
+
 ### Week 2: Threading + Memory
 
 | Task | Priority | Effort | Status |
@@ -45,6 +49,7 @@ research: ../research/relevant-doc.md
 
 ## Related Research
 
-- [Node.js Linux N-API + FFmpeg Research](../research/nodejs-linux-napi-ffmpeg.md) — Primary research document
+- [FFmpeg N-API Bindings Research](../research/ffmpeg-napi.md) — **NEW**: Existing N-API bindings that can accelerate development
+- [Node.js Linux N-API + FFmpeg Research](../research/nodejs-linux-napi-ffmpeg.md) — Primary research document (from-scratch approach)
 - [Node.js Implementation Overview](../research/nodejs-implementation.md) — Higher-level task breakdown
 - [Implementation Options](../research/options.md) — Comparison of implementation approaches