Skip to content

feat(encoder): add preserveFramePictType to allow forced keyframes - #315

Merged
seydx merged 1 commit into
seydx:mainfrom
Pradhumn115:fix/forced-keyframe-picttype
Jul 28, 2026
Merged

feat(encoder): add preserveFramePictType to allow forced keyframes#315
seydx merged 1 commit into
seydx:mainfrom
Pradhumn115:fix/forced-keyframe-picttype

Conversation

@Pradhumn115

Copy link
Copy Markdown
Contributor

Fixes #314.

Problem

The high-level Encoder clears frame.pictType on every frame in prepareFrameForEncoding:

// Clear pict_type - encoder will determine frame types based on its own settings
// Input stream's frame type hints are irrelevant when re-encoding
frame.pictType = AV_PICTURE_TYPE_NONE;

That makes it impossible to force a keyframe on demand. Setting frame.pictType = AV_PICTURE_TYPE_I (with forced-idr on encoders that support it) is the standard FFmpeg way to request an IDR for a specific frame, but it is erased before avcodec_send_frame, so the encoder never emits the forced keyframe.

For live streaming this matters: a decoder produces nothing before its first keyframe, and on a lossy or late-joining connection the client needs to request a fresh IDR (packet-loss recovery, a new viewer, a resized encode). Through the high-level API there was no way to honour that.

Change

Clearing pictType is still correct for transcoding, where decoded input frames carry frame-type hints that should not drive the output GOP. So this is an opt-in option rather than a default change:

if (!this.options.preserveFramePictType) {
  frame.pictType = AV_PICTURE_TYPE_NONE;
}
  • New EncoderOptions.preserveFramePictType?: boolean (default false).
  • When false (default), behaviour is unchanged.
  • When true, the frame's pictType is passed through, so AV_PICTURE_TYPE_I forces an IDR.

Verification

Tested against libx264 with a long GOP (so the only keyframes are the initial IDR and any forced one), encoding 10 frames and requesting a forced keyframe on frame 5:

forced keyframe at frame 5
default (no flag) not present — unchanged behaviour
preserveFramePictType: true present

Both cases are covered by new tests in test/encoder.test.ts. Confirmed non-vacuous: reverting the guard makes the "forces a keyframe" test fail while the default-behaviour test still passes.

Happy to adjust — e.g. a dedicated encoder.requestKeyframe()-style API instead of a flag — if you'd prefer a different shape.

The high-level Encoder clears `frame.pictType` on every frame in
`prepareFrameForEncoding`, so a caller cannot force a keyframe on demand:
setting `frame.pictType = AV_PICTURE_TYPE_I` (with `forced-idr`) — the standard
FFmpeg way to request an IDR — is erased before `avcodec_send_frame`, and the
encoder never emits the forced keyframe.

For live streaming that is not cosmetic: a decoder produces nothing before its
first keyframe, and on a lossy or late-joining connection the client needs to
be able to request a fresh IDR (loss recovery, a new viewer, a resized encode).
Through the high-level API there was no way to honour that request.

Clearing pictType is still correct for transcoding, where decoded input frames
carry frame-type hints that should not drive the output GOP. So this adds an
opt-in `preserveFramePictType` option rather than changing the default: when
set, the frame's pictType is passed through untouched. Existing callers are
unaffected.

Verified against libx264 with a long GOP: by default a forced keyframe request
on a mid-stream frame produces no keyframe (only the initial IDR); with
`preserveFramePictType: true` the forced IDR appears. Both cases are covered by
tests.

Closes seydx#314
Pradhumn115 added a commit to Pradhumn115/beamdesk that referenced this pull request Jul 28, 2026
Found a real bug in node-av while building the H.264 capture — the high-level
Encoder clears frame.pictType, making forced keyframes impossible — and
contributed the fix upstream: an opt-in preserveFramePictType option, verified
against libx264 with tests.

- issue: seydx/node-av#314
- PR:    seydx/node-av#315

Replaces the pre-submission draft with a record of what was filed, and links it
to the existing explanation in agent/src/capture/h264.ts of why we use the
low-level sendFrame() path.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011D5fBGzCVCbegUMeeS91pF
@seydx
seydx merged commit 1243426 into seydx:main Jul 28, 2026
@seydx

seydx commented Jul 28, 2026

Copy link
Copy Markdown
Owner

Thanks!

@Pradhumn115

Copy link
Copy Markdown
Contributor Author

Thanks a lot for the quick review and merge! 🙏

node-av has been great to build a live H.264 streamer on — the low-level sendFrame() path is what let me work around this cleanly in the meantime, and it's nice to have the high-level API cover the case now too. Looking forward to switching over to preserveFramePictType once it lands in a release.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

High-level Encoder makes forced keyframes impossible: prepareFrameForEncoding unconditionally clears pict_type

2 participants