Skip to content

feat(blocks): add builtin.audioenc, an audio encoder block - #744

Draft
wagenet wants to merge 1 commit into
Eyevinn:mainfrom
wagenet:wagenet/audioenc-block
Draft

feat(blocks): add builtin.audioenc, an audio encoder block#744
wagenet wants to merge 1 commit into
Eyevinn:mainfrom
wagenet:wagenet/audioenc-block

Conversation

@wagenet

@wagenet wagenet commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

What

Adds builtin.audioenc, an audio encoder block, as the counterpart to the existing builtin.videoenc.

The Recorder only accepts pre-encoded audio, and the only audio encoders in the tree were buried inside the SRT and WHIP output blocks. A raw audio pad — a decoded WHIP input feeding a vision mixer, say — therefore could not be recorded at all.

The block builds the chain:

audioconvert -> audioresample -> capsfilter -> encoder -> parser -> capsfilter

Codecs: AAC (default), Opus, MP3, AC-3, with bitrate, sample_rate and channels properties. sample_rate and channels both accept "source" to leave the upstream value untouched.

Details worth review

  • The encoder is the first available candidate for the codec, skipping any factory whose rank is NONE, so GST_PLUGIN_FEATURE_RANK=element:0 still works as the way to take an encoder out of play.
  • Bitrate is applied with set_property_from_str throughout, avoiding the enum-property panic class. lamemp3enc is special-cased: it takes kbps rather than bps, and only honours the value when target=bitrate.
  • Opus accepts only 8/12/16/24/48 kHz. An illegal rate is rejected at build time with a message naming the rate, rather than surfacing later as a not-negotiated error on a running flow.
  • AAC output caps deliberately leave stream-format unpinned, since aacparse converts between raw and adts and pinning either breaks the muxers wanting the other.

Testing

Which commands actually ran, and where:

  • cargo clippy --workspace --all-targets -- -D warningspassed against this exact commit (fe6fedb), dev profile. --all-targets covers the lib-test target, so the block and its test module both typecheck.
  • cargo fmt --all -- --checkpassed against this exact commit.
  • The 12 audioenc unit tests, openapi_test (1 passed) and pipeline_lifecycle_test (3 passed) were run in a separate checkout. That tree differed from this commit only in doc comments and one needless_borrow cleanup — no behavioural difference — so the results carry over, but they were not re-run against fe6fedb itself.

One caveat, stated plainly rather than left implicit: test_build_produces_the_full_chain returns early when no AAC encoder is present, and it was skipped on the macOS machine used for development for that reason. The guard is not green-washing in CI — the Linux jobs install gstreamer1.0-libav, so avenc_aac is available and the test really executes there.

No end-to-end flow was run through the new block on real media. The evidence above is compile-level plus unit tests.

🤖 Generated with Claude Code

The Recorder only accepts pre-encoded audio, and the only audio encoders
in the tree were buried inside the SRT and WHIP output blocks. A raw audio
pad — a decoded WHIP input feeding a vision mixer, say — therefore could not
be recorded at all. There was a videoenc block but no audio counterpart.

This adds one, mirroring videoenc's structure:

  audioconvert -> audioresample -> capsfilter -> encoder -> parser -> capsfilter

AAC (default), Opus, MP3 and AC-3, with bitrate, sample_rate and channels
properties. Sample rate and channels both accept "source" to leave the
upstream value untouched.

Notes on the details:

- The encoder is the first available candidate for the codec, skipping any
  factory whose rank is NONE so GST_PLUGIN_FEATURE_RANK=element:0 still works
  as the way to take an encoder out of play.
- Bitrate is applied with set_property_from_str throughout, avoiding the
  enum-property panic class. lamemp3enc is special-cased: it takes kbps
  rather than bps, and only honours the value when target=bitrate.
- Opus accepts only 8/12/16/24/48 kHz. An illegal rate is rejected at build
  time with a message naming the rate, rather than surfacing later as a
  not-negotiated error on a running flow.
- AAC output caps deliberately leave stream-format unpinned, since aacparse
  converts between raw and adts and pinning either breaks the muxers wanting
  the other.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@wagenet
wagenet force-pushed the wagenet/audioenc-block branch from eb64d05 to fe6fedb Compare September 2, 2026 16:41
@wagenet
wagenet marked this pull request as ready for review September 2, 2026 16:45
@wagenet
wagenet marked this pull request as draft September 2, 2026 16:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant