ffmpeg v5.1.1 seems broken

[SomaticIT]

Hello,

Thank you for your work on this project. It allows a lot of awesome implementations of ffmpeg.

I'm trying to make this work with ffmpeg v5.1.1 (with ffmpeg-next v5.1.1).
However I'm facing many issues with the transcode-x264.rs example.

Here is an output of running the example:

x264 options: {"preset": "medium"}
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '/path/to/video.mp4':
  Metadata:
    major_brand     : mp42
    minor_version   : 0
    compatible_brands: mp42mp41
    creation_time   : 2021-08-25T09:45:49.000000Z
  Duration: 00:00:16.04, start: 0.000000, bitrate: 2549 kb/s
  Stream #0:0[0x1](eng): Video: h264 (Main) (avc1 / 0x31637661), yuv420p(progressive), 1920x1080 [SAR 1:1 DAR 16:9], 2170 kb/s, 29.97 fps, 29.97 tbr, 30k tbn (default)
    Metadata:
      creation_time   : 2021-08-25T09:45:49.000000Z
      handler_name    : ?Mainconcept Video Media Handler
      vendor_id       : [0][0][0][0]
      encoder         : AVC Coding
  Stream #0:1[0x2](eng): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 317 kb/s (default)
    Metadata:
      creation_time   : 2021-08-25T09:45:49.000000Z
      handler_name    : #Mainconcept MP4 Sound Media Handler
      vendor_id       : [0][0][0][0]
thread 'main' panicked at 'called `Option::unwrap()` on a `None` value', def/main.rs:62:52
stack backtrace:
   0: rust_begin_unwind
             at /rustc/a55dd71d5fb0ec5a6a3a9e8c27b2127ba491ce52/library/std/src/panicking.rs:584:5
   1: core::panicking::panic_fmt
             at /rustc/a55dd71d5fb0ec5a6a3a9e8c27b2127ba491ce52/library/core/src/panicking.rs:142:14
   2: core::panicking::panic
             at /rustc/a55dd71d5fb0ec5a6a3a9e8c27b2127ba491ce52/library/core/src/panicking.rs:48:5
   3: core::option::Option<T>::unwrap
             at /rustc/a55dd71d5fb0ec5a6a3a9e8c27b2127ba491ce52/library/core/src/option.rs:775:21
   4: def::Transcoder::new
             at ./def/main.rs:62:31
   5: def::main
             at ./def/main.rs:210:17
   6: core::ops::function::FnOnce::call_once
             at /rustc/a55dd71d5fb0ec5a6a3a9e8c27b2127ba491ce52/library/core/src/ops/function.rs:248:5
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

As you can see, the decoder.frame_rate() is None but it's not the case when using v4.
If I try to enforce frame_rate and time_base with static values, I have this error:

---SAME OUTPUT---
[NULL @ 0x5630fb191800] No codec provided to avcodec_open2()
thread 'main' panicked at 'error opening libx264 encoder with supplied settings: ffmpeg::Error(22: Invalid argument)', def/main.rs:69:14
stack backtrace:
   0: rust_begin_unwind
             at /rustc/a55dd71d5fb0ec5a6a3a9e8c27b2127ba491ce52/library/std/src/panicking.rs:584:5
   1: core::panicking::panic_fmt
             at /rustc/a55dd71d5fb0ec5a6a3a9e8c27b2127ba491ce52/library/core/src/panicking.rs:142:14
   2: core::result::unwrap_failed
             at /rustc/a55dd71d5fb0ec5a6a3a9e8c27b2127ba491ce52/library/core/src/result.rs:1814:5
   3: core::result::Result<T,E>::expect
             at /rustc/a55dd71d5fb0ec5a6a3a9e8c27b2127ba491ce52/library/core/src/result.rs:1064:23
   4: def::Transcoder::new
             at ./def/main.rs:67:9
   5: def::main
             at ./def/main.rs:210:17
   6: core::ops::function::FnOnce::call_once
             at /rustc/a55dd71d5fb0ec5a6a3a9e8c27b2127ba491ce52/library/core/src/ops/function.rs:248:5
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

Any idea?

Edit 1

FYI, if I log the output of decoder.frame_rate() and decoder.time_base(), I get: fr: None, tb: Rational(0/2)

Edit 2:

If I enforce the codec using encoder.open_as_with(), I get another error:

---SAME OUTPUT---
[libx264 @ 0x55de35f0d800] broken ffmpeg default settings detected
[libx264 @ 0x55de35f0d800] use an encoding preset (e.g. -vpre medium)
[libx264 @ 0x55de35f0d800] preset usage: -vpre <speed> -vpre <profile>
[libx264 @ 0x55de35f0d800] speed presets are listed in x264 --help
[libx264 @ 0x55de35f0d800] profile is optional; x264 defaults to high
thread 'main' panicked at 'error opening libx264 encoder with supplied settings: ffmpeg::Error(542398533: Generic error in an external library)', def/main.rs:69:14
stack backtrace:
   0: rust_begin_unwind
             at /rustc/a55dd71d5fb0ec5a6a3a9e8c27b2127ba491ce52/library/std/src/panicking.rs:584:5
   1: core::panicking::panic_fmt
             at /rustc/a55dd71d5fb0ec5a6a3a9e8c27b2127ba491ce52/library/core/src/panicking.rs:142:14
   2: core::result::unwrap_failed
             at /rustc/a55dd71d5fb0ec5a6a3a9e8c27b2127ba491ce52/library/core/src/result.rs:1814:5
   3: core::result::Result<T,E>::expect
             at /rustc/a55dd71d5fb0ec5a6a3a9e8c27b2127ba491ce52/library/core/src/result.rs:1064:23
   4: def::Transcoder::new
             at ./def/main.rs:67:9
   5: def::main
             at ./def/main.rs:210:17
   6: core::ops::function::FnOnce::call_once
             at /rustc/a55dd71d5fb0ec5a6a3a9e8c27b2127ba491ce52/library/core/src/ops/function.rs:248:5
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

Edit 3

Another interesting input:

let codec = encoder::find(codec::Id::H264);
let mut ost = octx.add_stream(codec)?;
println!(
  "{:?} = {:?}",
  codec::context::Context::from_parameters(ost.parameters())?
      .codec()
      .map(|c| c.name().to_string()),
  codec.map(|c| c.name().to_string())
);
//prints:
None = Some("libx264")

It seems that the method codec::context::Context::from_parameters is not including the codec.

[marcusstenbeck]

I see this same problem using ffmpeg 5.1.2 and ffmpeg-next v5.1.1

[gil0mendes]

On the same versions as @marcusstenbeck, I also see the same problem on macOS.

It is my first time using this crate as well as ffmpeg as lib, so I was thought that I was doing something wrong 😅

[gil0mendes]

I did a quick test and seems like if we pass a codec to avcodec_alloc_context3 instead of a null pointer, it works. 🤔

impl Context {
    pub fn new() -> Self {
        unsafe {
            let codec = super::encoder::find_by_name("libx264").unwrap();
            Context {
                ptr: avcodec_alloc_context3(codec.as_ptr()),
                owner: None,
            }
        }
    }

The function docs can be found here.

This was the code that I used to test:

let encoder = AvContext::from_parameters(writer_stream.parameters())?
            .encoder()
            .video()?;

[SnyderTheOne]

Even with the change of @gil0mendes, I'm still unable to instantiate a 'open' a libx264 encoder. Did anyone manage to get the x264 transcoding example working on ffmpeg 5.0.0 or higher?

[eliasericsson]

Finding myself in the same seat as @SnyderTheOne, unable to run the x264 transcoding example with ffmpeg 5.1.1

[1392492818]

Test available

    let global_header = octx.format().flags().contains(format::Flags::GLOBAL_HEADER);
    let decoder = ffmpeg::codec::context::Context::from_parameters(ist.parameters())?
        .decoder()
        .video()?;
    let mut ost = octx.add_stream(encoder::find(codec::Id::H264))?;


    let mut encoder = codec::context::Context::from_parameters(ost.parameters())?
        .encoder()
        .video()?;
    encoder.set_height(decoder.height());
    encoder.set_width(decoder.width());
    encoder.set_aspect_ratio(decoder.aspect_ratio());
    encoder.set_format(decoder.format());
    encoder.set_frame_rate(Some(ist.rate()));
    encoder.set_time_base(ist.rate().invert());

    if global_header {
        encoder.set_flags(codec::Flags::GLOBAL_HEADER);
    }

    unsafe {
        (*encoder.as_mut_ptr()).me_range = 16;
        (*encoder.as_mut_ptr()).max_qdiff = 4;
        (*encoder.as_mut_ptr()).qmin = 0;
        (*encoder.as_mut_ptr()).qmax = 69;
        (*encoder.as_mut_ptr()).qcompress = 0.6;
    }

    let encoder2 = encoder
       .open_as_with(encoder::find(codec::Id::H264), x264_opts)
        // .open_with(x264_opts)
        .expect("error opening libx264 encoder with supplied settings");

    ost.set_parameters(&encoder2);
    ost.set_rate(ist.rate());
    let mut encoderVideo = codec::context::Context::from_parameters(ost.parameters())?
        .encoder()
        .video()?;

    unsafe {
        // println!("time base {}" , decoder.frame_rate().unwrap());

        // println!("time base {} {}", ist.time_base().0, ist.time_base().1);
        encoderVideo.set_time_base(ist.rate().invert());
        encoderVideo.set_frame_rate(Some(ist.rate()));
        encoderVideo.set_height(decoder.height());
        encoderVideo.set_width(decoder.width());
        encoderVideo.set_aspect_ratio(decoder.aspect_ratio());
        (*encoderVideo.as_mut_ptr()).me_range = 16;
        (*encoderVideo.as_mut_ptr()).max_qdiff = 4;
        (*encoderVideo.as_mut_ptr()).qmin = 0;
        (*encoderVideo.as_mut_ptr()).qmax = 69;
        (*encoderVideo.as_mut_ptr()).qcompress = 0.6;
        avcodec_open2(encoderVideo.as_mut_ptr(), encoder::find(codec::Id::H264).expect("REASON").as_ptr(), &mut parse_opts(
            env::args()
                .nth(3)
                .unwrap_or_else(|| DEFAULT_X264_OPTS.to_string()),
        ).to_owned().expect("x264").disown());
    }
    Ok( Self {
        ost_index,
        decoder,
        encoder: encoderVideo,
        logging_enabled: enable_logging,
        frame_count: 0,
        last_log_frame_count: 0,
        starting_time: Instant::now(),
        last_log_time: Instant::now(),
    })

[Sculas]

Encountering this issue as well on FFmpeg 5.1.2 (ffmpeg-next 6.0.0)
Even with #129 applied locally to test, decoder.frame_rate() is always None for me.
This is my current code:

let mut context = codec::context::Context::new_with_codec(
    decoder::find(ist.parameters().id()).expect("failed to find decoder"),
);
context.set_parameters(ist.parameters())?;
let decoder = context.decoder().video()?;

let codec = encoder::find(codec::Id::H264).expect("failed to find encoder");
let mut ost = octx.add_stream(codec)?;
let mut context = codec::context::Context::new_with_codec(codec);
context.set_parameters(ost.parameters())?;
let mut encoder = context.encoder().video()?;

encoder.set_height(decoder.height());
encoder.set_width(decoder.width());
encoder.set_aspect_ratio(decoder.aspect_ratio());
encoder.set_format(decoder.format());
encoder.set_frame_rate(decoder.frame_rate());
encoder.set_time_base(decoder.frame_rate().unwrap().invert()); // called `Option::unwrap()` on a `None` value

[fuzing]

Looks like there were several breaking changes to the ffmpeg API that have not made it to the APIs for this project, leading to this (and other) examples not working.

For example:

this line:

       let mut ost = octx.add_stream(encoder::find(codec::Id::H264))?;

results in a call to:

      AVStream * avformat_new_stream (AVFormatContext *s, const AVCodec *c)

Pre ffmpeg 5, the avformat_new_stream() function worked like this:

s = media file handle
c = If non-NULL, the AVCodecContext corresponding to the new stream will be initialized to use this codec. This is needed for e.g. codec-specific defaults to be set, so codec should be provided if it is known.

But ffmpeg v5 and greater now ignores the codec information.

s = media file handle
c = unused, does nothing

So various parameters on the stream related to the codec are not being set, and therefore:

        let mut encoder = codec::context::Context::from_parameters(ost.parameters())?
            .encoder()
            .video()?;

sets up an encoder (i.e. codec context) based on parameters (ost.parameters()) that won't have been correctly set.

Note that this example doesn't necessarily explain the fact that frame_rate is not set, but is indicative of the fact that there are likely (many) other breaking changes in the ffmpeg codebase that have not been propagated through this codebase.

Note that I've tried getting this transcoder code to work with ffmpeg versions <= 4 and they still break. The only success I had was reverting both ffmpeg AND this repo to a commit from October 2021.

Unfortunately I'm not experienced enough with rust to be able to offer suggestions as to fixes right now.

[fuzing]

Further to my earlier comment, the following will work for setting the framerate, provided you want the output (encoder) rate to equal the input (decoder rate):

   encoder.set_frame_rate(Some(ist.avg_frame_rate()));
   encoder.set_time_base(ist.avg_frame_rate().invert());

The

   ist.rate()

Could also be used. Here, I'm just making the frame_rate and time_base equivalent to those found on the input stream.

Alas, the code then breaks at:

   encoder
          .open_with(x264_opts)
          .expect("error opening libx264 encoder with supplied settings");

due to an

    avcodec_open2()

call with bad data (it complains that no codec is being passed, likely as a result of incorrect encoder (context) setup as outlined in my previous comment.

[fuzing]

After messing around with this crate I've now found quite a few breaking API changes......... there don't appear to be any decent safe bindings for ffmpeg for rust......... so have abandoned rust ffmpeg in favor of gstreamer, which has excellent rust support.

[Type1J]

I'm looking for a crate that I can create a DASH init.mp4 and many segment files from RGB frames. I believe this crate to be the one to do so, but it seems to be currently broken. Hopefully, someone will fix it soon.

[nununoisy]

I have a working encoder implementation here (MIT licensed): https://github.com/nununoisy/spc-presenter-rs/tree/v0.1.1/src/video_builder

I've verified it works for:

• H.264 (libx264, yuv420p) + AAC (aac, fltp) in MP4
• H.264 + AAC in MKV
• H.264 + AAC in MOV
• ProRes 4444 (prores_ks, yuv444p10le) + AAC in MOV

I've tested it with FFmpeg 4.4 on Linux, FFmpeg 5 on Windows (from vcpkg on msvc), and FFmpeg 6 on Linux. FFmpeg 6 requires this code in your build.rs:

// Call this somewhere in main()
fn ffmpeg_sys_version_detect() {
    for (name, _value) in std::env::vars() {
        if name.starts_with("DEP_FFMPEG_") {
            println!(
                r#"cargo:rustc-cfg=feature="{}""#,
                name["DEP_FFMPEG_".len()..name.len()].to_lowercase()
            );
        }
    }
}

I did need to write some unsafe functions to interact with ffmpeg-sys-next directly in some cases. Hopefully, those functions can be used to extend the current API to allow for constructing a working encoder.

[iamdb]

I just started a project using this crate and am seeing this issue attempting to implement the transcode-x264 example.

Is there a workaround for this or a reason a PR hasn't been merged to address it?

[carlosdp]

For anyone else running into this, the library does have support for v6+ (and the v7 that just launched today, props to the guy who merged that on launch day 🥳 ), but as mentioned above, the API is a little different than v4, and the transcoding x264 example is assuming ffmpeg v4.

The fix is very simple to make it v6/v7 compatible:

// old
let output_context = ffmpeg::codec::context::Context::from_parameters(otx.parameters())?;
// new
let codec = ffmpeg::encoder::find(ffmpeg::codec::Id::H264)?;
let output_context = ffmpeg::codec::context::Context::new_with_codec(codec);

also, toward the end of Transcoder::new(), ignore the part where they re-initialize a new encoder from the output parameters for some reason, and instead use the one returned from the open_with command:

encoder = encoder.open_with(libx264_opts)?;

[Polochon-street]

For anyone else running into this, the library does have support for v6+ (and the v7 that just launched today, props to the guy who merged that on launch day 🥳 ), but as mentioned above, the API is a little different than v4, and the transcoding x264 example is assuming ffmpeg v4.

The fix is very simple to make it v6/v7 compatible:

// old
let output_context = ffmpeg::codec::context::Context::from_parameters(otx.parameters())?;
// new
let codec = ffmpeg::encoder::find(ffmpeg::codec::Id::H264)?;
let output_context = ffmpeg::codec::context::Context::new_with_codec(codec);

also, toward the end of Transcoder::new(), ignore the part where they re-initialize a new encoder from the output parameters for some reason, and instead use the one returned from the open_with command:

encoder = encoder.open_with(libx264_opts)?;

hi!

Would you make a PR with a conditional compilation depending on the ffmpeg version (like this for instance https://github.com/zmwangx/rust-ffmpeg/blob/master/src/codec/capabilities.rs#L9) that would make it compatible with ffmpeg6 onwards? I wouldn't mind merging it :)