Support for fractional playback speeds in mlt_produces

[Survey845]

Support fractional playback speeds in mlt_producer

Background & Problem

Currently, mlt_producer_prepare_next() advances the playhead position using:

if (mlt_producer_get_speed(self) != 0)
    mlt_producer_seek(self, mlt_producer_position(self) + mlt_producer_get_speed(self));

Because mlt_producer_position() returns mlt_position (int32_t) and mlt_producer_seek() takes mlt_position, non-integer speeds are integer-truncated on every frame tick:

• 1.5X speed: position + 1.5 truncates to +1 frame every step (plays at 1.0X).
• 0.5X speed: position + 0.5 truncates to +0 every step (freezes playback completely).

As discussed in #541, changing mlt_position from int32_t to double is not viable because it breaks the external/internal ABI and C++ API

---

Proposed Solution:

We can support precise fractional playback speeds without modifying mlt_position by using a sub-frame accumulator stored as a producer property (_sub_position):

1. Initialize _sub_position = 0.0 in mlt_producer_init().
2. Reset _sub_position = 0.0 in mlt_producer_set_speed() when speed changes.
3. In mlt_producer_prepare_next(), accumulate the fractional speed on each step:

void mlt_producer_prepare_next(mlt_producer self)
{
    double speed = mlt_producer_get_speed(self);
    if (speed != 0) {
        mlt_properties properties = MLT_PRODUCER_PROPERTIES(self);
        double sub_position = mlt_properties_get_double(properties, "_sub_position");
        sub_position += speed;
        int step = (int) sub_position;
        sub_position -= step;
        mlt_properties_set_double(properties, "_sub_position", sub_position);
        if (step) {
            mlt_producer_seek(self, mlt_producer_position(self) + step);
        }
    }
}

---

How It Works in Practice

• At 1.5X speed:
  • Tick 1: sub_pos = 1.5 -> step = 1, remainder = 0.5
  • Tick 2: sub_pos = 2.0 -> step = 2, remainder = 0.0
  • Frame sequence: 0 -> 1 -> 3 -> 4 -> 6 -> 7 -> 9
• At 0.5X speed (Slow-motion):
  • Tick 1: sub_pos = 0.5 -> step = 0, remainder = 0.5
  • Tick 2: sub_pos = 1.0 -> step = 1, remainder = 0.0
  • Frame sequence: 0, 0, 1, 1, 2, 2...
• At integer speeds (1.0X, 2.0X):
  • step is always an exact integer, remainder is 0.0, resulting in behavior identical to the current implementation.

---

I have started testing this change on Kdenlive locally, please let me know what you think!

[bmatherly]

I don't have a strong opposition to this change since it is self-contained and backwards compatible. But...

If you give the user fractional speeds, they will ask for continuous audio.
And if you give them continuous audio, they will ask for pitch compensation..

I have been thinking about this in the past and had considered adding this to the consumer implementation instead of the producer service. I would see it working something like this:

1. In the consumer implementation, add a property for the speed factor.
2. Set the producer speed to 1.0 regardless of the consumer speed
3. In the real-time consumer thread, when it is time to chose an image for display, skip frames until the appropriate image is found. The math logic could be similar to what you propose above
4. For audio, accumulate multiple frames and combine their audio into frame chunks that are proportional to the speed increase.
5. Apply resampling before feeding the larger audio chunks to the audio playout API (there are examples in MLT that do this today)
6. Optionally, apply pitch compensation before sending to the API (again, we already have examples that can be followed)

The downside to this is that the consumer implementation would add an optional dependency on a resampler (probably SRC) and pitch compensation (probably rubberband).

Since we already have proven examples for these capabilities in producer_timewarp and link_timeremap, your favorite AI coding assistant could probably whip up a demo pretty fast.