incorrect frame.time due to use of codec_time_base

[edhalter]

System: Linux, ffmpeg 3.3.6, PyAV from git (last commit on April 10)

Was using PyAV to iterate through a 25 FPS video and found frame1's .pts is 0 and .time is 0.0. All good. But then frame2's .pts is 40 and .time is 1.6 (instead of 0.04). Hm...

av/frame.pyx:

    property time:
        def __get__(self):
            if self.ptr.pts == lib.AV_NOPTS_VALUE:
                return None
            else:
                return float(self.ptr.pts) * self._time_base.num / self._time_base.den

That looks fine, but Frame._time_base is evidently set to 1/25, not the 1/1000 it should be (in this case). Using ffprobe terminology, it's calculating time using the stream's codec_time_base instead of the stream's time_base.

Was curious, so then I looked at all the time-related values from ffprobe and tried to match them up w/ values from PyAV. Run these cmds:

ffprobe -select_streams v -show_format -show_streams /path/to/video.mp4 2>/dev/null | grep -e '^\[' -e 'duration' -e '_time' -e 'frame_rate' -e 'start' -e 'base' --color=never
ffprobe -select_streams v -show_frames /path/to/video.mp4 2>/dev/null | sed -n '26,50p;51q' | grep -e '^\[' -e 'pts' -e 'dts' -e 'time' -e 'duration' --color=never

(Note that the 2nd cmd grabs the 2nd frame's values, since the 1st frame has mostly 0s.) Output is:

[STREAM]
codec_time_base=1/25
r_frame_rate=25/1
avg_frame_rate=25/1
time_base=1/1000
start_pts=0
start_time=0.000000
duration_ts=N/A
duration=N/A
[/STREAM]
[FORMAT]
start_time=0.000000
duration=90.600000
[/FORMAT]

[FRAME]
pkt_pts=40
pkt_pts_time=0.040000
pkt_dts=40
pkt_dts_time=0.040000
best_effort_timestamp=40
best_effort_timestamp_time=0.040000
pkt_duration=N/A
pkt_duration_time=N/A
[/FRAME]

Then, in a python console:

import av
path = '/path/to/video.mp4'
container = av.open(path)
stream = container.streams.video[0]
packet = next(container.demux(stream))
frames = packet.decode()
frame = frames[0]
properties = {}
properties['format'] = {'start_time':container.start_time, 'duration':container.duration}
properties['stream'] = {'codec_time_base':str(stream.codec_context.time_base), 'avg_frame_rate':str(stream.average_rate), 'time_base':stream.time_base, 'start_time':stream.start_time}
properties['packet'] = {'dts':packet.dts, 'duration':packet.duration, 'pts':packet.pts, 'time_base':packet.time_base}
properties['frame'] = {'dts':frame.dts, 'pts':frame.pts, 'time':frame.time, 'time_base':str(frame.time_base)}
import pprint
pprint.pprint(properties)

Output is:

{'format': {'duration': 90600000, 'start_time': 0},
 'frame': {'dts': 0, 'pts': 0, 'time': 0.0, 'time_base': '1/25'},
 'packet': {'dts': 0, 'duration': 0, 'pts': 0, 'time_base': Fraction(1, 1000)},
 'stream': {'avg_frame_rate': '25',
            'codec_time_base': '1/25',
            'start_time': 0,
            'time_base': Fraction(1, 1000)}}

Some differences between ffprobe and PyAV:

• aforementioned issue w/ calculation of frame.time
• couldn't find some properties in PyAV: stream.start_pts, stream.r_frame_rate, stream.duration_ts, stream.duration, frame.best_effort_timestamp, frame.best_effort_timestamp_time
• ffprobe reports times and durations as floats; PyAV sometimes uses floats (e.g. frame.time), but at other times uses ints representing millisec (or microsec, for format.duration)

[mikeboers]

Thank you for looking into this so thoroughly. This needs a good sit and read along with my time overview.

I hope to do that in the near future. If you keep digging into it please let me know how the rules need adjusting.

[jlaine]

I ran into the same issue, and using the stream's time_base seems to yield the correct presentation time as in:

frame_time = frame.pts * stream.time_base

I'll have a look at the code and submit a PR

[mikeboers]

I'm pretty sure this was fixed in f2425f6.

[jlaine]

Ah sorry, I was using the version on pypi, I'll give master a try.

[mikeboers]

No apologies will be accepted for this 😛 . This is a horribly complex project IMO with tons of fixes and tiny features left dangling in branches forever.