Skip to content

Commit

Permalink
WebCodecs remote video encoders get timestamps in microseconds and no…
Browse files Browse the repository at this point in the history
…t nanoseconds

rdar://123684436
https://bugs.webkit.org/show_bug.cgi?id=270159

Reviewed by Eric Carlson.

In WebRTC code path, we are getting a microsecond timestamp.
We were converting it to a MediaTime in LibWebRTCCodecs which was then converted back to
a nanosecond timestamp in LibWebRTCCodecsProxy.
A factor of 1000 was applied in LibWebRTCCodecs and handled correctly on LibWebRTCCodecsProxy.

The WebCodecs path is getting a MediaTime directly but without the 1000 factor.
This was causing the timestamps given to WebCodecs encoder to be in microseconds and not nanoseconds.
This can have an impact on encoders in some circumstances.

We fix this by using Seconds/MediaTime routines on WebProcess WebRTC side (this removes the 1000 factor),
and we update LibWebRTCCodecsProxy to do the same.

* Source/WebKit/GPUProcess/webrtc/LibWebRTCCodecsProxy.mm:
(WebKit::LibWebRTCCodecsProxy::encodeFrame):
* Source/WebKit/WebProcess/GPU/webrtc/LibWebRTCCodecs.cpp:
(WebKit::LibWebRTCCodecs::encodeFrame):

Canonical link: https://commits.webkit.org/275392@main
  • Loading branch information
youennf committed Feb 27, 2024
1 parent 75b646b commit aef6a76
Show file tree
Hide file tree
Showing 2 changed files with 2 additions and 2 deletions.
2 changes: 1 addition & 1 deletion Source/WebKit/GPUProcess/webrtc/LibWebRTCCodecsProxy.mm
Original file line number Diff line number Diff line change
Expand Up @@ -440,7 +440,7 @@ static bool validateEncoderConfiguration(VideoCodecType codecType, const String&

#if !PLATFORM(MACCATALYST)
encoder->encodingCallbacks.append(WTFMove(callback));
webrtc::encodeLocalEncoderFrame(encoder->webrtcEncoder, pixelBuffer.get(), sharedVideoFrame.time.toTimeScale(1000000).timeValue(), timeStamp, duration, toWebRTCVideoRotation(sharedVideoFrame.rotation), shouldEncodeAsKeyFrame);
webrtc::encodeLocalEncoderFrame(encoder->webrtcEncoder, pixelBuffer.get(), Seconds(sharedVideoFrame.time.toDouble()).nanoseconds(), timeStamp, duration, toWebRTCVideoRotation(sharedVideoFrame.rotation), shouldEncodeAsKeyFrame);
#else
callback(false);
#endif
Expand Down
2 changes: 1 addition & 1 deletion Source/WebKit/WebProcess/GPU/webrtc/LibWebRTCCodecs.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -693,7 +693,7 @@ template<typename Frame> int32_t LibWebRTCCodecs::encodeFrameInternal(Encoder& e

int32_t LibWebRTCCodecs::encodeFrame(Encoder& encoder, const webrtc::VideoFrame& frame, bool shouldEncodeAsKeyFrame)
{
return encodeFrameInternal(encoder, frame, shouldEncodeAsKeyFrame, toVideoRotation(frame.rotation()), MediaTime(frame.timestamp_us() * 1000, 1000000), frame.timestamp(), { }, [](auto) { });
return encodeFrameInternal(encoder, frame, shouldEncodeAsKeyFrame, toVideoRotation(frame.rotation()), MediaTime::createWithDouble(Seconds::fromMicroseconds(frame.timestamp_us()).value()), frame.timestamp(), { }, [](auto) { });
}

int32_t LibWebRTCCodecs::encodeFrame(Encoder& encoder, const WebCore::VideoFrame& frame, int64_t timestamp, std::optional<uint64_t> duration, bool shouldEncodeAsKeyFrame, Function<void(bool)>&& callback)
Expand Down

0 comments on commit aef6a76

Please sign in to comment.