Skip to content
Permalink
Browse files
Implement WebCodecs video encoder flush
https://bugs.webkit.org/show_bug.cgi?id=246686
rdar://problem/101288914

Reviewed by Eric Carlson.

Implement remote encoder flush by sending an IPC to GPUProcess to flush encoders.
Drive-by fix to make full-cycle-test.https.any_h264_avc test pass by providing the correct AVC description by looking at the sample buffer format description.

Covered by rebased tests.

* LayoutTests/imported/w3c/web-platform-tests/webcodecs/full-cycle-test.https.any_h264_annexb-expected.txt:
* LayoutTests/imported/w3c/web-platform-tests/webcodecs/full-cycle-test.https.any_h264_avc-expected.txt:
* LayoutTests/imported/w3c/web-platform-tests/webcodecs/reconfiguring-encoder.https.any_h264_annexb-expected.txt:
* LayoutTests/imported/w3c/web-platform-tests/webcodecs/reconfiguring-encoder.https.any_h264_avc-expected.txt:
* LayoutTests/imported/w3c/web-platform-tests/webcodecs/temporal-svc-encoding.https.any_h264-expected.txt:
* Source/ThirdParty/libwebrtc/Configurations/libwebrtc.iOS.exp:
* Source/ThirdParty/libwebrtc/Configurations/libwebrtc.iOSsim.exp:
* Source/ThirdParty/libwebrtc/Configurations/libwebrtc.mac.exp:
* Source/ThirdParty/libwebrtc/Source/webrtc/sdk/WebKit/WebKitEncoder.h:
* Source/ThirdParty/libwebrtc/Source/webrtc/sdk/WebKit/WebKitEncoder.mm:
(-[WK_RTCLocalVideoH264H265Encoder flush]):
(webrtc::flushLocalEncoder):
* Source/ThirdParty/libwebrtc/Source/webrtc/sdk/objc/components/video_codec/RTCVideoEncoderH264.h:
* Source/ThirdParty/libwebrtc/Source/webrtc/sdk/objc/components/video_codec/RTCVideoEncoderH264.mm:
(-[RTCVideoEncoderH264 setUseAnnexB:]):
(-[RTCVideoEncoderH264 configureCompressionSession]):
(-[RTCVideoEncoderH264 setEncoderBitrateBps:frameRate:]):
(-[RTCVideoEncoderH264 frameWasEncoded🎏sampleBuffer:codecSpecificInfo:width:height:renderTimeMs:timestamp:rotation:isKeyFrameRequired:]):
(-[RTCVideoEncoderH264 flush]):
* Source/ThirdParty/libwebrtc/Source/webrtc/sdk/objc/components/video_codec/RTCVideoEncoderH265.h:
* Source/ThirdParty/libwebrtc/Source/webrtc/sdk/objc/components/video_codec/RTCVideoEncoderH265.mm:
(-[RTCVideoEncoderH265 setUseAnnexB:]):
(-[RTCVideoEncoderH265 frameWasEncoded🎏sampleBuffer:width:height:renderTimeMs:timestamp:rotation:]):
(-[RTCVideoEncoderH265 flush]):
* Source/WebKit/GPUProcess/webrtc/LibWebRTCCodecsProxy.h:
* Source/WebKit/GPUProcess/webrtc/LibWebRTCCodecsProxy.messages.in:
* Source/WebKit/GPUProcess/webrtc/LibWebRTCCodecsProxy.mm:
(WebKit::LibWebRTCCodecsProxy::flushEncoder):
* Source/WebKit/WebProcess/GPU/media/RemoteVideoCodecFactory.cpp:
(WebKit::RemoteVideoEncoder::flush):
* Source/WebKit/WebProcess/GPU/webrtc/LibWebRTCCodecs.cpp:
(WebKit::LibWebRTCCodecs::createEncoderAndWaitUntilReady):
(WebKit::LibWebRTCCodecs::createEncoderInternal):
(WebKit::LibWebRTCCodecs::flushEncoder):
(WebKit::LibWebRTCCodecs::flushEncoderCompleted):
* Source/WebKit/WebProcess/GPU/webrtc/LibWebRTCCodecs.h:
* Source/WebKit/WebProcess/GPU/webrtc/LibWebRTCCodecs.messages.in:

Canonical link: https://commits.webkit.org/255785@main
  • Loading branch information
youennf committed Oct 20, 2022
1 parent 6f0bde8 commit 7cf8c6db3ef5499b2a5c5d92fa14ffe329e7d009
Show file tree
Hide file tree
Showing 21 changed files with 218 additions and 106 deletions.
@@ -1,4 +1,3 @@
CONSOLE MESSAGE: Error: assert_unreached: Decoder failure Reached unreachable code

FAIL Encoding and decoding cycle promise_test: Unhandled rejection with value: object "AbortError: aborting flush as decoder is reset"
PASS Encoding and decoding cycle

@@ -1,11 +1,3 @@
CONSOLE MESSAGE: DataError: Key frame is required
CONSOLE MESSAGE: DataError: Key frame is required

Harness Error (FAIL), message = DataError: Key frame is required

TIMEOUT Encoding and decoding cycle Test timed out

Harness Error (FAIL), message = DataError: Key frame is required

TIMEOUT Encoding and decoding cycle Test timed out
PASS Encoding and decoding cycle

@@ -1,6 +1,7 @@
CONSOLE MESSAGE: Error: assert_unreached: assert_equals: expected 800 but got 640 Reached unreachable code
CONSOLE MESSAGE: Error: assert_unreached: assert_equals: expected 800 but got 640 Reached unreachable code

Harness Error (FAIL), message = Error: assert_unreached: assert_equals: expected 800 but got 640 Reached unreachable code

FAIL Reconfiguring encoder assert_equals: expected 16 but got 1
PASS Reconfiguring encoder

@@ -1,37 +1,7 @@
CONSOLE MESSAGE: Error: assert_unreached: assert_equals: expected 800 but got 640 Reached unreachable code
CONSOLE MESSAGE: Error: assert_unreached: assert_equals: expected 800 but got 640 Reached unreachable code
CONSOLE MESSAGE: Error: assert_unreached: assert_equals: expected 800 but got 640 Reached unreachable code
CONSOLE MESSAGE: Error: assert_unreached: assert_equals: expected 800 but got 640 Reached unreachable code
CONSOLE MESSAGE: Error: assert_unreached: assert_equals: expected 800 but got 640 Reached unreachable code
CONSOLE MESSAGE: Error: assert_unreached: assert_equals: expected 800 but got 640 Reached unreachable code
CONSOLE MESSAGE: Error: assert_unreached: assert_equals: expected 800 but got 640 Reached unreachable code
CONSOLE MESSAGE: Error: assert_unreached: assert_equals: expected 800 but got 640 Reached unreachable code
CONSOLE MESSAGE: Error: assert_unreached: assert_equals: expected 800 but got 640 Reached unreachable code
CONSOLE MESSAGE: Error: assert_unreached: assert_equals: expected 800 but got 640 Reached unreachable code
CONSOLE MESSAGE: Error: assert_unreached: assert_equals: expected 800 but got 640 Reached unreachable code
CONSOLE MESSAGE: Error: assert_unreached: assert_equals: expected 800 but got 640 Reached unreachable code
CONSOLE MESSAGE: Error: assert_unreached: assert_equals: expected 800 but got 640 Reached unreachable code
CONSOLE MESSAGE: Error: assert_unreached: assert_equals: expected 800 but got 640 Reached unreachable code
CONSOLE MESSAGE: Error: assert_unreached: assert_equals: expected 800 but got 640 Reached unreachable code
CONSOLE MESSAGE: Error: assert_unreached: assert_equals: expected 800 but got 640 Reached unreachable code
CONSOLE MESSAGE: Error: assert_unreached: assert_equals: expected 640 but got 800 Reached unreachable code
CONSOLE MESSAGE: Error: assert_unreached: assert_equals: expected 640 but got 800 Reached unreachable code
CONSOLE MESSAGE: Error: assert_unreached: assert_equals: expected 640 but got 800 Reached unreachable code
CONSOLE MESSAGE: Error: assert_unreached: assert_equals: expected 640 but got 800 Reached unreachable code
CONSOLE MESSAGE: Error: assert_unreached: assert_equals: expected 640 but got 800 Reached unreachable code
CONSOLE MESSAGE: Error: assert_unreached: assert_equals: expected 640 but got 800 Reached unreachable code
CONSOLE MESSAGE: Error: assert_unreached: assert_equals: expected 640 but got 800 Reached unreachable code
CONSOLE MESSAGE: Error: assert_unreached: assert_equals: expected 640 but got 800 Reached unreachable code
CONSOLE MESSAGE: Error: assert_unreached: assert_equals: expected 640 but got 800 Reached unreachable code
CONSOLE MESSAGE: Error: assert_unreached: assert_equals: expected 640 but got 800 Reached unreachable code
CONSOLE MESSAGE: Error: assert_unreached: assert_equals: expected 640 but got 800 Reached unreachable code
CONSOLE MESSAGE: Error: assert_unreached: assert_equals: expected 640 but got 800 Reached unreachable code
CONSOLE MESSAGE: Error: assert_unreached: assert_equals: expected 640 but got 800 Reached unreachable code
CONSOLE MESSAGE: Error: assert_unreached: assert_equals: expected 640 but got 800 Reached unreachable code
CONSOLE MESSAGE: Error: assert_unreached: assert_equals: expected 640 but got 800 Reached unreachable code
CONSOLE MESSAGE: Error: assert_unreached: assert_equals: expected 640 but got 800 Reached unreachable code

Harness Error (FAIL), message = Error: assert_unreached: assert_equals: expected 640 but got 800 Reached unreachable code
Harness Error (FAIL), message = Error: assert_unreached: assert_equals: expected 800 but got 640 Reached unreachable code

PASS Reconfiguring encoder

@@ -1,8 +1,54 @@
CONSOLE MESSAGE: Error: assert_own_property: expected property "svc" missing
CONSOLE MESSAGE: Error: assert_own_property: expected property "svc" missing
CONSOLE MESSAGE: Error: assert_own_property: expected property "svc" missing
CONSOLE MESSAGE: Error: assert_own_property: expected property "svc" missing
CONSOLE MESSAGE: Error: assert_own_property: expected property "svc" missing
CONSOLE MESSAGE: Error: assert_own_property: expected property "svc" missing
CONSOLE MESSAGE: Error: assert_own_property: expected property "svc" missing
CONSOLE MESSAGE: Error: assert_own_property: expected property "svc" missing
CONSOLE MESSAGE: Error: assert_own_property: expected property "svc" missing
CONSOLE MESSAGE: Error: assert_own_property: expected property "svc" missing
CONSOLE MESSAGE: Error: assert_own_property: expected property "svc" missing
CONSOLE MESSAGE: Error: assert_own_property: expected property "svc" missing
CONSOLE MESSAGE: Error: assert_own_property: expected property "svc" missing
CONSOLE MESSAGE: Error: assert_own_property: expected property "svc" missing
CONSOLE MESSAGE: Error: assert_own_property: expected property "svc" missing
CONSOLE MESSAGE: Error: assert_own_property: expected property "svc" missing
CONSOLE MESSAGE: Error: assert_own_property: expected property "svc" missing
CONSOLE MESSAGE: Error: assert_own_property: expected property "svc" missing
CONSOLE MESSAGE: Error: assert_own_property: expected property "svc" missing
CONSOLE MESSAGE: Error: assert_own_property: expected property "svc" missing
CONSOLE MESSAGE: Error: assert_own_property: expected property "svc" missing
CONSOLE MESSAGE: Error: assert_own_property: expected property "svc" missing
CONSOLE MESSAGE: Error: assert_own_property: expected property "svc" missing
CONSOLE MESSAGE: Error: assert_own_property: expected property "svc" missing
CONSOLE MESSAGE: Error: assert_own_property: expected property "svc" missing
CONSOLE MESSAGE: Error: assert_own_property: expected property "svc" missing
CONSOLE MESSAGE: Error: assert_own_property: expected property "svc" missing
CONSOLE MESSAGE: Error: assert_own_property: expected property "svc" missing
CONSOLE MESSAGE: Error: assert_own_property: expected property "svc" missing
CONSOLE MESSAGE: Error: assert_own_property: expected property "svc" missing
CONSOLE MESSAGE: Error: assert_own_property: expected property "svc" missing
CONSOLE MESSAGE: Error: assert_own_property: expected property "svc" missing
CONSOLE MESSAGE: Error: assert_own_property: expected property "svc" missing
CONSOLE MESSAGE: Error: assert_own_property: expected property "svc" missing
CONSOLE MESSAGE: Error: assert_own_property: expected property "svc" missing
CONSOLE MESSAGE: Error: assert_own_property: expected property "svc" missing
CONSOLE MESSAGE: Error: assert_own_property: expected property "svc" missing
CONSOLE MESSAGE: Error: assert_own_property: expected property "svc" missing
CONSOLE MESSAGE: Error: assert_own_property: expected property "svc" missing
CONSOLE MESSAGE: Error: assert_own_property: expected property "svc" missing
CONSOLE MESSAGE: Error: assert_own_property: expected property "svc" missing
CONSOLE MESSAGE: Error: assert_own_property: expected property "svc" missing
CONSOLE MESSAGE: Error: assert_own_property: expected property "svc" missing
CONSOLE MESSAGE: Error: assert_own_property: expected property "svc" missing
CONSOLE MESSAGE: Error: assert_own_property: expected property "svc" missing
CONSOLE MESSAGE: Error: assert_own_property: expected property "svc" missing
CONSOLE MESSAGE: Error: assert_own_property: expected property "svc" missing
CONSOLE MESSAGE: Error: assert_own_property: expected property "svc" missing

Harness Error (FAIL), message = Error: assert_own_property: expected property "svc" missing

FAIL SVC L1T2 assert_equals: expected 24 but got 1
FAIL SVC L1T3 assert_equals: expected 24 but got 1
FAIL SVC L1T2 assert_equals: expected 12 but got 0
FAIL SVC L1T3 assert_equals: expected 6 but got 0

@@ -219,6 +219,7 @@ __ZN6webrtc18createLocalEncoderERKNS_14SdpVideoFormatEbU13block_pointerFvPKhmRKN
__ZN6webrtc14SdpVideoFormatD1Ev
__ZN6webrtc19releaseLocalEncoderEPv
__ZN6webrtc23encodeLocalEncoderFrameEPvP10__CVBufferxjNS_13VideoRotationEb
__ZN6webrtc17flushLocalEncoderEPv
__ZN3rtc17CopyOnWriteBuffer24UnshareAndEnsureCapacityEm
__ZN3rtc6Thread14InvokeInternalERKNS_8LocationENS_12FunctionViewIFvvEEE
__ZN6webrtc18RtpSenderInterface38SetEncoderToPacketizerFrameTransformerEN3rtc13scoped_refptrINS_25FrameTransformerInterfaceEEE
@@ -219,6 +219,7 @@ __ZN6webrtc18createLocalEncoderERKNS_14SdpVideoFormatEbU13block_pointerFvPKhmRKN
__ZN6webrtc14SdpVideoFormatD1Ev
__ZN6webrtc19releaseLocalEncoderEPv
__ZN6webrtc23encodeLocalEncoderFrameEPvP10__CVBufferxjNS_13VideoRotationEb
__ZN6webrtc17flushLocalEncoderEPv
__ZN3rtc17CopyOnWriteBuffer24UnshareAndEnsureCapacityEm
__ZN3rtc6Thread14InvokeInternalERKNS_8LocationENS_12FunctionViewIFvvEEE
__ZN6webrtc18RtpSenderInterface38SetEncoderToPacketizerFrameTransformerEN3rtc13scoped_refptrINS_25FrameTransformerInterfaceEEE
@@ -219,6 +219,7 @@ __ZN6webrtc18createLocalEncoderERKNS_14SdpVideoFormatEbU13block_pointerFvPKhmRKN
__ZN6webrtc14SdpVideoFormatD1Ev
__ZN6webrtc19releaseLocalEncoderEPv
__ZN6webrtc23encodeLocalEncoderFrameEPvP10__CVBufferxjNS_13VideoRotationEb
__ZN6webrtc17flushLocalEncoderEPv
__ZN3rtc17CopyOnWriteBuffer24UnshareAndEnsureCapacityEm
__ZN3rtc6Thread14InvokeInternalERKNS_8LocationENS_12FunctionViewIFvvEEE
__ZN6webrtc18RtpSenderInterface38SetEncoderToPacketizerFrameTransformerEN3rtc13scoped_refptrINS_25FrameTransformerInterfaceEEE
@@ -82,6 +82,7 @@ void encodeLocalEncoderFrame(LocalEncoder, CVPixelBufferRef, int64_t timeStampNs
void setLocalEncoderRates(LocalEncoder, uint32_t bitRate, uint32_t frameRate);
void setLocalEncoderLowLatency(LocalEncoder, bool isLowLatencyEnabled);
void encoderVideoTaskComplete(void*, webrtc::VideoCodecType, const uint8_t* buffer, size_t length, const WebKitEncodedFrameInfo&);
void flushLocalEncoder(LocalEncoder);

template<class Decoder>
bool WebKitEncodedFrameInfo::decode(Decoder& decoder, WebKitEncodedFrameInfo& info)
@@ -50,6 +50,7 @@ - (int)setBitrate:(uint32_t)bitrateKbit framerate:(uint32_t)framerate;
- (void)setLowLatency:(bool)lowLatencyEnabled;
- (void)setUseAnnexB:(bool)useAnnexB;
- (void)setDescriptionCallback:(RTCVideoEncoderDescriptionCallback)callback;
- (void)flush;
@end

@implementation WK_RTCLocalVideoH264H265Encoder {
@@ -123,6 +124,14 @@ - (void)setDescriptionCallback:(RTCVideoEncoderDescriptionCallback)callback {
}
[m_h265Encoder setDescriptionCallback:callback];
}

- (void)flush {
if (m_h264Encoder) {
[m_h264Encoder flush];
return;
}
[m_h265Encoder flush];
}
@end

namespace webrtc {
@@ -406,4 +415,10 @@ void setLocalEncoderLowLatency(LocalEncoder localEncoder, bool isLowLatencyEnabl
[encoder setLowLatency:isLowLatencyEnabled];
}

void flushLocalEncoder(LocalEncoder localEncoder)
{
auto *encoder = (__bridge WK_RTCLocalVideoH264H265Encoder *)(localEncoder);
[encoder flush];
}

}
@@ -22,4 +22,5 @@ __attribute__((objc_runtime_name("WK_RTCVideoEncoderH264")))
- (void)setH264LowLatencyEncoderEnabled:(bool)enabled;
- (void)setUseAnnexB:(bool)useAnnexB;
- (void)setDescriptionCallback:(RTCVideoEncoderDescriptionCallback)callback;
- (void)flush;
@end
@@ -364,6 +364,7 @@ @implementation RTCVideoEncoderH264 {
uint32_t _frameCount;
int64_t _lastFrameRateEstimationTime;
bool _useAnnexB;
bool _needsToSendDescription;
RTCVideoEncoderDescriptionCallback _descriptionCallback;
}

@@ -443,6 +444,7 @@ - (NSInteger)startEncodeWithSettings:(RTCVideoEncoderSettings *)settings
- (void)setUseAnnexB:(bool)useAnnexB
{
_useAnnexB = useAnnexB;
_needsToSendDescription = !useAnnexB;
}

- (void)setDescriptionCallback:(RTCVideoEncoderDescriptionCallback)callback
@@ -844,7 +846,7 @@ - (void)configureCompressionSession {
RTC_DCHECK([self hasCompressionSession]);
SetVTSessionProperty(_vtCompressionSession, _vcpCompressionSession, kVTCompressionPropertyKey_RealTime, true);
#if ENABLE_VCP_FOR_H264_BASELINE
if (_useBaseline && _useVCP)
if (_useBaseline && _useVCP && _isH264LowLatencyEncoderEnabled)
SetVTSessionProperty(_vtCompressionSession, _vcpCompressionSession, kVTCompressionPropertyKey_ProfileLevel, kVTProfileLevel_H264_ConstrainedBaseline_AutoLevel);
else
#endif
@@ -907,29 +909,30 @@ - (void)setEncoderBitrateBps:(uint32_t)bitrateBps frameRate:(uint32_t)frameRate
_vtCompressionSession, _vcpCompressionSession, kVTCompressionPropertyKey_ExpectedFrameRate, frameRate);
}

// TODO(tkchin): Add a helper method to set array value.
int64_t dataLimitBytesPerSecondValue =
static_cast<int64_t>(_isBelowExpectedFrameRate ? actualTarget / 8 : actualTarget * kLimitToAverageBitRateFactor / 8);
CFNumberRef bytesPerSecond =
CFNumberCreate(kCFAllocatorDefault, kCFNumberSInt64Type, &dataLimitBytesPerSecondValue);
int64_t oneSecondValue = 1;
CFNumberRef oneSecond =
CFNumberCreate(kCFAllocatorDefault, kCFNumberSInt64Type, &oneSecondValue);
const void *nums[2] = {bytesPerSecond, oneSecond};
CFArrayRef dataRateLimits = CFArrayCreate(nullptr, nums, 2, &kCFTypeArrayCallBacks);

SetVTSessionProperty(_vtCompressionSession, _vcpCompressionSession, kVTCompressionPropertyKey_DataRateLimits, dataRateLimits);

if (bytesPerSecond) {
CFRelease(bytesPerSecond);
}
if (oneSecond) {
CFRelease(oneSecond);
}
if (dataRateLimits) {
CFRelease(dataRateLimits);
if (_isH264LowLatencyEncoderEnabled) {
// TODO(tkchin): Add a helper method to set array value.
int64_t dataLimitBytesPerSecondValue =
static_cast<int64_t>(_isBelowExpectedFrameRate ? actualTarget / 8 : actualTarget * kLimitToAverageBitRateFactor / 8);
CFNumberRef bytesPerSecond =
CFNumberCreate(kCFAllocatorDefault, kCFNumberSInt64Type, &dataLimitBytesPerSecondValue);
int64_t oneSecondValue = 1;
CFNumberRef oneSecond =
CFNumberCreate(kCFAllocatorDefault, kCFNumberSInt64Type, &oneSecondValue);
const void *nums[2] = {bytesPerSecond, oneSecond};
CFArrayRef dataRateLimits = CFArrayCreate(nullptr, nums, 2, &kCFTypeArrayCallBacks);

SetVTSessionProperty(_vtCompressionSession, _vcpCompressionSession, kVTCompressionPropertyKey_DataRateLimits, dataRateLimits);

if (bytesPerSecond) {
CFRelease(bytesPerSecond);
}
if (oneSecond) {
CFRelease(oneSecond);
}
if (dataRateLimits) {
CFRelease(dataRateLimits);
}
}

_encoderBitrateBps = bitrateBps;
_encoderFrameRate = frameRate;
}
@@ -1006,22 +1009,30 @@ - (void)frameWasEncoded:(OSStatus)status
return;
}
} else {
if (_descriptionCallback) {
buffer->SetSize(0);
CMBlockBufferRef blockBuffer = CMSampleBufferGetDataBuffer(sampleBuffer);
size_t currentStart = 0;
size_t size = CMBlockBufferGetDataLength(blockBuffer);
while (currentStart < size) {
char* data = nullptr;
size_t length;
if (auto error = CMBlockBufferGetDataPointer(blockBuffer, currentStart, &length, nullptr, &data)) {
RTC_LOG(LS_ERROR) << "H264 decoder: CMBlockBufferGetDataPointer failed with error " << error;
return;
buffer->SetSize(0);
CMBlockBufferRef blockBuffer = CMSampleBufferGetDataBuffer(sampleBuffer);
size_t currentStart = 0;
size_t size = CMBlockBufferGetDataLength(blockBuffer);
while (currentStart < size) {
char* data = nullptr;
size_t length;
if (auto error = CMBlockBufferGetDataPointer(blockBuffer, currentStart, &length, nullptr, &data)) {
RTC_LOG(LS_ERROR) << "H264 decoder: CMBlockBufferGetDataPointer failed with error " << error;
return;
}
buffer->AppendData(data, size);
currentStart += size;
}
if (_descriptionCallback && _needsToSendDescription) {
auto formatDescription = CMSampleBufferGetFormatDescription(sampleBuffer);
auto sampleExtensionsDict = static_cast<CFDictionaryRef>(CMFormatDescriptionGetExtension(formatDescription, kCMFormatDescriptionExtension_SampleDescriptionExtensionAtoms));
if (sampleExtensionsDict) {
auto sampleExtensions = static_cast<CFDataRef>(CFDictionaryGetValue(sampleExtensionsDict, CFSTR("avcC")));
if (sampleExtensions) {
_needsToSendDescription = false;
_descriptionCallback(reinterpret_cast<const uint8_t*>(CFDataGetBytePtr(sampleExtensions)), CFDataGetLength(sampleExtensions));
}
buffer->AppendData(data, size);
currentStart += size;
}
_descriptionCallback(buffer->data(), buffer->size());
}
}

@@ -1043,9 +1054,11 @@ - (void)frameWasEncoded:(OSStatus)status
RTCVideoContentTypeUnspecified;
frame.flags = webrtc::VideoSendTiming::kInvalid;

_h264BitstreamParser.ParseBitstream(*buffer);
auto qp = _h264BitstreamParser.GetLastSliceQp();
frame.qp = @(qp.value_or(0));
if (_useAnnexB) {
_h264BitstreamParser.ParseBitstream(*buffer);
auto qp = _h264BitstreamParser.GetLastSliceQp();
frame.qp = @(qp.value_or(0));
}

BOOL res = _callback(frame, codecSpecificInfo, nullptr);
if (!res) {
@@ -1062,4 +1075,9 @@ - (nullable RTCVideoEncoderQpThresholds *)scalingSettings {
high:kHighH264QpThreshold];
}

- (void)flush {
if (_vtCompressionSession)
VTCompressionSessionCompleteFrames(_vtCompressionSession, kCMTimeInvalid);
}

@end
@@ -21,4 +21,5 @@ __attribute__((objc_runtime_name("WK_RTCVideoEncoderH265")))
- (instancetype)initWithCodecInfo:(RTCVideoCodecInfo *)codecInfo;
- (void)setUseAnnexB:(bool)useAnnexB;
- (void)setDescriptionCallback:(RTCVideoEncoderDescriptionCallback)callback;
- (void)flush;
@end

0 comments on commit 7cf8c6d

Please sign in to comment.