New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. Weβll occasionally send you account related emails.
Already on GitHub? Sign in to your account
AX: Support VTT-based extended audio descriptions #4129
AX: Support VTT-based extended audio descriptions #4129
Conversation
EWS run on previous version of this PR (hash ea057dc) |
ea057dc
to
9778d06
Compare
EWS run on previous version of this PR (hash 9778d06) |
9778d06
to
2278150
Compare
EWS run on previous version of this PR (hash 2278150) |
2278150
to
6c18987
Compare
EWS run on previous version of this PR (hash 6c18987) |
6c18987
to
8f54ca7
Compare
EWS run on previous version of this PR (hash 8f54ca7) |
@@ -74,9 +74,9 @@ SpeechSynthesis::SpeechSynthesis(ScriptExecutionContext& context) | |||
|
|||
SpeechSynthesis::~SpeechSynthesis() = default; | |||
|
|||
void SpeechSynthesis::setPlatformSynthesizer(std::unique_ptr<PlatformSpeechSynthesizer> synthesizer) | |||
void SpeechSynthesis::setPlatformSynthesizer(Ref<PlatformSpeechSynthesizer> synthesizer) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This should take a Ref<PlatformSpeechSynthesizer>&&
.
{ | ||
m_platformSpeechSynthesizer = WTFMove(synthesizer); | ||
m_platformSpeechSynthesizer = synthesizer.ptr(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
...and then you can get rid of this.
8f54ca7
to
ebfd896
Compare
https://bugs.webkit.org/show_bug.cgi?id=244931 <rdar://99697422> Reviewed by Jer Noble. * LayoutTests/media/track/captions-webvtt/captions-extended-descriptions.vtt: Added. * LayoutTests/media/track/track-description-cue.html: * LayoutTests/media/track/track-extended-descriptions-expected.txt: Added. * LayoutTests/media/track/track-extended-descriptions.html: Added. * LayoutTests/TestExpectations: Feature is Cocoa-specific so far, skip test globally. * LayoutTests/platform/ios/TestExpectations: Enable test on iOS. * LayoutTests/platform/mac/TestExpectations: Enable test on macOS. * Source/WTF/Scripts/Preferences/WebPreferencesExperimental.yaml: Add 'ExtendedAudioDescriptionsEnabled' setting. * Source/WebCore/Modules/speech/SpeechSynthesis.cpp: (WebCore::SpeechSynthesis::setPlatformSynthesizer): Take a Ref<> synthesizer instead of a unique_ptr. (WebCore::SpeechSynthesis::ensurePlatformSpeechSynthesizer): Use PlatformSpeechSynthesizer::create. * Source/WebCore/Modules/speech/SpeechSynthesis.h: * Source/WebCore/html/HTMLMediaElement.cpp: (WebCore::convertEnumerationToString): Added. (WebCore::HTMLMediaElement::unregisterWithDocument): Cancel speech, clear synthesize. (WebCore::HTMLMediaElement::updateActiveTextTrackCues): Cleanup for readability. (WebCore::HTMLMediaElement::setSpeechSynthesisState): Update for extended descriptions. (WebCore::HTMLMediaElement::speakCueText): Call cue.prepareToSpeak as we don't always want a cue to begin speaking immediately. (WebCore::HTMLMediaElement::pauseSpeakingCueText): Support completing an extended description. (WebCore::HTMLMediaElement::resumeSpeakingCueText): Ditto. (WebCore::HTMLMediaElement::pausePlaybackForExtendedTextDescription): New, pause playback to allow an extended description to complete. (WebCore::HTMLMediaElement::speechSynthesis): Have the media element own the synthesizer since the logic for managing it is here. (WebCore::HTMLMediaElement::executeCueEnterOrExitActionForTime): Support extended descriptions. (WebCore::HTMLMediaElement::addTextTrack): Set m_userPrefersExtendedDescriptions. (WebCore::HTMLMediaElement::configureTextTrackGroup): Use m_userPrefersTextDescriptions. (WebCore::HTMLMediaElement::updatePlayState): Pause and resume speaking here, not in `playPlayer` and `pausePlayer`. (WebCore::HTMLMediaElement::playPlayer): Ditto. (WebCore::HTMLMediaElement::pausePlayer): Ditto. (WebCore::HTMLMediaElement::captionPreferencesChanged): set m_userPrefersExtendedDescriptions. (WebCore::HTMLMediaElement::executeCueEnterOrLeaveAction): Deleted. * Source/WebCore/html/HTMLMediaElement.h: (WTF::LogArgument<WebCore::HTMLMediaElement::SpeechSynthesisState>::toString): * Source/WebCore/html/track/TextTrack.cpp: (WebCore::TextTrack::speechSynthesis): Deleted. * Source/WebCore/html/track/TextTrack.h: * Source/WebCore/html/track/TextTrackCue.h: (WebCore::TextTrackCue::prepareToSpeak): Renamed from `speak`. (WebCore::TextTrackCue::beginSpeaking): Added. (WebCore::TextTrackCue::pauseSpeaking): Added. (WebCore::TextTrackCue::cancelSpeaking): Added. (WebCore::TextTrackCue::speak): Deleted. * Source/WebCore/html/track/VTTCue.cpp: (WebCore::mapVideoRateToSpeechRate): Convert playback rate to web speech rate. (WebCore::VTTCue::prepareToSpeak): Call completion handler when there is nothing to speak or when the track is null. Stash `speechSynthesis` for future use. (WebCore::VTTCue::beginSpeaking): Begin or resume speaking. (WebCore::VTTCue::pauseSpeaking): (WebCore::VTTCue::cancelSpeaking): (WebCore::VTTCue::speak): Deleted. * Source/WebCore/html/track/VTTCue.h: * Source/WebCore/page/CaptionUserPreferences.cpp: (WebCore::CaptionUserPreferences::userPrefersTextDescriptions const): Consider setting for extended descriptions. * Source/WebCore/platform/PlatformSpeechSynthesizer.h: Make refcounted. * Source/WebCore/platform/cocoa/PlatformSpeechSynthesizerCocoa.mm: (-[WebSpeechSynthesisWrapper speakUtterance:]): `client()` is a ref, not a pointer. (-[WebSpeechSynthesisWrapper speechSynthesizer:didStartSpeechUtterance:]): Ditto. (-[WebSpeechSynthesisWrapper speechSynthesizer:didFinishSpeechUtterance:]): Ditto. (-[WebSpeechSynthesisWrapper speechSynthesizer:didPauseSpeechUtterance:]): Ditto. (-[WebSpeechSynthesisWrapper speechSynthesizer:didContinueSpeechUtterance:]): Ditto. (-[WebSpeechSynthesisWrapper speechSynthesizer:didCancelSpeechUtterance:]): Ditto. (-[WebSpeechSynthesisWrapper speechSynthesizer:willSpeakRangeOfSpeechString:utterance:]): Ditto. (WebCore::PlatformSpeechSynthesizer::create): Client can never be null so take ref, not a pointer. (WebCore::PlatformSpeechSynthesizer::PlatformSpeechSynthesizer): * Source/WebCore/platform/mock/PlatformSpeechSynthesizerMock.cpp: (WebCore::PlatformSpeechSynthesizerMock::create): Client can never be null so take ref, not a pointer. (WebCore::PlatformSpeechSynthesizerMock::PlatformSpeechSynthesizerMock): (WebCore::PlatformSpeechSynthesizerMock::speakingFinished): `client()` is a ref, not a pointer. (WebCore::PlatformSpeechSynthesizerMock::speak): Ditto. Use configurable utterance duration. (WebCore::PlatformSpeechSynthesizerMock::cancel): Ditto. (WebCore::PlatformSpeechSynthesizerMock::pause): Ditto. Null-check `m_utterance`. (WebCore::PlatformSpeechSynthesizerMock::resume): Ditto. * Source/WebCore/platform/mock/PlatformSpeechSynthesizerMock.h: (WebCore::PlatformSpeechSynthesizerMock::setUtteranceDuration): * Source/WebCore/testing/Internals.cpp: (WebCore::Internals::enableMockSpeechSynthesizer): (WebCore::Internals::enableMockSpeechSynthesizerForMediaElement): (WebCore::Internals::setSpeechUtteranceDuration): * Source/WebCore/testing/Internals.h: * Source/WebCore/testing/Internals.idl: * Source/WebKit/UIProcess/WebPageProxy.cpp: (WebKit::WebPageProxy::resetSpeechSynthesizer): PlatformSpeechSynthesizer is a Ref, not a unique_ptr. (WebKit::WebPageProxy::speechSynthesisData): Ditto. * Source/WebKit/UIProcess/WebPageProxy.h: Canonical link: https://commits.webkit.org/254502@main
ebfd896
to
c2f7594
Compare
Committed 254502@main (c2f7594): https://commits.webkit.org/254502@main Reviewed commits have been landed. Closing PR #4129 and removing active labels. |
c2f7594
ebfd896