Skip to content
Permalink
Browse files
[Modern Media Controls] REGRESSION(r254389) media controls needs the …
…full list of language preferences for ordering tracks

https://bugs.webkit.org/show_bug.cgi?id=226038

Reviewed by Myles C. Maxfield and Eric Carlson.

Source/JavaScriptCore:

Media controls need access to the full list of language preferences in order to properly
sort/order the list of tracks in the controls UI. For example, if a `<video>` has subtitles
for English, Spanish, and French, and the user has English (default) and French (alternate)
configured in the Language & Region view of System Preferences on macOS, WebKit should order
the subtitles list English, French, and then Spanish.

* shell/playstation/TestShell.cpp:
(preTest):
Instead of propagating the override, just clear it out.

Source/WebCore:

Media controls need access to the full list of language preferences in order to properly
sort/order the list of tracks in the controls UI. For example, if a `<video>` has subtitles
for English, Spanish, and French, and the user has English (default) and French (alternate)
configured in the Language & Region view of System Preferences on macOS, WebKit should order
the subtitles list English, French, and then Spanish.

Test: media/modern-media-controls/tracks-support/sorted-by-user-preferred-languages.html

* page/CaptionUserPreferences.cpp:
(WebCore::CaptionUserPreferences::preferredLanguages const):
(WebCore::CaptionUserPreferences::textTrackSelectionScore const):
(WebCore::CaptionUserPreferences::primaryAudioTrackLanguageOverride const):
* page/CaptionUserPreferencesMediaAF.cpp:
(WebCore::CaptionUserPreferencesMediaAF::preferredLanguages const):
(WebCore::buildDisplayStringForTrackBase):
(WebCore::trackDisplayName):
(WebCore::textTrackCompare):
* platform/graphics/avfoundation/MediaSelectionGroupAVFObjC.mm:
(WebCore::MediaSelectionGroupAVFObjC::updateOptions):

* bindings/js/JSDOMWindowBase.cpp:
* bindings/js/JSWorkerGlobalScopeBase.cpp:
* bindings/js/JSWorkletGlobalScopeBase.cpp:
Wrap `defaultLanguage` in a lambda since it now has a parameter.

* platform/LocalizedStrings.h:
* platform/LocalizedStrings.cpp:
(WebCore::captionsTextTrackKindDisplayName): Added.
(WebCore::descriptionsTextTrackKindDisplayName): Added.
(WebCore::chaptersTextTrackKindDisplayName): Added.
(WebCore::metadataTextTrackKindDisplayName): Added.
(WebCore::closedCaptionKindTrackDisplayName): Added.
(WebCore::sdhTrackKindDisplayName): Added.
(WebCore::easyReaderKindDisplayName): Added.
(WebCore::forcedTrackKindDisplayName): Added.
Move the logic added in r277797 from `buildDisplayStringForTrackBase` to `trackDisplayName`
and only perform it if the text track display name doesn't already contain the display name
for the text track kind (e.g. if the `label` is "English Captions" don't add "Captions", but
do if the `label` is only "English").

Source/WTF:

Media controls need access to the full list of language preferences in order to properly
sort/order the list of tracks in the controls UI. For example, if a `<video>` has subtitles
for English, Spanish, and French, and the user has English (default) and French (alternate)
configured in the Language & Region view of System Preferences on macOS, WebKit should order
the subtitles list English, French, and then Spanish.

* wtf/Language.h:
* wtf/Language.cpp:
(WTF::cachedFullPlatformPreferredLanguages): Added.
(WTF::cachedMinimizedPlatformPreferredLanguages): Added.
(WTF::languageDidChange):
(WTF::defaultLanguage):
(WTF::userPreferredLanguages):
(WTF::platformLanguageDidChange): Deleted.
Move the caching in `wtf/cf/LanguageCF.cpp` to `wtf/Language.cpp` so that all platforms can
benefit from caching the result of `platformUserPreferredLanguages`.

* wtf/cf/LanguageCF.cpp:
(WTF::platformUserPreferredLanguages):
(WTF::platformLanguageDidChange): Deleted.
* wtf/playstation/LanguagePlayStation.cpp:
(WTF::platformUserPreferredLanguages):
* wtf/spi/cocoa/NSLocaleSPI.h:
* wtf/unix/LanguageUnix.cpp:
(WTF::platformUserPreferredLanguages):
* wtf/win/LanguageWin.cpp:
(WTF::platformUserPreferredLanguages):
Add `WTF::ShouldMinimizeLanguages` that controls whether `WTF::minimizedPreferredLanguages`
is called. Since the result is cached, create separate `Vector<String>` for each enum value.

LayoutTests:

* media/modern-media-controls/tracks-support/sorted-by-user-preferred-languages.html: Added.
* media/modern-media-controls/tracks-support/sorted-by-user-preferred-languages-expected.txt: Added.

* media/content/lorem-ipsum.vtt: Added.


Canonical link: https://commits.webkit.org/238145@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@278064 268f45cc-cd09-0410-ab3c-d52691b4dbfc
  • Loading branch information
dcrousso committed May 26, 2021
1 parent 1733b8b commit 985831945d63c3d5922e1459b80014472450849e
Showing 23 changed files with 411 additions and 152 deletions.
@@ -1,3 +1,15 @@
2021-05-25 Devin Rousso <drousso@apple.com>

[Modern Media Controls] REGRESSION(r254389) media controls needs the full list of language preferences for ordering tracks
https://bugs.webkit.org/show_bug.cgi?id=226038

Reviewed by Myles C. Maxfield and Eric Carlson.

* media/modern-media-controls/tracks-support/sorted-by-user-preferred-languages.html: Added.
* media/modern-media-controls/tracks-support/sorted-by-user-preferred-languages-expected.txt: Added.

* media/content/lorem-ipsum.vtt: Added.

2021-05-25 Amir Mark Jr <amir_mark@apple.com>

[ Catalina E Wk1 ] imported/w3c/web-platform-tests/css/css-backgrounds/hidpi/simple-bg-color.html is flaky failure
@@ -0,0 +1,9 @@
WEBVTT
1
00:00:00.000 --> 00:00:05.000
lorem

2
00:00:05.000 --> 00:00:10.000
ipsum
@@ -0,0 +1,28 @@
This test checks that tracks are sorted based on the user's preferred languages.

On success, you will see a series of "PASS" messages, followed by "TEST COMPLETE".


PASS shadowRoot.querySelector('button.tracks') became different from null
PASS shadowRoot.querySelector('button.tracks').getBoundingClientRect().width became different from 0
Tapping tracks button...
Subtitles: [
"Off",
"Auto (Recommended)",
"label-en Captions",
"Descriptions",
"label-en",
"Spanish Captions",
"Descriptions (Spanish)",
"Spanish",
"label-fr (French) Captions",
"Descriptions (French)",
"label-fr (French)",
"German Captions",
"Descriptions (German)",
"German"
]
PASS successfullyParsed is true

TEST COMPLETE

@@ -0,0 +1,55 @@
<!DOCTYPE html> <!-- webkit-test-runner [ language=en-US,es-US useFlexibleViewport=true ] -->
<meta name="viewport" content="width=device-width">
<script src="../resources/media-controls-utils.js"></script>
<script src="../../../resources/ui-helper.js"></script>
<script src="../../../resources/js-test-pre.js"></script>
<body>
<video src="../../content/counting.mp4" style="position: absolute; left: 0; top: 0; width: 600px;" autoplay controls muted playsinline>
<track src="../../lorem-ipsum.vtt" kind="descriptions" srclang="en" label="Descriptions">
<track src="../../lorem-ipsum.vtt" kind="captions" srclang="en" label="label-en">
<track src="../../lorem-ipsum.vtt" kind="subtitles" srclang="en" label="label-en">
<track src="../../lorem-ipsum.vtt" kind="descriptions" srclang="de" label="Descriptions">
<track src="../../lorem-ipsum.vtt" kind="captions" srclang="de">
<track src="../../lorem-ipsum.vtt" kind="subtitles" srclang="de">
<track src="../../lorem-ipsum.vtt" kind="descriptions" srclang="fr" label="Descriptions">
<track src="../../lorem-ipsum.vtt" kind="captions" srclang="fr" label="label-fr">
<track src="../../lorem-ipsum.vtt" kind="subtitles" srclang="fr" label="label-fr">
<track src="../../lorem-ipsum.vtt" kind="descriptions" srclang="es" label="Descriptions">
<track src="../../lorem-ipsum.vtt" kind="captions" srclang="es">
<track src="../../lorem-ipsum.vtt" kind="subtitles" srclang="es">
</video>
<script type="text/javascript">

window.jsTestIsAsync = true;

description(`This test checks that tracks are sorted based on the user's preferred languages.`);

const media = document.querySelector("video");
const shadowRoot = window.internals.shadowRoot(media);

window.internals.setMediaControlsMaximumRightContainerButtonCountOverride(media, 100); // A finite value larger than anything likely.
window.internals.setCaptionDisplayMode("manual"); // Show all tracks.

media.addEventListener("play", () => {
media.pause();

shouldBecomeDifferent("shadowRoot.querySelector('button.tracks')", "null", () => {
shouldBecomeDifferent("shadowRoot.querySelector('button.tracks').getBoundingClientRect().width", "0", async () => {
debug("Tapping tracks button...");
await pressOnElement(shadowRoot.querySelector("button.tracks"));

let contextmenu = await getTracksContextMenu();
let items = contextmenu[0].children;
if (items[0].title === "Subtitles")
items = items.slice(1);
debug("Subtitles: " + JSON.stringify(items.map((item) => item.title), null, 2));

media.remove();
finishJSTest();
});
});
});

</script>
<script src="../../../resources/js-test-post.js"></script>
</body>
@@ -1,3 +1,20 @@
2021-05-25 Devin Rousso <drousso@apple.com>

[Modern Media Controls] REGRESSION(r254389) media controls needs the full list of language preferences for ordering tracks
https://bugs.webkit.org/show_bug.cgi?id=226038

Reviewed by Myles C. Maxfield and Eric Carlson.

Media controls need access to the full list of language preferences in order to properly
sort/order the list of tracks in the controls UI. For example, if a `<video>` has subtitles
for English, Spanish, and French, and the user has English (default) and French (alternate)
configured in the Language & Region view of System Preferences on macOS, WebKit should order
the subtitles list English, French, and then Spanish.

* shell/playstation/TestShell.cpp:
(preTest):
Instead of propagating the override, just clear it out.

2021-05-25 Chris Dumez <cdumez@apple.com>

Stop using UncheckedLock in JSC::PerfLog
@@ -62,7 +62,7 @@ extern "C" void preTest()
JSC::Options::name_() = orig##name_;
FOR_EACH_JSC_OPTION(INIT_OPTION)
#undef INIT_OPTION
overrideUserPreferredLanguages(platformUserPreferredLanguages());
overrideUserPreferredLanguages({ });
}

extern "C" int runTest(int argc, char* argv[])
@@ -1,3 +1,40 @@
2021-05-25 Devin Rousso <drousso@apple.com>

[Modern Media Controls] REGRESSION(r254389) media controls needs the full list of language preferences for ordering tracks
https://bugs.webkit.org/show_bug.cgi?id=226038

Reviewed by Myles C. Maxfield and Eric Carlson.

Media controls need access to the full list of language preferences in order to properly
sort/order the list of tracks in the controls UI. For example, if a `<video>` has subtitles
for English, Spanish, and French, and the user has English (default) and French (alternate)
configured in the Language & Region view of System Preferences on macOS, WebKit should order
the subtitles list English, French, and then Spanish.

* wtf/Language.h:
* wtf/Language.cpp:
(WTF::cachedFullPlatformPreferredLanguages): Added.
(WTF::cachedMinimizedPlatformPreferredLanguages): Added.
(WTF::languageDidChange):
(WTF::defaultLanguage):
(WTF::userPreferredLanguages):
(WTF::platformLanguageDidChange): Deleted.
Move the caching in `wtf/cf/LanguageCF.cpp` to `wtf/Language.cpp` so that all platforms can
benefit from caching the result of `platformUserPreferredLanguages`.

* wtf/cf/LanguageCF.cpp:
(WTF::platformUserPreferredLanguages):
(WTF::platformLanguageDidChange): Deleted.
* wtf/playstation/LanguagePlayStation.cpp:
(WTF::platformUserPreferredLanguages):
* wtf/spi/cocoa/NSLocaleSPI.h:
* wtf/unix/LanguageUnix.cpp:
(WTF::platformUserPreferredLanguages):
* wtf/win/LanguageWin.cpp:
(WTF::platformUserPreferredLanguages):
Add `WTF::ShouldMinimizeLanguages` that controls whether `WTF::minimizedPreferredLanguages`
is called. Since the result is cached, create separate `Vector<String>` for each enum value.

2021-05-25 Chris Dumez <cdumez@apple.com>

Stop using UncheckedLock in html/canvas
@@ -37,6 +37,18 @@

namespace WTF {

static Lock cachedPlatformPreferredLanguagesLock;
static Vector<String>& cachedFullPlatformPreferredLanguages() WTF_REQUIRES_LOCK(cachedPlatformPreferredLanguagesLock)
{
static NeverDestroyed<Vector<String>> languages;
return languages;
}
static Vector<String>& cachedMinimizedPlatformPreferredLanguages() WTF_REQUIRES_LOCK(cachedPlatformPreferredLanguagesLock)
{
static NeverDestroyed<Vector<String>> languages;
return languages;
}

static Lock preferredLanguagesOverrideLock;
static Vector<String>& preferredLanguagesOverride() WTF_REQUIRES_LOCK(preferredLanguagesOverrideLock)
{
@@ -68,16 +80,21 @@ void removeLanguageChangeObserver(void* context)

void languageDidChange()
{
platformLanguageDidChange();
{
Locker locker { cachedPlatformPreferredLanguagesLock };
cachedFullPlatformPreferredLanguages().clear();
cachedMinimizedPlatformPreferredLanguages().clear();
}

for (auto& observer : copyToVector(observerMap())) {
if (observerMap().contains(observer.key))
observer.value(observer.key);
}
}

String defaultLanguage()
String defaultLanguage(ShouldMinimizeLanguages shouldMinimizeLanguages)
{
Vector<String> languages = userPreferredLanguages();
auto languages = userPreferredLanguages(shouldMinimizeLanguages);
if (languages.size())
return languages[0];

@@ -108,7 +125,7 @@ static Vector<String> isolatedCopy(const Vector<String>& strings)
return copy;
}

Vector<String> userPreferredLanguages()
Vector<String> userPreferredLanguages(ShouldMinimizeLanguages shouldMinimizeLanguages)
{
{
Locker locker { preferredLanguagesOverrideLock };
@@ -117,7 +134,11 @@ Vector<String> userPreferredLanguages()
return isolatedCopy(override);
}

return platformUserPreferredLanguages();
Locker locker { cachedPlatformPreferredLanguagesLock };
auto& languages = shouldMinimizeLanguages == ShouldMinimizeLanguages::Yes ? cachedMinimizedPlatformPreferredLanguages() : cachedFullPlatformPreferredLanguages();
if (languages.isEmpty())
languages = platformUserPreferredLanguages(shouldMinimizeLanguages);
return isolatedCopy(languages);
}

static String canonicalLanguageIdentifier(const String& languageCode)
@@ -184,10 +205,4 @@ String displayNameForLanguageLocale(const String& localeName)
return localeName;
}

#if !PLATFORM(COCOA)
void platformLanguageDidChange()
{
}
#endif

}
@@ -35,13 +35,13 @@

namespace WTF {

WTF_EXPORT_PRIVATE String defaultLanguage(); // Thread-safe.
WTF_EXPORT_PRIVATE Vector<String> userPreferredLanguages(); // Thread-safe, returns BCP 47 language tags.
enum class ShouldMinimizeLanguages { No, Yes };

WTF_EXPORT_PRIVATE String defaultLanguage(ShouldMinimizeLanguages = ShouldMinimizeLanguages::Yes); // Thread-safe.
WTF_EXPORT_PRIVATE Vector<String> userPreferredLanguages(ShouldMinimizeLanguages = ShouldMinimizeLanguages::Yes); // Thread-safe, returns BCP 47 language tags.
WTF_EXPORT_PRIVATE Vector<String> userPreferredLanguagesOverride();
WTF_EXPORT_PRIVATE void overrideUserPreferredLanguages(const Vector<String>&);
WTF_EXPORT_PRIVATE size_t indexOfBestMatchingLanguageInList(const String& language, const Vector<String>& languageList, bool& exactMatch);
WTF_EXPORT_PRIVATE Vector<String> platformUserPreferredLanguages();
WTF_EXPORT_PRIVATE void platformLanguageDidChange();

// Called from platform specific code when the user's preferred language(s) change.
WTF_EXPORT_PRIVATE void languageDidChange();
@@ -52,20 +52,22 @@ WTF_EXPORT_PRIVATE void addLanguageChangeObserver(void* context, LanguageChangeO
WTF_EXPORT_PRIVATE void removeLanguageChangeObserver(void* context);
WTF_EXPORT_PRIVATE String displayNameForLanguageLocale(const String&);

Vector<String> platformUserPreferredLanguages(ShouldMinimizeLanguages = ShouldMinimizeLanguages::Yes);

#if PLATFORM(COCOA)
bool canMinimizeLanguages();
WTF_EXPORT_PRIVATE void listenForLanguageChangeNotifications();
RetainPtr<CFArrayRef> minimizedLanguagesFromLanguages(CFArrayRef);
#endif

}
} // namespace WTF

using WTF::ShouldMinimizeLanguages;
using WTF::defaultLanguage;
using WTF::userPreferredLanguages;
using WTF::userPreferredLanguagesOverride;
using WTF::overrideUserPreferredLanguages;
using WTF::indexOfBestMatchingLanguageInList;
using WTF::platformUserPreferredLanguages;
using WTF::addLanguageChangeObserver;
using WTF::removeLanguageChangeObserver;
using WTF::displayNameForLanguageLocale;

0 comments on commit 9858319

Please sign in to comment.