Skip to content

[Cocoa] Support AVIF images for macOS Ventura and iOS 16 #1717

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jun 25, 2022
Merged

[Cocoa] Support AVIF images for macOS Ventura and iOS 16 #1717

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions Source/WTF/wtf/PlatformHave.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1273,6 +1273,14 @@
#define HAVE_APPLE_PUSH_SERVICE_URL_TOKEN_SUPPORT 1
#endif

#if (PLATFORM(MAC) && __MAC_OS_X_VERSION_MIN_REQUIRED >= 130000) \
|| (PLATFORM(MACCATALYST) && __IPHONE_OS_VERSION_MIN_REQUIRED >= 160000) \
|| (PLATFORM(IOS) && __IPHONE_OS_VERSION_MIN_REQUIRED >= 160000) \
|| (PLATFORM(APPLETV) && __TV_OS_VERSION_MAX_ALLOWED >= 160000) \
|| (PLATFORM(WATCHOS) && __WATCH_OS_VERSION_MIN_REQUIRED >= 90000)
#define HAVE_AVIF 1
#endif

#if !PLATFORM(IOS_FAMILY)
#define HAVE_MEDIA_VOLUME_PER_ELEMENT 1
#endif
Expand Down
34 changes: 27 additions & 7 deletions Source/WebCore/loader/cache/CachedResourceRequest.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -126,17 +126,37 @@ void CachedResourceRequest::setDomainForCachePartition(const String& domain)
m_resourceRequest.setDomainForCachePartition(domain);
}

static inline constexpr ASCIILiteral acceptHeaderValueForImageResource(bool supportsVideoImage)
static constexpr ASCIILiteral acceptHeaderValueForWebPImageResource()
{
#if HAVE(WEBP) || USE(WEBP)
#define WEBP_HEADER_PART "image/webp,"
return "image/webp,"_s;
#else
#define WEBP_HEADER_PART ""
return ""_s;
#endif
}

static constexpr ASCIILiteral acceptHeaderValueForAVIFImageResource()
{
#if HAVE(AVIF) || USE(AVIF)
return "image/avif,"_s;
#else
return ""_s;
#endif
}

static constexpr ASCIILiteral acceptHeaderValueForVideoImageResource(bool supportsVideoImage)
{
if (supportsVideoImage)
return WEBP_HEADER_PART "image/png,image/svg+xml,image/*;q=0.8,video/*;q=0.8,*/*;q=0.5"_s;
return WEBP_HEADER_PART "image/png,image/svg+xml,image/*;q=0.8,*/*;q=0.5"_s;
#undef WEBP_HEADER_PART
return "video/*;q=0.8,"_s;
return ""_s;
}

static String acceptHeaderValueForImageResource()
{
return String(acceptHeaderValueForWebPImageResource())
+ acceptHeaderValueForAVIFImageResource()
+ acceptHeaderValueForVideoImageResource(ImageDecoder::supportsMediaType(ImageDecoder::MediaType::Video))
+ "image/png,image/svg+xml,image/*;q=0.8,*/*;q=0.5"_s;
}

String CachedResourceRequest::acceptHeaderValueFromType(CachedResource::Type type)
Expand All @@ -145,7 +165,7 @@ String CachedResourceRequest::acceptHeaderValueFromType(CachedResource::Type typ
case CachedResource::Type::MainResource:
return "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8"_s;
case CachedResource::Type::ImageResource:
return acceptHeaderValueForImageResource(ImageDecoder::supportsMediaType(ImageDecoder::MediaType::Video));
return acceptHeaderValueForImageResource();
case CachedResource::Type::CSSStyleSheet:
return "text/css,*/*;q=0.1"_s;
case CachedResource::Type::SVGDocumentResource:
Expand Down
4 changes: 2 additions & 2 deletions Source/WebCore/platform/MIMETypeRegistry.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -93,7 +93,7 @@ constexpr ComparableCaseFoldingASCIILiteral supportedImageMIMETypeArray[] = {
#if USE(CG) || ENABLE(APNG)
"image/apng",
#endif
#if USE(AVIF)
#if HAVE(AVIF) || USE(AVIF)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

uh oh. Why is there both HAVE(AVIF) and USE(AVIF)? Can we consolidate them?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

HAVE(AVIF) is used by Apple ports where the system frameworks provide the needed support to render these images.
USE(AVIF) is used by non Apple ports. It is defined in Source/cmake/OptionsWPE.cmake and Source/cmake/OptionsGTK.cmake. They use libavif to handle the AVIF images. If we want to merge HAVE(AVIF) and USE(AVIF) and merge HAVE(WEBP) and USE(WEBP), I think it is better to handle this in a separate patch.

Copy link
Contributor

@litherum litherum Jun 25, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we make sure we have a bug open for the separate patch? (Presumably the separate patch would be for both AVIF and WEBP.)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

https://bugs.webkit.org/show_bug.cgi?id=241997

"image/avif",
#endif
"image/bmp",
Expand Down Expand Up @@ -133,7 +133,7 @@ constexpr ComparableCaseFoldingASCIILiteral supportedImageMIMETypeArray[] = {
#if PLATFORM(IOS_FAMILY)
"image/vnd.switfview-jpeg",
#endif
#if (USE(CG) && HAVE(WEBP)) || (!USE(CG) && USE(WEBP))
#if HAVE(WEBP) || USE(WEBP)
"image/webp",
#endif
#if PLATFORM(IOS_FAMILY)
Expand Down
13 changes: 12 additions & 1 deletion Source/WebCore/platform/graphics/cg/ImageDecoderCG.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -76,7 +76,7 @@ static RetainPtr<CFMutableDictionaryRef> createImageSourceOptions()
CFDictionarySetValue(options.get(), kCGImageSourceUseHardwareAcceleration, kCFBooleanFalse);

#if HAVE(IMAGE_RESTRICTED_DECODING) && USE(APPLE_INTERNAL_SDK)
if (ImageDecoderCG::decodingHEICEnabled())
if (ImageDecoderCG::decodingHEICEnabled() || ImageDecoderCG::decodingAVIFEnabled())
CFDictionarySetValue(options.get(), kCGImageSourceEnableRestrictedDecoding, kCFBooleanTrue);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Still not super clear if this line has any repercussions beyond HEIC or AVIF, but okay.

#endif

Expand Down Expand Up @@ -269,6 +269,7 @@ void sharedBufferRelease(void* info)
#endif

bool ImageDecoderCG::s_enableDecodingHEIC = false;
bool ImageDecoderCG::s_enableDecodingAVIF = false;
bool ImageDecoderCG::s_hardwareAcceleratedDecodingDisabled = false;

ImageDecoderCG::ImageDecoderCG(FragmentedSharedBuffer& data, AlphaOption, GammaAndColorProfileOption)
Expand Down Expand Up @@ -621,6 +622,16 @@ bool ImageDecoderCG::decodingHEICEnabled()
return s_enableDecodingHEIC;
}

void ImageDecoderCG::enableDecodingAVIF()
{
s_enableDecodingAVIF = true;
}

bool ImageDecoderCG::decodingAVIFEnabled()
{
return s_enableDecodingAVIF;
}

void ImageDecoderCG::disableHardwareAcceleratedDecoding()
{
s_hardwareAcceleratedDecodingDisabled = true;
Expand Down
4 changes: 4 additions & 0 deletions Source/WebCore/platform/graphics/cg/ImageDecoderCG.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,9 @@ class ImageDecoderCG final : public ImageDecoder {
WEBCORE_EXPORT static void enableDecodingHEIC();
static bool decodingHEICEnabled();

WEBCORE_EXPORT static void enableDecodingAVIF();
static bool decodingAVIFEnabled();

WEBCORE_EXPORT static void disableHardwareAcceleratedDecoding();
static bool hardwareAcceleratedDecodingDisabled();

Expand All @@ -81,6 +84,7 @@ class ImageDecoderCG final : public ImageDecoder {
mutable EncodedDataStatus m_encodedDataStatus { EncodedDataStatus::Unknown };
RetainPtr<CGImageSourceRef> m_nativeDecoder;
static bool s_enableDecodingHEIC;
static bool s_enableDecodingAVIF;
static bool s_hardwareAcceleratedDecodingDisabled;
};

Expand Down
3 changes: 3 additions & 0 deletions Source/WebCore/platform/graphics/cg/UTIRegistry.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,9 @@ const MemoryCompactLookupOnlyRobinHoodHashSet<String>& defaultSupportedImageType
"public.webp"_s,
"com.google.webp"_s,
"org.webmproject.webp"_s,
#endif
#if HAVE(AVIF)
"public.avif"_s,
#endif
};

Expand Down
7 changes: 7 additions & 0 deletions Source/WebKit/Shared/WebProcessCreationParameters.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -164,6 +164,7 @@ void WebProcessCreationParameters::encode(IPC::Encoder& encoder) const
encoder << trustdExtensionHandle;
#endif
encoder << enableDecodingHEIC;
encoder << enableDecodingAVIF;
#endif

#if PLATFORM(IOS_FAMILY)
Expand Down Expand Up @@ -455,6 +456,12 @@ bool WebProcessCreationParameters::decode(IPC::Decoder& decoder, WebProcessCreat
if (!enableDecodingHEIC)
return false;
parameters.enableDecodingHEIC = *enableDecodingHEIC;

std::optional<bool> enableDecodingAVIF;
decoder >> enableDecodingAVIF;
if (!enableDecodingAVIF)
return false;
parameters.enableDecodingAVIF = *enableDecodingAVIF;
#endif

#if PLATFORM(IOS_FAMILY)
Expand Down
1 change: 1 addition & 0 deletions Source/WebKit/Shared/WebProcessCreationParameters.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -205,6 +205,7 @@ struct WebProcessCreationParameters {
SandboxExtension::Handle trustdExtensionHandle;
#endif
bool enableDecodingHEIC { false };
bool enableDecodingAVIF { false };
#endif

#if PLATFORM(IOS_FAMILY)
Expand Down
2 changes: 2 additions & 0 deletions Source/WebKit/UIProcess/Cocoa/WebProcessPoolCocoa.mm
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -402,9 +402,11 @@ static AccessibilityPreferences accessibilityPreferences()
if (auto trustdExtensionHandle = SandboxExtension::createHandleForMachLookup("com.apple.trustd.agent"_s, std::nullopt))
parameters.trustdExtensionHandle = WTFMove(*trustdExtensionHandle);
parameters.enableDecodingHEIC = true;
parameters.enableDecodingAVIF = true;
}
#else
parameters.enableDecodingHEIC = true;
parameters.enableDecodingAVIF = true;
#endif // PLATFORM(MAC)
#endif // HAVE(VIDEO_RESTRICTED_DECODING)

Expand Down
14 changes: 13 additions & 1 deletion Source/WebKit/WebProcess/cocoa/WebProcessCocoa.mm
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -270,9 +270,21 @@ static void softlinkDataDetectorsFrameworks()
SandboxExtension::consumePermanently(parameters.trustdExtensionHandle);
#endif // PLATFORM(MAC)
#if USE(APPLE_INTERNAL_SDK)
OptionSet<VideoDecoderBehavior> videoDecoderBehavior;

if (parameters.enableDecodingHEIC) {
ImageDecoderCG::enableDecodingHEIC();
setVideoDecoderBehaviors({ VideoDecoderBehavior::AvoidIOSurface, VideoDecoderBehavior::AvoidHardware, VideoDecoderBehavior::EnableHEIC });
videoDecoderBehavior.add(VideoDecoderBehavior::EnableHEIC);
}

if (parameters.enableDecodingAVIF) {
ImageDecoderCG::enableDecodingAVIF();
videoDecoderBehavior.add(VideoDecoderBehavior::EnableAVIF);
}

if (videoDecoderBehavior) {
videoDecoderBehavior.add({ VideoDecoderBehavior::AvoidIOSurface, VideoDecoderBehavior::AvoidHardware });
setVideoDecoderBehaviors(videoDecoderBehavior);
}
#endif
#endif // HAVE(VIDEO_RESTRICTED_DECODING)
Expand Down