Skip to content
Permalink
Browse files
[iOS][GPU Process] support <attachment>
https://bugs.webkit.org/show_bug.cgi?id=230781
<rdar://problem/70884096>

Reviewed by Myles Maxfield.

Source/WebCore:

* rendering/RenderThemeIOS.mm:
(WebCore::RenderAttachmentInfo::addLine):
(WebCore::RenderAttachmentInfo::buildWrappedLines):
(WebCore::RenderAttachmentInfo::buildSingleLine):
(WebCore::paintAttachmentText):
Instead of drawing directly into the `GraphicsContext::platformContext` (which will not be
valid in the WebProcess when DOM rendering happens in the GPUProcess), use `DrawGlyphsRecorder`
to "translate" native `CTLineDraw` into a sequence of actions from which a `GraphicsContext`
method can be derived, thereby hooking into and benefiting from existing GPUProcess support.

* platform/graphics/DrawGlyphsRecorder.h:
(WebCore::DrawGlyphsRecorder::deconstructDrawGlyphs const): Renamed from `drawGlyphsDeconstruction`.
* platform/graphics/coretext/DrawGlyphsRecorderCoreText.cpp:
(WebCore::DrawGlyphsRecorder::DrawGlyphsRecorder):
(WebCore::DrawGlyphsRecorder::recordDrawGlyphs):
(WebCore::DrawGlyphsRecorder::drawGlyphs):
(WebCore::DrawGlyphsRecorder::drawNativeText):
* platform/graphics/displaylists/DisplayListRecorder.cpp:
* platform/graphics/harfbuzz/DrawGlyphsRecorderHarfBuzz.cpp:
(WebCore::DrawGlyphsRecorder::DrawGlyphsRecorder):
* platform/graphics/win/DrawGlyphsRecorderWin.cpp:
(WebCore::DrawGlyphsRecorder::DrawGlyphsRecorder):
Add a new parameter to `DrawGlyphsRenderer` that controls whether fonts other than the one
initially provided to `drawGlyphs`/`drawNativeText` can be used to draw glyphs. This is only
used by the above as we know that the initial font is a system font, and therefore that the
fallback list will also only contain system fonts, meaning that sending the font to the
GPUProcess is cheap (a file path instead of actual data).

(WebCore::DisplayList::Recorder::Recorder):
* platform/graphics/displaylists/DisplayListRecorder.h:
* platform/graphics/FontCascade.cpp:
(WebCore::FontCascade::displayListForTextRun const):
Drive-by: Rename `DrawGlyphsDeconstruction` to `DeconstructDrawGlyphs` as it reads better.

Source/WebCore/PAL:

* pal/spi/cg/CoreGraphicsSPI.h:


Canonical link: https://commits.webkit.org/242356@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@283339 268f45cc-cd09-0410-ab3c-d52691b4dbfc
  • Loading branch information
dcrousso committed Sep 30, 2021
1 parent e30e218 commit 977741f3d0dab8b91917202e5943c805ca16c6f3
Showing 11 changed files with 112 additions and 33 deletions.
@@ -1,3 +1,45 @@
2021-09-30 Devin Rousso <drousso@apple.com>

[iOS][GPU Process] support `<attachment>`
https://bugs.webkit.org/show_bug.cgi?id=230781
<rdar://problem/70884096>

Reviewed by Myles Maxfield.

* rendering/RenderThemeIOS.mm:
(WebCore::RenderAttachmentInfo::addLine):
(WebCore::RenderAttachmentInfo::buildWrappedLines):
(WebCore::RenderAttachmentInfo::buildSingleLine):
(WebCore::paintAttachmentText):
Instead of drawing directly into the `GraphicsContext::platformContext` (which will not be
valid in the WebProcess when DOM rendering happens in the GPUProcess), use `DrawGlyphsRecorder`
to "translate" native `CTLineDraw` into a sequence of actions from which a `GraphicsContext`
method can be derived, thereby hooking into and benefiting from existing GPUProcess support.

* platform/graphics/DrawGlyphsRecorder.h:
(WebCore::DrawGlyphsRecorder::deconstructDrawGlyphs const): Renamed from `drawGlyphsDeconstruction`.
* platform/graphics/coretext/DrawGlyphsRecorderCoreText.cpp:
(WebCore::DrawGlyphsRecorder::DrawGlyphsRecorder):
(WebCore::DrawGlyphsRecorder::recordDrawGlyphs):
(WebCore::DrawGlyphsRecorder::drawGlyphs):
(WebCore::DrawGlyphsRecorder::drawNativeText):
* platform/graphics/displaylists/DisplayListRecorder.cpp:
* platform/graphics/harfbuzz/DrawGlyphsRecorderHarfBuzz.cpp:
(WebCore::DrawGlyphsRecorder::DrawGlyphsRecorder):
* platform/graphics/win/DrawGlyphsRecorderWin.cpp:
(WebCore::DrawGlyphsRecorder::DrawGlyphsRecorder):
Add a new parameter to `DrawGlyphsRenderer` that controls whether fonts other than the one
initially provided to `drawGlyphs`/`drawNativeText` can be used to draw glyphs. This is only
used by the above as we know that the initial font is a system font, and therefore that the
fallback list will also only contain system fonts, meaning that sending the font to the
GPUProcess is cheap (a file path instead of actual data).

(WebCore::DisplayList::Recorder::Recorder):
* platform/graphics/displaylists/DisplayListRecorder.h:
* platform/graphics/FontCascade.cpp:
(WebCore::FontCascade::displayListForTextRun const):
Drive-by: Rename `DrawGlyphsDeconstruction` to `DeconstructDrawGlyphs` as it reads better.

2021-09-29 Simon Fraser <simon.fraser@apple.com>

Replace the confusing isPinnedForScrollDelta() logic with code that uses BoxSide
@@ -1,3 +1,13 @@
2021-09-30 Devin Rousso <drousso@apple.com>

[iOS][GPU Process] support `<attachment>`
https://bugs.webkit.org/show_bug.cgi?id=230781
<rdar://problem/70884096>

Reviewed by Myles Maxfield.

* pal/spi/cg/CoreGraphicsSPI.h:

2021-09-29 Eric Carlson <eric.carlson@apple.com>

[Cocoa] Support in-band chapter tracks
@@ -270,6 +270,7 @@ void* CGContextDelegateGetInfo(CGContextDelegateRef);
void CGContextDelegateRelease(CGContextDelegateRef);
CGFloat CGGStateGetAlpha(CGGStateRef);
CGFontRef CGGStateGetFont(CGGStateRef);
CGFloat CGGStateGetFontSize(CGGStateRef);
const CGAffineTransform *CGGStateGetCTM(CGGStateRef);
CGColorRef CGGStateGetFillColor(CGGStateRef);
CGColorRef CGGStateGetStrokeColor(CGGStateRef);
@@ -35,6 +35,12 @@

#if USE(CORE_TEXT)
#include <CoreGraphics/CoreGraphics.h>
#include <CoreText/CoreText.h>
#if PLATFORM(WIN)
#include <pal/spi/win/CoreTextSPIWin.h>
#else
#include <pal/spi/cf/CoreTextSPI.h>
#endif
#include <pal/spi/cg/CoreGraphicsSPI.h>
#endif

@@ -47,22 +53,22 @@ class GraphicsContext;

class DrawGlyphsRecorder {
public:
enum class DrawGlyphsDeconstruction {
Deconstruct,
DontDeconstruct
};
explicit DrawGlyphsRecorder(GraphicsContext&, DrawGlyphsDeconstruction);
enum class DeconstructDrawGlyphs : bool { No, Yes };
enum class DeriveFontFromContext : bool { No, Yes };
explicit DrawGlyphsRecorder(GraphicsContext&, DeconstructDrawGlyphs = DeconstructDrawGlyphs::No, DeriveFontFromContext = DeriveFontFromContext::No);

void drawGlyphs(const Font&, const GlyphBufferGlyph*, const GlyphBufferAdvance*, unsigned numGlyphs, const FloatPoint& anchorPoint, FontSmoothingMode);

#if USE(CORE_TEXT) && !PLATFORM(WIN)
void drawNativeText(CTFontRef, CGFloat fontSize, CTLineRef, CGRect lineRect);

void recordBeginLayer(CGRenderingStateRef, CGGStateRef, CGRect);
void recordEndLayer(CGRenderingStateRef, CGGStateRef);
void recordDrawGlyphs(CGRenderingStateRef, CGGStateRef, const CGAffineTransform*, const CGGlyph[], const CGPoint positions[], size_t count);
void recordDrawImage(CGRenderingStateRef, CGGStateRef, CGRect, CGImageRef);
#endif

DrawGlyphsDeconstruction drawGlyphsDeconstruction() const { return m_drawGlyphsDeconstruction; }
DeconstructDrawGlyphs deconstructDrawGlyphs() const { return m_deconstructDrawGlyphs; }

private:
#if USE(CORE_TEXT) && !PLATFORM(WIN)
@@ -89,7 +95,8 @@ class DrawGlyphsRecorder {
#endif

GraphicsContext& m_owner;
DrawGlyphsDeconstruction m_drawGlyphsDeconstruction;
DeconstructDrawGlyphs m_deconstructDrawGlyphs;
DeriveFontFromContext m_deriveFontFromContext;

#if USE(CORE_TEXT) && !PLATFORM(WIN)
UniqueRef<GraphicsContext> m_internalContext;
@@ -213,7 +213,7 @@ std::unique_ptr<DisplayList::InMemoryDisplayList> FontCascade::displayListForTex
return nullptr;

std::unique_ptr<DisplayList::InMemoryDisplayList> displayList = makeUnique<DisplayList::InMemoryDisplayList>();
DisplayList::Recorder recordingContext(*displayList, context.state(), FloatRect(), AffineTransform(), nullptr, DrawGlyphsRecorder::DrawGlyphsDeconstruction::DontDeconstruct);
DisplayList::Recorder recordingContext(*displayList, context.state(), FloatRect(), AffineTransform(), nullptr, DrawGlyphsRecorder::DeconstructDrawGlyphs::No);

FloatPoint startPoint = toFloatPoint(WebCore::size(glyphBuffer.initialAdvance()));
drawGlyphBuffer(recordingContext, glyphBuffer, startPoint, customFontNotReadyAction);
@@ -31,6 +31,7 @@
#include "FloatPoint.h"
#include "Font.h"
#include "FontCascade.h"
#include "FontPlatformData.h"
#include "GlyphBuffer.h"
#include "GraphicsContextCG.h"

@@ -84,9 +85,10 @@ UniqueRef<GraphicsContext> DrawGlyphsRecorder::createInternalContext()
return makeUniqueRef<GraphicsContextCG>(context.get());
}

DrawGlyphsRecorder::DrawGlyphsRecorder(GraphicsContext& owner, DrawGlyphsDeconstruction drawGlyphsDeconstruction)
DrawGlyphsRecorder::DrawGlyphsRecorder(GraphicsContext& owner, DeconstructDrawGlyphs deconstructDrawGlyphs, DeriveFontFromContext deriveFontFromContext)
: m_owner(owner)
, m_drawGlyphsDeconstruction(drawGlyphsDeconstruction)
, m_deconstructDrawGlyphs(deconstructDrawGlyphs)
, m_deriveFontFromContext(deriveFontFromContext)
, m_internalContext(createInternalContext())
{
}
@@ -289,11 +291,13 @@ static Vector<CGSize> computeAdvancesFromPositions(const CGPoint positions[], si

void DrawGlyphsRecorder::recordDrawGlyphs(CGRenderingStateRef, CGGStateRef gstate, const CGAffineTransform*, const CGGlyph glyphs[], const CGPoint positions[], size_t count)
{
ASSERT_IMPLIES(m_deriveFontFromContext == DeriveFontFromContext::No, m_originalFont);

if (!count)
return;

CGFontRef usedFont = CGGStateGetFont(gstate);
if (usedFont != adoptCF(CTFontCopyGraphicsFont(m_originalFont->platformData().ctFont(), nullptr)).get())
if (m_deriveFontFromContext == DeriveFontFromContext::No && usedFont != adoptCF(CTFontCopyGraphicsFont(m_originalFont->platformData().ctFont(), nullptr)).get())
return;

#if ASSERT_ENABLED
@@ -333,7 +337,9 @@ void DrawGlyphsRecorder::recordDrawGlyphs(CGRenderingStateRef, CGGStateRef gstat
updateStrokeColor(Color::createAndPreserveColorSpace(strokeColor));
updateShadow(CGGStateGetStyle(gstate));

m_owner.drawGlyphsAndCacheFont(*m_originalFont, glyphs, computeAdvancesFromPositions(positions, count, currentTextMatrix).data(), count, currentTextMatrix.mapPoint(positions[0]), m_smoothingMode);
auto fontSize = CGGStateGetFontSize(gstate);
Ref font = m_deriveFontFromContext == DeriveFontFromContext::No ? *m_originalFont : Font::create(FontPlatformData(adoptCF(CTFontCreateWithGraphicsFont(usedFont, fontSize, nullptr, nullptr)), fontSize));
m_owner.drawGlyphsAndCacheFont(font, glyphs, computeAdvancesFromPositions(positions, count, currentTextMatrix).data(), count, currentTextMatrix.mapPoint(positions[0]), m_smoothingMode);

m_owner.concatCTM(inverseCTMFixup);
}
@@ -410,12 +416,12 @@ static GlyphsAndAdvances filterOutOTSVGGlyphs(const Font& font, const GlyphBuffe

void DrawGlyphsRecorder::drawGlyphs(const Font& font, const GlyphBufferGlyph* glyphs, const GlyphBufferAdvance* advances, unsigned numGlyphs, const FloatPoint& startPoint, FontSmoothingMode smoothingMode)
{
if (m_drawGlyphsDeconstruction == DrawGlyphsDeconstruction::DontDeconstruct) {
if (m_deconstructDrawGlyphs == DeconstructDrawGlyphs::No) {
m_owner.drawGlyphsAndCacheFont(font, glyphs, advances, numGlyphs, startPoint, smoothingMode);
return;
}

ASSERT(m_drawGlyphsDeconstruction == DrawGlyphsDeconstruction::Deconstruct);
ASSERT(m_deconstructDrawGlyphs == DeconstructDrawGlyphs::Yes);

// FIXME: <rdar://problem/70166552> Record OTSVG glyphs.
GlyphsAndAdvances glyphsAndAdvancesWithoutOTSVGGlyphs = filterOutOTSVGGlyphs(font, glyphs, advances, numGlyphs);
@@ -427,4 +433,19 @@ void DrawGlyphsRecorder::drawGlyphs(const Font& font, const GlyphBufferGlyph* gl
concludeInternalContext();
}

void DrawGlyphsRecorder::drawNativeText(CTFontRef font, CGFloat fontSize, CTLineRef line, CGRect lineRect)
{
ASSERT(m_deconstructDrawGlyphs == DeconstructDrawGlyphs::Yes);

GraphicsContextStateSaver saver(m_owner);

m_owner.translate(lineRect.origin.x, lineRect.origin.y + lineRect.size.height);
m_owner.scale(FloatSize(1, -1));

prepareInternalContext(Font::create(FontPlatformData(font, fontSize)), FontSmoothingMode::SubpixelAntialiased);
CGContextSetTextPosition(m_internalContext->platformContext(), 0, 0);
CTLineDraw(line, m_internalContext->platformContext());
concludeInternalContext();
}

} // namespace WebCore
@@ -40,11 +40,11 @@
namespace WebCore {
namespace DisplayList {

Recorder::Recorder(DisplayList& displayList, const GraphicsContextState& state, const FloatRect& initialClip, const AffineTransform& initialCTM, Delegate* delegate, DrawGlyphsRecorder::DrawGlyphsDeconstruction drawGlyphsDeconstruction)
Recorder::Recorder(DisplayList& displayList, const GraphicsContextState& state, const FloatRect& initialClip, const AffineTransform& initialCTM, Delegate* delegate, DrawGlyphsRecorder::DeconstructDrawGlyphs deconstructDrawGlyphs)
: m_displayList(displayList)
, m_delegate(delegate)
, m_isNested(false)
, m_drawGlyphsRecorder(*this, drawGlyphsDeconstruction)
, m_drawGlyphsRecorder(*this, deconstructDrawGlyphs)
{
LOG_WITH_STREAM(DisplayLists, stream << "\nRecording with clip " << initialClip);
m_stateStack.append({ state, initialCTM, initialClip });
@@ -54,7 +54,7 @@ Recorder::Recorder(Recorder& parent, const GraphicsContextState& state, const Fl
: m_displayList(parent.m_displayList)
, m_delegate(parent.m_delegate)
, m_isNested(true)
, m_drawGlyphsRecorder(*this, parent.m_drawGlyphsRecorder.drawGlyphsDeconstruction())
, m_drawGlyphsRecorder(*this, parent.m_drawGlyphsRecorder.deconstructDrawGlyphs())
{
m_stateStack.append({ state, initialCTM, initialClip });
}
@@ -55,7 +55,7 @@ class Recorder : public GraphicsContext {
WTF_MAKE_NONCOPYABLE(Recorder);
public:
class Delegate;
WEBCORE_EXPORT Recorder(DisplayList&, const GraphicsContextState&, const FloatRect& initialClip, const AffineTransform&, Delegate* = nullptr, DrawGlyphsRecorder::DrawGlyphsDeconstruction = DrawGlyphsRecorder::DrawGlyphsDeconstruction::Deconstruct);
WEBCORE_EXPORT Recorder(DisplayList&, const GraphicsContextState&, const FloatRect& initialClip, const AffineTransform&, Delegate* = nullptr, DrawGlyphsRecorder::DeconstructDrawGlyphs = DrawGlyphsRecorder::DeconstructDrawGlyphs::Yes);
WEBCORE_EXPORT virtual ~Recorder();

WEBCORE_EXPORT void getPixelBuffer(const PixelBufferFormat& outputFormat, const IntRect& sourceRect);
@@ -32,7 +32,7 @@

namespace WebCore {

DrawGlyphsRecorder::DrawGlyphsRecorder(GraphicsContext& owner, DrawGlyphsDeconstruction)
DrawGlyphsRecorder::DrawGlyphsRecorder(GraphicsContext& owner, DeconstructDrawGlyphs, DeriveFontFromContext)
: m_owner(owner)
{
}
@@ -32,7 +32,7 @@

namespace WebCore {

DrawGlyphsRecorder::DrawGlyphsRecorder(GraphicsContext& owner, DrawGlyphsDeconstruction)
DrawGlyphsRecorder::DrawGlyphsRecorder(GraphicsContext& owner, DeconstructDrawGlyphs, DeriveFontFromContext)
: m_owner(owner)
{
}
@@ -37,6 +37,7 @@
#import "ColorIOS.h"
#import "DateComponents.h"
#import "Document.h"
#import "DrawGlyphsRecorder.h"
#import "File.h"
#import "FloatRoundedRect.h"
#import "FontCache.h"
@@ -1536,6 +1537,7 @@ static CGFloat attachmentDynamicTypeScaleFactor()
struct LabelLine {
FloatRect rect;
RetainPtr<CTLineRef> line;
RetainPtr<CTFontRef> font;
};
Vector<LabelLine> lines;

@@ -1545,10 +1547,10 @@ static CGFloat attachmentDynamicTypeScaleFactor()
void buildWrappedLines(const String&, CTFontRef, UIColor *, unsigned maximumLineCount);
void buildSingleLine(const String&, CTFontRef, UIColor *);

void addLine(CTLineRef);
void addLine(CTFontRef, CTLineRef);
};

void RenderAttachmentInfo::addLine(CTLineRef line)
void RenderAttachmentInfo::addLine(CTFontRef font, CTLineRef line)
{
CGRect lineBounds = CTLineGetBoundsWithOptions(line, kCTLineBoundsExcludeTypographicLeading);
CGFloat trailingWhitespaceWidth = CTLineGetTrailingWhitespaceWidth(line);
@@ -1557,6 +1559,7 @@ static CGFloat attachmentDynamicTypeScaleFactor()

CGFloat xOffset = (attachmentRect.width() / 2) - (lineWidthIgnoringTrailingWhitespace / 2);
LabelLine labelLine;
labelLine.font = font;
labelLine.line = line;
labelLine.rect = FloatRect(xOffset, 0, lineWidthIgnoringTrailingWhitespace, lineHeight);

@@ -1591,7 +1594,7 @@ static CGFloat attachmentDynamicTypeScaleFactor()
CFIndex lineIndex = 0;
CFIndex nonTruncatedLineCount = std::min<CFIndex>(maximumLineCount - 1, lineCount);
for (; lineIndex < nonTruncatedLineCount; ++lineIndex)
addLine((CTLineRef)CFArrayGetValueAtIndex(ctLines, lineIndex));
addLine(font, (CTLineRef)CFArrayGetValueAtIndex(ctLines, lineIndex));

if (lineIndex == lineCount)
return;
@@ -1611,7 +1614,7 @@ static CGFloat attachmentDynamicTypeScaleFactor()
if (!truncatedLine)
truncatedLine = remainingLine;

addLine(truncatedLine.get());
addLine(font, truncatedLine.get());
}

void RenderAttachmentInfo::buildSingleLine(const String& text, CTFontRef font, UIColor *color)
@@ -1625,7 +1628,7 @@ static CGFloat attachmentDynamicTypeScaleFactor()
};
RetainPtr<NSAttributedString> attributedText = adoptNS([[NSAttributedString alloc] initWithString:text attributes:textAttributes]);

addLine(adoptCF(CTLineCreateWithAttributedString((CFAttributedStringRef)attributedText.get())).get());
addLine(font, adoptCF(CTLineCreateWithAttributedString((CFAttributedStringRef)attributedText.get())).get());
}

static BOOL getAttachmentProgress(const RenderAttachment& attachment, float& progress)
@@ -1766,15 +1769,10 @@ static void paintAttachmentIcon(GraphicsContext& context, RenderAttachmentInfo&

static void paintAttachmentText(GraphicsContext& context, RenderAttachmentInfo& info)
{
for (const auto& line : info.lines) {
GraphicsContextStateSaver saver(context);
DrawGlyphsRecorder recorder(context, DrawGlyphsRecorder::DeconstructDrawGlyphs::Yes, DrawGlyphsRecorder::DeriveFontFromContext::Yes);

context.translate(toFloatSize(line.rect.minXMaxYCorner()));
context.scale(FloatSize(1, -1));

CGContextSetTextPosition(context.platformContext(), 0, 0);
CTLineDraw(line.line.get(), context.platformContext());
}
for (const auto& line : info.lines)
recorder.drawNativeText(line.font.get(), CTFontGetSize(line.font.get()), line.line.get(), line.rect);
}

static void paintAttachmentProgress(GraphicsContext& context, RenderAttachmentInfo& info)

0 comments on commit 977741f

Please sign in to comment.