Extend Canvas TextMetrics for editing and text styling #11000

AndresRPerez12 · 2025-02-05T15:09:52Z

This change adds the following functionalities to TextMetrics objects:

Get the set of rectangles that the user agent would render as the selection background when a particular character range of the measured text is selected.
Get the rectangle corresponding to the actual bounding box for a character range of the measured text. This is the equivalent to TextMetrics.actualBoundingBox restricted to the given range.
Get the string index for the character at a given offset distance from the start position of the measured text.
Split a character range of the measured text into grapheme clusters. That is, for the given range, it returns the minimal logical units of text, each of which can be rendered, along with their corresponding positional data.

Additionally, a new method is added to the Canvas 2D rendering context to render these clusters. When rendering, all relevant CanvasTextDrawingStyles are used as they were when the text was measured.

See the discussion in the issue: #10677

At least two implementers are interested (and none opposed):
- Chromium: The new methods have been implemented (https://issues.chromium.org/u/1/issues/341213359)
- Gecko: Position has been requested ([HTML Canvas] TextMetrics Additions mozilla/standards-positions#1144)
- WebKit: Position has been requested, and some portions have already started landing ([HTML Canvas] TextMetrics Additions for Editing and Graphic Design WebKit/standards-positions#436)
Tests are written and can be reviewed and commented upon at:
- https://github.com/web-platform-tests/wpt/tree/master/html/canvas/element/text
- https://github.com/web-platform-tests/wpt/tree/master/html/canvas/offscreen/text
- Relevant tests: 2d.text.measure.selection-rects*, 2d.text.measure.getActualBoundingBox*, 2d.text.measure.index-from-offset*, and 2d.text.measure.text-clusters*.
Implementation bugs are filed:
- Chromium: https://crbug.com/341213359
- Gecko: …
- WebKit: https://bugs.webkit.org/show_bug.cgi?id=284464
Corresponding HTML AAM & ARIA in HTML issues & PRs:
MDN issue is filed: …
The top of this comment includes a clear commit message to use.

(See WHATWG Working Mode: Changes for more details.)

/acknowledgements.html ( diff )
/canvas.html ( diff )
/infrastructure.html ( diff )

source

AndresRPerez12 · 2025-02-05T16:26:15Z

Tagging: @whatwg/canvas

schenney-chromium

Sorry for leaving some comments out of the review.

source

schenney-chromium · 2025-02-05T16:30:17Z

source

+
+      <dt><dfn attribute for="TextCluster"><code data-x="dom-TextCluster-end">end</code></dfn> attribute</dt>
+
+      <dd><p>The ending index for the range of <span data-x="code point">code points</span> that are


Is this the index one past the end? Range APIs seem to behave that way but I'm not sure how conventional it is.

Yes, it is one past the end. I hadn't realized I didn't clarify it in the spec. I'll add it explicitly as I don't know how conventional it is either.

schenney-chromium · 2025-02-05T16:34:21Z

source

+    <p class="note">The user agent might internally store in the <code>TextCluster</code> object the
+    complete text, as well as all the <code>CanvasTextDrawingStyles</code> at the time that <code
+    data-x="dom-context-2d-measureText">measureText()</code> was called, as they will be needed to
+    correctly render the clusters as they were measured.</p>


Is it a requirement that the drawing command uses the same parameters as the measurement? If so I would say that directly here, because right now it seems suggestive but not prescribed.

Yes, that is specified in the section for fillTextCluster(). The idea when I first wrote it was to just mention the behavior expected, even if the implementation would most likely be to store the parameters in the cluster (as is implemented right now in Chrome).

But, following a suggestion by @Kaiido in another comment, I changed the phrasing of that requirement by using TextCluster as an opaque object that stores those parameters in cluster when they are created. Hopefully this is more clear. Let me know what you think!

Kaiido

Thanks for doing this!

A couple of questions that came to mind:

Should there be a strokeTextCluster() method akin to strokeText()?
Should the TextCluster be serializable? (so that we can send it to/from a Worker).

Also, I didn't receive the @whatwg/canvas notification from your previous comment. Hopefully this one will work better.

Kaiido · 2025-02-06T05:20:43Z

source

+  The cluster is rendered where it would be if the whole text that was passed to <code
+  data-x="dom-context-2d-measureText">measureText()</code> to obtain the cluster was rendered at


Disclaimer: I'm not an editor, so take this with a grain of salt.

I feel this would be a lot clearer if the TextMetrics interface had an internal value to store the original text for each instance and reuse it where needed, instead of this tedious "the text that was passed to measureText() to obtain the cluster".

See for instance how CanvasPattern objects have a transformation matrix that isn't exposed:

Patterns have a transformation matrix, which controls how the pattern is used when it is painted. Initially, a pattern's transformation matrix must be the identity matrix.

It might even be needed to have a clear relationship established between the TextCluster and the TextMetrics from where it's been created, so that you can do something like the cluster's textmetrics's text, or, given that when the TextCluster is created text is available, and TextMetrics may not need it otherwise, you could maybe store it directly along the TextCluster (still in an internal value).

I hadn't noticed that objects like CanvasPattern were defined in this way as opaque. I like that concept and I feel it applies well to this use case. Thank you so much for the suggestion!

I have updated it to reflect this. For now I went with the second approach you mentioned (storing the text and the CanvasTextDrawingStyles directly in the cluster as internal values). Hopefully this provides more clarity. Let me know what you think!

Kaiido · 2025-02-06T05:35:21Z

source

+    <li><p>Run the <span>text preparation algorithm</span>, passing it the <var>text</var>
+    originally passed to <code data-x="dom-context-2d-measureText">measureText()</code> to obtain
+    the cluster, and an object with the <code>CanvasTextDrawingStyles</code> values as they were
+    when the cluster was measured, with the exception of <code


This object should probably be created explicitly sooner, in the getTextClusters() method. However I'm not sure how this should be done since we want the computed values at that time.

With the use of the internal values of the opaque object, hopefully the relationship is clearer now. Passing the exact object from the cluster won't always be possible due to the possibility of passing align and baseline to to fillTextCluster() when rendering, but most of the values now come from the internally stored value.

source

AndresRPerez12 · 2025-02-12T20:31:05Z

Thank you both for your comments! I have addressed them and uploaded the changes. Please let me know what you think.

Answering some of the open questions:

Should there be a strokeTextCluster() method akin to strokeText()?

I have added it to the PR. I agree that it makes sense to have both.

Should the TextCluster be serializable? (so that we can send it to/from a Worker).

We feel that for now, it's simpler to not make it serializable. Once we merge this and we know exactly what clusters look like and what values they are storing internally that would require serialization, we can look into it.

Also, I didn't receive the @whatwg/canvas notification from your previous comment. Hopefully this one will work better.

Thanks! I'm not fully sure how the tags work, so I really appreciate it!

annevk · 2025-02-24T13:39:58Z

source

@@ -4165,6 +4165,7 @@ a.setAttribute('href', 'https://example.com/'); // change the content attribute
         <dfn data-x="DOMPointInit-x" data-xref="https://drafts.fxtf.org/geometry/#dom-dompointinit-x">x</dfn> and
         <dfn data-x="DOMPointInit-y" data-xref="https://drafts.fxtf.org/geometry/#dom-dompointinit-y">y</dfn> members</li>
      <li><dfn data-x-href="https://drafts.fxtf.org/geometry/#matrix-multiply">Matrix multiplication</dfn></li>
+      <li><dfn data-x-href="https://www.w3.org/TR/geometry-1/#DOMRect">DOMRectReadOnly and DOMRect</dfn></li>


Each needs its own reference.

I added both since the section on W3C is named DOMRect, but since I’m only using DOMRectReadOnly I will just leave that one pointing to its definition.

Please do not use TR links; use https://drafts.fxtf.org/geometry/ .

And objects should have <code> as well.

annevk · 2025-02-24T13:42:10Z

source

+  <p>The <dfn method for="CanvasText"><code
+  data-x="dom-context-2d-fillTextCluster">fillTextCluster(<var>cluster</var>, <var>x</var>,
+  <var>y</var>, <var>options</var>)</code></dfn> and <dfn method for="CanvasText"><code
+  data-x="dom-context-2d-strokeTextCluster">strokeTextCluster(<var>cluster</var>, <var>x</var>,
+  <var>y</var>, <var>options</var>)</code></dfn> methods render a <code>TextCluster</code> object.
+  The cluster is rendered where it would be if the whole text that was passed to <code
+  data-x="dom-context-2d-measureText">measureText()</code> to obtain the cluster was rendered at
+  position (<var>x</var>,<var>y</var>), unless the positioning is modified with the options
+  argument. Specifically, when the method is invoked, the user agent must run these steps:</p>


These need to be rephrased to use the modern way of defining methods. "The X method steps are ..."

Done, although I kept the structure of bundling the fill and stroke version together.

If this version is what is desired, I can create a PR moving fillText() and strokeText() to this modern format once this PR lands.

annevk · 2025-02-24T13:42:31Z

source

+  <ol>
+    <li><p>If any of the arguments are infinite or NaN, then return.</p></li>


Indentation and formatting here doesn't appear to follow the guidelines.

Done, thanks for pointing it out!

annevk · 2025-02-24T13:44:16Z

source

@@ -69327,6 +69456,125 @@ try {
   positive numbers indicating that the given baseline is below the <span>ideographic-under
   baseline</span>. (Zero if the given baseline is the <span>ideographic-under
   baseline</span>.)</p></dd>
+
+   <dt><dfn method for="TextMetrics"><code data-x="dom-textmetrics-getSelectionRects">getSelectionRects(<var>start</var>, <var>end</var>)</code></dfn> method</dt>


This should not follow the bad precedent set of the current set of attributes. Please define these as modern methods.

I had to assign some variables from the result of the text preparation algorithm being run when measureText() was called. Let me know what you think of this please.

I don't understand how this relates to the concern?

annevk · 2025-02-24T13:46:38Z

source

+    // opaque object
+    readonly attribute double <span data-x="dom-TextCluster-x">x</span>;
+    readonly attribute double <span data-x="dom-TextCluster-y">y</span>;
+    readonly attribute unsigned long <span data-x="dom-TextCluster-begin">begin</span>;


Sometimes you use "begin" and sometimes "start". That should be made consistent.

Done, I settled with start. Thanks for pointing it out!

annevk · 2025-02-24T13:47:22Z

source

+
+[Exposed=(Window,Worker)]
+interface <dfn interface>TextCluster</dfn> {
+    // opaque object


I'm not sure what this comment buys us.

I wanted to show that TextCluster objects have hidden valued stored inside that are defined when creating the cluster, and used when rendering it. I added that comment since I saw it on CanvasGradient and CanvasPattern, that have this kind of hidden attributes. Should I express this in a different way?

I see, if it's consistent with CanvasPattern we can keep it I suppose.

This CL creates a new method for CanvasRenderingContext2d that strokes a TextCluster object. Its fill counterpart, ctx.fillTextCluster(), was merged in https://crrev.com/c/5854270. This new method came to be from the discussion around the spec for the TextMetrics additions: whatwg/html#11000 The existing tests for fillTextCluster() were renamed to include the method name in the title, since before only "rendering" was mentioned. The new tests are explicitly named using strokeTextCluster(). Bug: 341213359 Change-Id: Icd792b56e48f3ea5091e958671c19fcefe67fdec

This CL creates a new method for CanvasRenderingContext2d that strokes a TextCluster object. Its fill counterpart, ctx.fillTextCluster(), was merged in https://crrev.com/c/5854270. This new method came to be from the discussion around the spec for the TextMetrics additions: whatwg/html#11000 The existing tests for fillTextCluster() were renamed to include the method name in the title, since before only "rendering" was mentioned. The new tests are explicitly named using strokeTextCluster(). A line that was duplicated on the tests for fillTextCluster() was also removed. Bug: 341213359 Change-Id: Icd792b56e48f3ea5091e958671c19fcefe67fdec

This CL creates a new method for CanvasRenderingContext2d that strokes a TextCluster object. Its fill counterpart, ctx.fillTextCluster(), was merged in https://crrev.com/c/5854270. This new method came to be from the discussion around the spec for the TextMetrics additions: whatwg/html#11000 The existing tests for fillTextCluster() were renamed to include the method name in the title, since before only "rendering" was mentioned. The new tests are explicitly named using strokeTextCluster(). A line that was duplicated on the tests for fillTextCluster() was also removed. Bug: 341213359 Change-Id: Icd792b56e48f3ea5091e958671c19fcefe67fdec Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/6282180 Commit-Queue: Andres Ricardo Perez <andresrperez@chromium.org> Reviewed-by: Fernando Serboncini <fserb@chromium.org> Reviewed-by: Stephen Chenney <schenney@chromium.org> Cr-Commit-Position: refs/heads/main@{#1425171}

…tonly Automatic update from web-platform-tests Implement ctx.strokeTextCluster() This CL creates a new method for CanvasRenderingContext2d that strokes a TextCluster object. Its fill counterpart, ctx.fillTextCluster(), was merged in https://crrev.com/c/5854270. This new method came to be from the discussion around the spec for the TextMetrics additions: whatwg/html#11000 The existing tests for fillTextCluster() were renamed to include the method name in the title, since before only "rendering" was mentioned. The new tests are explicitly named using strokeTextCluster(). A line that was duplicated on the tests for fillTextCluster() was also removed. Bug: 341213359 Change-Id: Icd792b56e48f3ea5091e958671c19fcefe67fdec Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/6282180 Commit-Queue: Andres Ricardo Perez <andresrperez@chromium.org> Reviewed-by: Fernando Serboncini <fserb@chromium.org> Reviewed-by: Stephen Chenney <schenney@chromium.org> Cr-Commit-Position: refs/heads/main@{#1425171} -- wpt-commits: e515834325719ce9e2a15e2d41bfce49f020db7a wpt-pr: 50942

…tonly Automatic update from web-platform-tests Implement ctx.strokeTextCluster() This CL creates a new method for CanvasRenderingContext2d that strokes a TextCluster object. Its fill counterpart, ctx.fillTextCluster(), was merged in https://crrev.com/c/5854270. This new method came to be from the discussion around the spec for the TextMetrics additions: whatwg/html#11000 The existing tests for fillTextCluster() were renamed to include the method name in the title, since before only "rendering" was mentioned. The new tests are explicitly named using strokeTextCluster(). A line that was duplicated on the tests for fillTextCluster() was also removed. Bug: 341213359 Change-Id: Icd792b56e48f3ea5091e958671c19fcefe67fdec Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/6282180 Commit-Queue: Andres Ricardo Perez <andresrperezchromium.org> Reviewed-by: Fernando Serboncini <fserbchromium.org> Reviewed-by: Stephen Chenney <schenneychromium.org> Cr-Commit-Position: refs/heads/main{#1425171} -- wpt-commits: e515834325719ce9e2a15e2d41bfce49f020db7a wpt-pr: 50942 UltraBlame original commit: c4d5d718989db2d1c940af46af2b61bd0b938b97

…tonly Automatic update from web-platform-tests Implement ctx.strokeTextCluster() This CL creates a new method for CanvasRenderingContext2d that strokes a TextCluster object. Its fill counterpart, ctx.fillTextCluster(), was merged in https://crrev.com/c/5854270. This new method came to be from the discussion around the spec for the TextMetrics additions: whatwg/html#11000 The existing tests for fillTextCluster() were renamed to include the method name in the title, since before only "rendering" was mentioned. The new tests are explicitly named using strokeTextCluster(). A line that was duplicated on the tests for fillTextCluster() was also removed. Bug: 341213359 Change-Id: Icd792b56e48f3ea5091e958671c19fcefe67fdec Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/6282180 Commit-Queue: Andres Ricardo Perez <andresrperez@chromium.org> Reviewed-by: Fernando Serboncini <fserb@chromium.org> Reviewed-by: Stephen Chenney <schenney@chromium.org> Cr-Commit-Position: refs/heads/main@{#1425171} -- wpt-commits: e515834325719ce9e2a15e2d41bfce49f020db7a wpt-pr: 50942

annevk · 2025-03-07T11:22:35Z

source

+
+[Exposed=(Window,Worker)]
+interface <dfn interface>TextCluster</dfn> {
+    // opaque object


I see, if it's consistent with CanvasPattern we can keep it I suppose.

annevk · 2025-03-07T11:23:20Z

source

+  <span>CanvasTextAlign</span> align;
+  <span>CanvasTextBaseline</span> baseline;
+  double x;
+  double y;


These members should probably be <dfn> so they can be referenced from prose.

annevk · 2025-03-07T11:24:27Z

source

+  data-x="dom-context-2d-fillTextCluster">fillTextCluster(<var>cluster</var>, <var>x</var>,
+  <var>y</var>, <var>options</var>)</code></dfn> and <dfn method for="CanvasText"><code
+  data-x="dom-context-2d-strokeTextCluster">strokeTextCluster(<var>cluster</var>, <var>x</var>,
+  <var>y</var>, <var>options</var>)</code></dfn> methods, when invoked, must run these steps:</p>


"The X method steps are:" is the modern style.

annevk · 2025-03-07T11:25:03Z

source

+  <var>y</var>, <var>options</var>)</code></dfn> methods, when invoked, must run these steps:</p>
+
+  <ol>
+   <li><p>If any of the arguments are infinite or NaN, then return.</p></li>


Given that these also take a dictionary, I think we should be more explicit here.

annevk · 2025-03-07T11:26:41Z

source

+
+   <li><p>Run the <span>text preparation algorithm</span>, passing it the complete text and the
+   <code>CanvasTextDrawingStyles</code> from the opaque <code>TextCluster</code>
+   <var>cluster</var>, with the exception of <code


This exception also refers to the cluster. So I don't really follow this.

This is also talking about public API whereas we shouldn't be touching that here.

annevk · 2025-03-07T11:28:38Z

source

 interface mixin <dfn interface>CanvasText</dfn> {
  // text (see also the <span>CanvasPathDrawingStyles</span> and <span>CanvasTextDrawingStyles</span> interfaces)
  undefined <span data-x="dom-context-2d-fillText">fillText</span>(DOMString text, unrestricted double x, unrestricted double y, optional unrestricted double maxWidth);
  undefined <span data-x="dom-context-2d-strokeText">strokeText</span>(DOMString text, unrestricted double x, unrestricted double y, optional unrestricted double maxWidth);
+  undefined <span data-x="dom-context-2d-fillTextCluster">fillTextCluster</span>(<span>TextCluster</span> cluster, double x, double y, optional <span>TextClusterOptions</span> options);


Optional dictionaries must default to = {}

annevk · 2025-03-07T11:29:56Z

source

+   <li>
+    <p>If a <code>TextClusterOptions</code> options dictionary is passed and it has an <code
+    data-x="">options["x"]</code> value, move all the shapes in <var>glyphs</var> to the right by
+    <code data-x="">options["x"] &minus; cluster["x"]</code>.</p>


The dictionary is always passed (see other comment). But this language and markup needs updating. I recommend looking at something recent, such as TogglePopoverOptions.

annevk · 2025-03-07T11:31:05Z

source

+     style</span> must be applied to the result of <span data-x="trace a path">tracing</span> the
+     shapes using the object implementing the <code>CanvasText</code> interface for the line
+     styles, and <span>this</span>'s <span data-x="concept-CanvasFillStrokeStyles-fill-style">fill
+     style</span> must be ignored.</p>


I think it would be cleaner if there was a shared algorithm that was passed either "fill" or "stroke" or some such as an internal enum value that you could branch on here.

annevk · 2025-03-07T11:32:22Z

source

@@ -69327,6 +69456,125 @@ try {
   positive numbers indicating that the given baseline is below the <span>ideographic-under
   baseline</span>. (Zero if the given baseline is the <span>ideographic-under
   baseline</span>.)</p></dd>
+
+   <dt><dfn method for="TextMetrics"><code data-x="dom-textmetrics-getSelectionRects">getSelectionRects(<var>start</var>, <var>end</var>)</code></dfn> method</dt>


I don't understand how this relates to the concern?

annevk · 2025-03-07T11:32:52Z

source

@@ -69339,8 +69609,6 @@ try {
  documents, rendered using CSS, straight to the canvas. This would be provided in preference to a
  dedicated way of doing multiline layout.</p>

-
-


Probably best not to remove those here.

Initial commit: TextMetrics additions

20e36f8

AndresRPerez12 marked this pull request as draft February 5, 2025 15:10

schenney-chromium reviewed Feb 5, 2025

View reviewed changes

source Show resolved Hide resolved

schenney-chromium reviewed Feb 5, 2025

View reviewed changes

source Show resolved Hide resolved

schenney-chromium reviewed Feb 5, 2025

View reviewed changes

source Show resolved Hide resolved

schenney-chromium reviewed Feb 5, 2025

View reviewed changes

Kaiido reviewed Feb 6, 2025

View reviewed changes

Kaiido added the topic: canvas label Feb 6, 2025

past mentioned this pull request Feb 7, 2025

Upcoming WHATNOT meeting on 2025-02-06 #10972

Closed

AndresRPerez12 added 5 commits February 12, 2025 12:03

Group similar calls together, fix typos, and use infra maps

370fb3c

Clarify index inclusion for ranges

1f94348

Add strokeTextCluster()

1b9e735

Make TextCluster an opaque object

9cab38c

Fix indentantion around getTextClusters

85c06f1

AndresRPerez12 added 2 commits February 14, 2025 18:36

Add description for new methods in intro section

b6a2c7a

Clarify cluster definition

6e08c93

AndresRPerez12 marked this pull request as ready for review February 18, 2025 21:09

annevk reviewed Feb 24, 2025

View reviewed changes

chromium-wpt-export-bot mentioned this pull request Feb 25, 2025

Implement ctx.strokeTextCluster() web-platform-tests/wpt#50942

Merged

AndresRPerez12 added 2 commits February 28, 2025 15:14

Formatting and DOMRect references

a20a528

Modern phrasing for fill/strokeTextCluster()

b286aa3

Rename TextCluster begin attribute to start

8062ab0

Modern phrasing for getSelectionRects() and getActualBoundingBox()

1f8aa8f

annevk reviewed Mar 7, 2025

View reviewed changes


		<dt><dfn attribute for="TextCluster"><code data-x="dom-TextCluster-end">end</code></dfn> attribute</dt>

		<dd><p>The ending index for the range of <span data-x="code point">code points</span> that are

		The cluster is rendered where it would be if the whole text that was passed to <code
		data-x="dom-context-2d-measureText">measureText()</code> to obtain the cluster was rendered at

		<ol>
		<li><p>If any of the arguments are infinite or NaN, then return.</p></li>

		@@ -69339,8 +69609,6 @@ try {
		documents, rendered using CSS, straight to the canvas. This would be provided in preference to a
		dedicated way of doing multiline layout.</p>

Extend Canvas TextMetrics for editing and text styling #11000

Are you sure you want to change the base?

Extend Canvas TextMetrics for editing and text styling #11000

Conversation

AndresRPerez12 commented Feb 5, 2025 • edited by pr-preview bot Loading

AndresRPerez12 commented Feb 5, 2025

schenney-chromium left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Kaiido left a comment

Choose a reason for hiding this comment

Kaiido Feb 6, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

AndresRPerez12 commented Feb 12, 2025

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

AndresRPerez12 commented Feb 5, 2025 •

edited by pr-preview bot

Loading

Kaiido Feb 6, 2025 •

edited

Loading