Fix grapheme-vs-UTF-16 range bug in RichContentFormatter

[jkmassel]

Description

RichContentFormatter builds the NSRange it hands to NSRegularExpression from content.count — Swift's grapheme cluster count — but NSRegularExpression matches over UTF‑16. When content contains multi-code-unit characters (emoji, flags, ZWJ/combining sequences) the grapheme count is shorter than the UTF‑16 length, so the search range is truncated at the end and any forbidden tag or inline style near the end silently escapes stripping — the unsanitized markup then reaches the rendered Reader post or comment.

Concretely: a single family emoji 👨‍👩‍👧‍👦 is one grapheme but eleven UTF‑16 units, so a <script> placed right after it starts ten units past where the grapheme-count range ends — the whole tag falls outside the search and survives.

removeTrailingBreakTags carried a second form of the same confusion — it fed a UTF‑16 match offset to String.index(_:offsetBy:), which counts graphemes. That was already wrong for multi-code-unit content, and would have become a hard crash (String index is out of bounds) the moment the range was widened, so the two are fixed together.

Fix

• Size every search range from content.utf16.count instead of content.count — 12 sites across removeForbiddenTags, normalizeParagraphs, filterNewLines, removeInlineStyles, and removeTrailingBreakTags.
• Convert the trailing-BR match to a String range with Range(match.range, in: content) rather than a grapheme offset.

No behavior change on ASCII content — there the grapheme count equals the UTF‑16 length, which is why this went unnoticed. parseValueForAttribute, formatGutenbergGallery, and formatVideoTags already ranged over NSString.length and are untouched.

Tests

One isolated test per fix site — each forbidden-tag regex, each normalizeParagraphs sub-site, the three filterNewLines paths, the inline-style site, and the trailing-break range plus its index-offset cut — using a spread of multi-code-unit clusters (astral emoji, a ZWJ family, a flag, a keycap, a skin-tone modifier, and an NFD combining mark) positioned so the target token lands in the tail the grapheme-count range truncated. Each input holds only one tag type, so reverting any single site breaks exactly one test (confirmed by reverting each of the 13 sites individually); the exact-output assertions also confirm the cluster survives byte-for-byte. Two further tests pin the exact off-by-one boundary — a token whose closing delimiter is the single UTF‑16 unit the grapheme-count range drops — and confirm the corrected range strips the intended tag rather than everything (a style attribute in range plus one in the tail; only the tail one was surviving).

Verified by generating the set into a scratch suite and running it against both the buggy base source and the fixed HEAD:

• Fixed HEAD: Executed 23 tests, with 0 failures.
• Buggy base: 15 of the 16 new tests fail — the 13 site-guards plus the two boundary/selectivity tests — while the one idempotence guard (clean multibyte content, no tags) passes both, by design.

Stacking

Stacked on #25832 (base jkmassel/wordpressdata-swift-test). That PR relocates RichContentFormatter into cross-platform WordPressShared and makes RichContentFormatterTests run on macOS under swift test, which is what lets this fix and its regression tests live in that cross-platform set. The bug is pre-existing — it's also on trunk, in the WordPressSharedUI copy. Once #25832 merges, this PR retargets to trunk.

Testing instructions

• swift test --filter RichContentFormatterTests (macOS host) — 23/23 pass. 15 of the 16 new tests fail on the pre-fix base source — the 13 site-guards (each with its one tag left unstripped) plus the two boundary/selectivity tests — and all pass with the fix; the idempotence guard passes both.

[dangermattic]

	1 Message
📖	This PR is still a Draft: some checks will be skipped.

Generated by 🚫 Danger

[wpmobilebot]

📲 You can test the changes from this Pull Request in WordPress by scanning the QR code below to install the corresponding build.

	App Name	WordPress
	Configuration	Release-Alpha
	Build Number	33471
	Version	PR #25833
	Bundle ID	org.wordpress.alpha
	Commit	ea3df4b
	Installation URL	399rqo9jjpgqo

Automatticians: You can use our internal self-serve MC tool to give yourself access to those builds if needed.

[wpmobilebot]

📲 You can test the changes from this Pull Request in Jetpack by scanning the QR code below to install the corresponding build.

	App Name	Jetpack
	Configuration	Release-Alpha
	Build Number	33471
	Version	PR #25833
	Bundle ID	com.jetpack.alpha
	Commit	ea3df4b
	Installation URL	6lrbd8cocbo9g

Automatticians: You can use our internal self-serve MC tool to give yourself access to those builds if needed.