Skip to content

Fix edge cases in glyph rasterization#67

Merged
mourner merged 1 commit into
mainfrom
mourner/fix-edge-cases
May 5, 2026
Merged

Fix edge cases in glyph rasterization#67
mourner merged 1 commit into
mainfrom
mourner/fix-edge-cases

Conversation

@mourner
Copy link
Copy Markdown
Member

@mourner mourner commented May 4, 2026

Fixes two bugs in glyph rasterization, both rooted in misinterpreting TextMetrics:

Fix #63: Inner EDT misses outside-ink seeds in the buffer region

The inner Euclidean distance transform was restricted to [buffer, buffer, glyphWidth, glyphHeight], which excluded the surrounding buffer pixels as seeds. When ink touches the bbox edge (common — Math.ceil(actualBoundingBoxAscent) puts the topmost ink row exactly at the bbox top), the nearest "outside" pixel sits one row outside the bbox and is invisible to the EDT, collapsing the inner distance and producing a hard cutoff (see #63's video).

Fix: pad the inner EDT region by 1 px on each side. The buffer region is uniformly seeded as outside-ink, so the closest external seed for any interior pixel is always exactly one pixel outside the bbox — 1 px of padding is provably sufficient. Padding is clamped to min(buffer, 1) to stay in-bounds when buffer = 0.

Fix #65: actualBoundingBoxLeft sign convention

Per the HTML spec, actualBoundingBoxLeft is positive when ink extends to the left of the origin. The old code treated it as positive=right, so glyphs whose ink sits to the right of origin (typical for letters like "P") were placed at the wrong position within the SDF texture, with extra phantom padding on one side.

Fix: negate when computing glyphLeft = Math.floor(-actualBoundingBoxLeft). This preserves glyphLeft's existing public semantics (canvas-x of ink left edge, positive=right), so all the downstream formulas (glyphWidth, fillText placement) keep working unchanged.

Downstream impact

The standalone SDF output changes — glyphs are now correctly placed within their texture without phantom padding. Consumers that position quads via glyphLeft (e.g. mapbox-gl-js's metrics.left in symbol/quads.ts) render pixel-identically: the old bug was self-cancelling because the wrong ink offset within the texture was compensated by the equally wrong glyphLeft value used for placement. Glyph atlas entries become slightly smaller for glyphs with non-zero actualBoundingBoxLeft (memory win).

Tests

  • Existing fixtures regenerated — old fixtures encoded the buggy ink placement.
  • does not return negative-width glyphs mock updated: the original metrics described a legitimate (if unusual) glyph under the corrected sign convention; flipping ABBLeft's sign restores the test's pathological-input intent.

Closes #63, closes #65. Supersedes #64 and #66.

@mourner mourner merged commit ed19979 into main May 5, 2026
5 checks passed
@mourner mourner deleted the mourner/fix-edge-cases branch May 5, 2026 07:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ai AI coding agents co-authored the code

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Incorrect actualBoundingBoxLeft with negative value Inner EDT restricted to glyph bounding box causes incorrect SDF values at glyph edges

2 participants