Skip to content

Text Rendering (Demo 1)

Jump to bottom
Doug Moen edited this page Aug 25, 2020 · 26 revisions

Text rendering for the Demo 1 release.

Goals:

  • Truetype fonts (lines and quadratric Beziers), but not OpenType fonts with cubic beziers
  • ASCII, but not the full complexity of Unicode font rendering
  • Scaleable, from unreadably small, to huge, suitable for a Zoomable User Interface (ZUI). Large amounts of text can be rendered quickly. We can reproduce this: http://wdobbie.com/warandpeace/
  • A fixed set of SDF operators that are useful with text. Eg:
    • Vary font weight from light to bold
    • Rounded and mitred outlines
    • Glows and drop shadows
    • Affine and non-affine transformations
      • Eg, fisheye lens transformation (might be useful for a ZUI)

An ideal text rendering implementation has:

  1. Flexibility: Full support for SDF semantics. Exact and mitred distance fields, accurate in a relatively wide region around the glyph. Gradients.
  2. Performance: High rendering speed. Fully dynamic content (eg, glyphs can change in real time). Low latency: avoids expensive preprocessing on the CPU before text rendering can begin.
    • Raph Levien mentions a specific use case for dynamic content: "variable fonts, and especially the ability to vary the parameters dynamically, either as an animation or microtypography. Such applications are not compatible with precomputation and require a more dynamic pipeline."
  3. Quality: High quality at all zoom scales: accurate outlines with sharp corners, subpixel antialiasing.
  4. Portability: Runs on all currently used hardware and operating environments.
  5. Simplicity: Simple code.

No real implementation can match this ideal. There are tradeoffs among the goals. For this project:

  • Flexibility is paramount. It's the main purpose of the Dali project, and it's what no mainstream graphics engine delivers right now.
  • Rendering speed is important. However, the expanded palette of special effects has a performance cost, relative to the fastest known text renderers for body text in an upright orthogonal view. The goal is not to match Piet's performance, but to be fast enough to support responsive, non-traditional user interfaces, for games, UI research, 3D modelling tools, etc, given that a modern GPU is a prerequisite for running Dali.
  • Quality is important. Anti-aliasing is important, but there are two features from the era before retina displays and compositing window managers that we don't care about: font hinting, and the type of sub-pixel rendering that takes into account the layout of R, G, B phosphors within a pixel. (Apple ignores font hinting on MacOS & iOS.)
  • We get portability to all operating environments (including to WASM and web browsers) by targeting WebGPU, at the cost of supporting only modern GPU hardware.
  • Simplicity is bought by only targetting generic WebGPU, at the expense of the performance that could be gained with multiple specialized backends targetting Vulkan, DX12 and Metal, with separate tuned parameter sets for each GPU vendor + architecture.

Design Concepts

The easy and obvious approach to GPU text rendering is to use SDF textures. However, it has limitations:

  • Quality issues. Conversion of glyph outline data to an SDF texture discards information, and you get rendering artifacts that are especially noticeable when text is zoomed in or otherwise subject to interesting transformations. The quality is not competitive to Slug and other libraries that render directly from outline data. MSDFgen is the best known solution for mitigating artifacts, but still isn't competitive.
  • "Full SDF semantics" includes having a choice between exact and mitred distance fields, as seen in "Real-Time, Texture-Mapped Vector Glyphs". Both options are needed for different special effects. MSDFgen gives approximations of both options when used in mtsdf mode.
  • It doesn't support fully dynamic content. Generating an MSDF font is expensive.
  • It doesn't work for the general case of rendering SVG-style outlines. I'm inspired by how Piet uses one general purpose mechanism for rendering both font glyphs and general vector-based outlines.
  • It doesn't demonstrate an improvement in text rendering technology for the first demo.

I do want to support SDF textures as one kind of primitive shape, in both 2D and 3D, but in a general purpose way, not in a form that is specialized to font rendering. I do want to support general 2D vector graphics, and I'm hoping that text rendering is a special case of this.

  • Flexibility:
    • Full SDF semantics, as seen in "Real-Time, Texture-Mapped Vector Glyphs".
    • Vector textures (random access vector graphics). Each glyph is represented by a data structure that efficiently maps [x,y] coordinates to the sample value (signed distance) at that coordinate. As seen in "Real-time texture-mapped vector glyphs" (vector textures), "Massively Parallel Vector Graphics" (shortcut trees), etc.
  • High performance on a modern GPU:
    • A pipeline of compute shaders, as seen in Piet, Spinel, etc.
    • Offload all intensive work to the GPU. Just load raw vector graphics into the GPU, and the conversion to an efficiently renderable data structure happens very quickly, as seen in Piet, Spinel, Slug, "Massively Parallel Vector Graphics", etc.
    • Tile based rendering. The frame buffer is partitioned into 16x16 pixel tiles. Only geometric primitives that intersect a tile are considered when rendering a tile. As seen in Piet, Spinel, etc.

Implementation Ideas

I know how to create an architecture that satisfies the Flexibility requirements, but I can only guess how to make it fast. That's pretty normal at this stage. Performance claims aren't real until you measure them, and you have to iterate the design to achieve high performance. Pathfinder has gone through 3 designs in the search for better performance, and Piet has gone through many iterations.

There are two kinds of performance optimizations.

  1. Make the entire architecture fast in the general case. Which I can attempt to do by looking at "best practices" gleaned from other projects.
  2. Optimize special cases, making those special cases faster than the general case.

My strategy will be to start with a general purpose architecture (flexibility over speed), and then look for performance improvements, which will have to be an iterative process.

  • "Real-time texture-mapped vector glyphs" is a general purpose architecture. Font glyphs are represented using vector textures, which are random access, mapping an [x,y] coordinate to a signed distance that is computed directly from the outline data.
  • RTTMVG has an expensive preprocessing step to create the vector texture, which happens on the CPU. "Massively Parallel Vector Graphics" has a much faster preprocessing stage, which happens on the CPU. The vector texture data structure is a quadtree, called a "shortcut tree", and maybe I can adapt this data structure to do signed distance lookups.

The evidence suggests that vector textures are slower than the fastest GPU text renderers (which only handle a special case of what Dali handles). I'm still looking for insights on how to make things faster in the general case. Speeding up special cases looks like an easier job.

Clone this wiki locally