Skip to content

Photon

Jump to bottom
Edgar Mesquita edited this page Aug 7, 2026 · 12 revisions

Photon — The Native GPU Engine (Track N)

Photon is eQuantic.UI's proprietary GPU rendering engine for mobile (and eventually desktop): a from-scratch, Impeller-inspired renderer that draws the component tree pixel by pixel on the GPU — Metal on iOS/macOS, Vulkan on Android — with no Skia tier and no WebView. It is the second realization target of the write-once component architecture: the same C# components that lower to DOM+CSS on the web are rasterized by Photon natively.

The living plan is docs/NATIVE-GPU-ENGINE-PLAN.md in the main repo — vision, decisions (D1–D12), workstreams (W1–W8), milestones (M0–M5) and a dated status log. This page is the overview.

Why proprietary, why no Skia

The decision (2026-07): if the project was going to own a native tier at all, it would own the whole rendering stack — direct Metal/Vulkan backends with precompiled shaders, not a wrapper over Skia. That buys:

  • Predictable performance: fixed pipeline-state registry, zero shader compilation at runtime (the jank source Impeller was built to kill).
  • A tiny surface: UI rendering needs a handful of primitives, not a general 2D library.
  • One normative model: the same math, testable to the pixel, across CPU reference and every GPU backend.

Architecture at a glance

C# components (write-once, eQuantic.UI.Primitives vocabulary)
        │  Build(ComponentContext)          — pure, token-based, mode-free
        ▼
eQuantic.UI.Native.Framework                — C# flex layout engine (spec A2)
        ▼
eQuantic.UI.Native.Components               — PhotonRealizer: abstract tree → display list
        ▼
eQuantic.UI.Native.Engine                   — geometry, color model, Sdf.cs (NORMATIVE), DisplayList
        ▼
IRenderBackend
   ├── eQuantic.UI.Native.Engine.Reference  — scalar CPU rasterizer (golden ground truth, never shipped)
   ├── eQuantic.UI.Native.Engine.Metal      — Apple GPUs (offscreen spike landed)
   └── (Vulkan — planned)

The normative core

  • Sdf.cs is a spec, not a library: per-corner rounded-rect signed distance, centered stroke (|d| − w/2), 1-pixel anti-aliasing coverage ramp. Every rasterizer — the CPU reference and the GPU fragment shaders — implements exactly these formulas.
  • Color model: sRGB authoring → linear premultiplied blending → sRGB store, mirroring GPU hardware behavior with sRGB render targets.
  • Display lists: flat, heap-free DrawCommand records (Clear / FillRRect / StrokeRRect, solid or two-stop linear gradient paints, baked 2D transforms).

Golden-image harness

Every scene renders through a backend into a surface, reads back sRGB pixels, and compares against committed PNG goldens (dependency-free codec, EQ_UPDATE_GOLDENS=1 regeneration flow, ±2 tolerance, failure artifacts with amplified diffs). The scene catalog is shared: the Reference backend pins the goldens; GPU backends run the same catalog for parity.

Metal spike result (2026-07-04)

The first GPU frames validated the whole model: an offscreen Metal backend driven by ~100 lines of typed objc_msgSend P/Invoke (no binding framework, no C shims), one pipeline built at device init, runtime-compiled MSL that transliterates Sdf.cs. Measured against the CPU reference across all 14 golden scenes (fills, strokes, gradients, rotation/scale transforms, translucent blending, radius clamping):

Max channel difference: 1 (of 255). Zero pixels beyond ±2.

The GPU passes the golden harness's own tolerance — Sdf-as-spec and the color model hold end-to-end on hardware. Production shaders move to an offline Slang toolchain (single shader source → SPIR-V for Vulkan + metallib for Metal, embedded like the Bun binaries).

Component rendering today

The native realizer draws real component chrome (token-resolved fills, inside borders, per-corner radii) with documented placeholders where subsystems are pending:

Subsystem State (2026-07-31)
Shapes, borders, gradients, transforms, blending, clip ✅ Full, golden-tested
Layout (flex + wrap, grid, stacks, adaptive, truncation) ✅ C# engine
Shadows (elevation §05) ✅ Analytic ShadowCoverage, both backends
Text ✅ REAL — Texture primitive (A8 coverage × tint) + platform ITextRasterizer; macOS = CoreText (system frameworks only — no HarfBuzz/FreeType where the platform has a text engine). One engine measures AND rasters, so line breaks agree by construction. Two faces: the system's proportional one and its monospaced one (TypeStyle.Mono). No glyph is ever clipped — the bitmap is sized from the line's INK box (CTLineGetImageBounds, not the font's declared metrics: a deep g tail passes beyond them) and reports how far above the line box it had to reach, so the realizer raises the draw by exactly that and the line box still lands where layout put it
Icons ✅ REAL — pure-C# SVG path parser (full command set, arcs via F.6.5) + CoreGraphics rasterizer, same Texture primitive
Group opacity / transforms (S1) ✅ Engine layers (Reference exact; Metal per-command spike fence)
Scroll compositor ✅ Path-keyed offsets, real Sticky pinning (vertical)
Motion ✅ Loop motion, value transitions (B14), Presence enter/exit with command-snapshot replay
Gestures ✅ Hover pipeline, press with slop-cancel, drag-to-dismiss (follow + glide)
Anchored overlays ✅ Synthetic overlay layers positioned from absolute anchor bounds
Images SurfaceSubtle box until decode/upload (the remaining Texture consumer, A11/M4)

Interaction is wired: PhotonHost holds a retained root, SetState invalidates, taps dispatch to hit regions (topmost wins, disabled swallows; ≥48dp under a finger, the control's own size under a pointer — see Density) — the native Counter runs tap → SetState → rebuild end-to-end in tests. PhotonHost.FocusedPath reports what holds the keyboard, including a field being edited (the caret is that field's own focus indicator) — a platform accessibility bridge needs one honest answer, not the ring's half of it.

Layout: the CSS rules the engine owes you

The native engine is judged against the browser, because the web twin renders the same tree there:

  • Fill inherits indeterminacy — on an axis the parent sizes from content, a Fill child has nothing to fill and passes the question through; leftover is not distributed on such an axis.
  • Block semantics, width only — a box whose width is decided stretches an auto-sized CONTAINER child across it (a div's child div is full-width), while a button, link or input hugs there: they are inline-block, and only a FLEX stretch reaches through them (StretchKind).
  • flex-shrink with a min-content floor — a line that does not fit shrinks its items instead of overflowing, and never below each item's min-content (the longest word of a text, the sum of a row's children). Overflow used to be swallowed by the clip — silently, because a clipped press region cannot be pressed either.
  • Min/Max reflow — when a clamp changes a box's extent, its child re-measures against the final size.

The macOS shell — Photon's first real window (W5, 2026-07-31)

eQuantic.UI.Native.Shell.MacOS: an NSWindow hosting a CAMetalLayer, zero third-party packages — slim typed objc_msgSend AppKit bindings, the Metal-spike pattern. The Metal backend encodes each frame straight into the layer's drawable (per-pixel-format pipeline cache, BGRA8_sRGB for windows); PhotonHost.RenderScale rasters at backingScaleFactor while layout/input stay in dp (retina). OS mouse/scroll events route into the ordinary pointer pipeline — press visuals, hover diffs and anchored menus behave exactly like the tests. Real San Francisco text and real pack icons render on screen. Try it: dotnet run --project samples/PhotonDesktop. Two headless modes need no window at all: --Photon:MaxFrames 120 presents that many frames and exits, and --Photon:ScreenshotPath out.png (with --Photon:Mode Dark for the other palette) renders ONE settled frame through the reference backend with the same CoreText metrics — the CI screenshot step, and the way a fidelity pass against a design handoff is checked. Calling Run() off the main thread now says so in .NET terms instead of dying inside AppKit. Resize the window freely — the host adopts the viewport with all state intact.

The caret, and what a selection has to show

A selection has two ends and only one of them moves. Which one is not something a person can deduce from the band, so the caret is drawn WITH the selection, not instead of it: the band says which characters are held, the caret says which END you are holding — the one ⇧-arrow will move, the one the next character replaces from. Hiding it is why a selection used to feel directionless.

Two rules go with it, and both are about being visible when it matters:

  • The blink phase starts at the last caret MOVE, not at the epoch. Off a free-running phase, a caret that just arrived somewhere can be in its OFF half for half a second — you press ⇧→ and look at nothing. Every write to the caret restarts the cycle, which is why the host routes them all through one property.
  • While a drag is drawing a selection the caret does not blink at all. The end following your pointer is the one thing you are watching.

On the web the field is a real <input> and the browser owns its caret; what is described here is what Photon paints, and what the editor surface (which both targets paint themselves) does.

The blink is PACED, not pinned

A caret is 2Hz motion, not vsync motion. Holding NeedsRender for it pinned the whole loop to the display's refresh — 120.3 presents a second, measured on a ProMotion panel, to flip a rectangle twice a second, for as long as any field held focus. An editing caret now SCHEDULES instead:

  • RenderFrame leaves NeedsRender false when the caret is the only reason for a next frame, and records when the next blink TRANSITION is due;
  • the shells' idle ticks (the macOS 120Hz timer, the phones' vsync callbacks) ask host.IsFrameDue(nowMs) alongside NeedsRender — one present per half-period, landing on the transition, ~2 a second instead of the refresh rate.

Real motion (a spinner, a glide, a transition) still keeps the loop hot — the schedule exists only for the caret. While a drag draws a selection nothing is scheduled at all: the caret is forced solid and the pointer's own movement drives the frames. Ten simulated seconds of a focused field: 20 presents, every one flipping the blink — asserted in CaretPacingTests, count included.

The web needs none of this: the browser owns the <input> caret, and the editor surface's caret div can blink on the compositor.

Hot reload — dotnet watch run, and the app follows

Run any Photon app under dotnet watch run, edit a Build body, save — the window redraws with the new code and ALL state intact, within a tick. No SDK command, no configuration: the plumbing ships in eQuantic.UI.Native.Components and every host signs up at construction.

Why the slice is small is the interesting part. The architecture was already hot-reload-shaped:

  • the tree is RE-EXPANDED from the retained root on every frame (Build runs during measure), so a patched method body simply takes effect on the next frame;
  • component state lives in a PATH-keyed store, not in the objects an edit replaces, so nothing has to be migrated — three clicks of counter state survive a reload because nothing touched them.

What was missing was the next frame itself. An idle window presents nothing, an idle phone ticks its vsync and skips, and a metadata update arrives with no input event attached — the edit applied and nothing moved until the mouse did, which reads as "hot reload doesn't work" while it is working perfectly. PhotonHotReload (the [MetadataUpdateHandler]) does exactly two things:

  1. wakes every live host — a bool the render loops already poll (macOS within 50ms, the phones on their next vsync), and
  2. bumps a GENERATION the host compares on its own render thread to drop the content caches once (text rasters, icons, images). The keys are content, so most edits would miss them — but a patched rasterizer or an edited icon path is exactly the edit a person makes while tuning visuals, and serving yesterday's pixels turns a working reload into a haunted one. Nothing is mutated from the handler's thread.

Rude edits (new types, changed signatures) are the runtime's to refuse — dotnet watch answers those with a restart, which is correct and needs nothing from the framework.

Mobile: the same registry serves the iOS and Android hosts (their vsync ticks poll the same bool), so hot reload in the SIMULATOR comes with this slice. A physical device needs dotnet watch to reach the process, which is a deploy-channel question, not a framework one.

Shaders — ONE normative Slang source

Sdf.slang (transliterating Sdf.cs) compiles offline via the embedded slangc into committed MSL + SPIR-V — SDF fragment plus the W4b textured fragment (texel Load, nearest by definition — rasters are device-scale, exact Reference parity). GPU↔CPU parity holds at 17/17 scenes within the golden tolerance.

IME — composition is the platform's text (M4, 2026-08-07)

While a field holds the caret, the macOS shell hands key events to the PLATFORM's input context (NSTextInputClient — the view conforms, registered at runtime like every other override):

  • A dead key or a CJK method composes over several keystrokes as marked text: held by the host (SetMarkedText), rendered INLINE at the caret with an underline, while the field's VALUE stays untouched. Commit (CommitText) enters the field through the same door every keystroke uses; cancel leaves the field exactly as it was. Composing over a selection replaces it — the same rule typing has. An obscured field composes blind (echoing the composition would echo the secret).
  • Editing keys come BACK from the input context as selectors (doCommandBySelector:) and re-enter through the same OnKey door, re-spelled to DOM names — Backspace, arrows (with Shift/Alt variants), Home/End, Escape. Command/Control chords never enter the context: ⌘A/⌘C/⌘Z belong to the app.
  • firstRectForCharacterRange anchors the candidate window at the CARET (host CaretRect() → y-flip → view → window → screen), so the CJK picker appears under the composition.
  • Proof: 8 ImeCompositionTests at the host (dead-key sequence, cancel, compose-over-selection, caret-riding, ranges, focus-loss drops composition) and the window self-test drives the REAL registered selectors (setMarkedText: → insertText:) against the ⌘K search field: ime probe: marked '´' → committed. What only human fingers can prove is the system INVOKING those methods — type option+e e in any Studio field to see é compose.
  • Fence: code surfaces compose blind for now (commit works; the editor's face lives in its child's render, which has no style channel for a marked run yet).

Accessibility — the semantics tree (M4 bridge v1, 2026-08-07)

Photon draws its own pixels, so the OS sees one opaque view — without a semantics tree the app is INVISIBLE to a screen reader. The tree is now a first-class frame artifact:

  • SemanticsTree.Collect(frame) (shared, target-neutral) derives reading-order semantics from the realized layout — page AND overlay layers, so dialogs are seen. Roles: StaticText, Button, Link, TextField, CodeField, Slider (Adjustable), Image (labelled Icon). A control's inner text is its accessible name (the web's <button>text</button> rule); a labelled Icon announces, an unlabelled one is decoration. Identity is the layout PATH — the same identity presses, focus and scroll use — because the tree is rebuilt every frame and references go stale.
  • Scrolled-out content is included, exactly like FocusStop and deliberately unlike pointer hit regions: linear navigation reaches below the fold; clipping guards the pointer, not the reader.
  • PhotonHost.Semantics() exposes the tree; PhotonHost.ActivatePath(path) runs a control the way a tap does (resolved out of the current frame's regions, refusing disabled).
  • macOS bridge (PhotonAccessibility): the content view answers accessibilityChildren with EQAXElements (an NSAccessibilityElement that can be pressed) — role, label, value, enabled, y-flipped accessibilityFrameInParentSpace, the path as accessibilityIdentifier, and accessibilityPerformPress routed through ActivatePath. Elements rebuild per query; the previous batch is released.
  • Proof: SemanticsTests (8) pin order, naming, dialog visibility, disabled surfacing and the activate action — plus the PARITY GATE SemanticsMatchFocusStops, which keeps the semantics walk and the input walk from ever drifting apart. The window self-test prints the live AX answer (accessibility elements: 100 — first: AXButton "Back" on the Studio gallery).
  • Web parity is native to the web realizer (real <button>/aria-label/placeholder) — nothing new owed there. iOS (UIAccessibility) and Android (AccessibilityNodeInfo) consume this same tree when their bridges land.

Status

Native suite: 301 tests, 74 goldens (component pairs light+dark, mid-drag sheet states, texture coverage). The window runs the write-once showroom with real text, icons, menus and gestures. Next on this track: window resize (host viewport), image decode/upload, RHI extraction from the Metal spike's shape, Vulkan backend, iOS shell (NativeAOT gate).

Clone this wiki locally