Ship continuous color/size and density_val as u8 on the live drill wire

[Alek99]

Live drill and sample-overlay updates ship continuous color/size channels and the density_val crossfade weight as u8 LUT coordinates (dtype: "u8" marker; absent means f32 — the client accepts both), mirroring what categorical codes already do. Base: main. No build artifacts in the diff — bundles regenerate via node js/build.mjs / packaging (#214).

Measured impact

Same 190,105-point drill window on the FastAPI /drilldown example (100M-point source), before/after captures diffed buffer-by-buffer:

Metric	before (f32)	after (u8)	Δ
Drill payload on the wire (b64-in-JSON transport)	5,070,199 B	2,788,978 B	−45.0%
Raw buffer bytes per drill	3,802,100 B	2,091,155 B	−45.0%
Bytes/point (raw)	20	11	dossier budget is 12–24 B/pt direct
Channel bytes/point (color + size + density_val)	12	3	−75%
x/y position buffers	760,420 B ×2	byte-identical	unchanged (f32 offset-encoded, §16)
Max quantization error (unit range)	—	0.00196 (= ½/255 bound)	invisible: 256-texel LUT / ~16 px ramp
Server response time (same window)	~4.0 s	~4.0 s	encode is not the bottleneck
Rendering	—	pixel-equivalent	verified in-browser, 0 console errors

Where quantization is safe — and where it is forbidden

The split is by readback, enforced by making u8 opt-in per call site (quantize_continuous on ship_channels):

• Live drill/sample paths opt in — these payloads are rebuilt wholesale per view and their hover/pick answers come from the kernel's canonical f64 columns, so 8-bit channel coordinates are invisible everywhere but the wire.
• The build payload must not — the client retains those columns CPU-side (_cpu.color/_cpu.size) and denormalizes them for tooltip readouts, where 8-bit steps would surface as wrong digits. Pinned by test.

How

• channels.py: quantize_unit_u8 + quantize_continuous threading through ship_channels/ship_color_channel; the two live-interaction call sites in interaction.py opt in; density_val quantizes via _quantize_dval ([0,1] by construction).
• Client: u8 unit scalars bind as normalized attributes (shader unchanged, still sees [0,1]); categorical codes stay un-normalized. _tagChannelBuf re-ids a buffer on dtype change so cached VAOs rebuild rather than misread bytes. f32 remains accepted, so the encoding is backward-detectable per message.
• Spec: wire-protocol.md documents the live-wire channel encodings and the durable/ephemeral contract.

Relation to the #186–188 append stack (not yet on main)

• When the stack lands, this re-expresses through its ship_continuous chokepoint as enc: "u8" alongside enc: "raw" with a protocol bump — that variant is implemented and verified on branch alek/u8-drill-wire-stacked (protocol v9), ready to swap in.
• The review fixes to the stack's _raw_wire_ok guard (P1 f32-collapsed domains, P2 seterr(over="raise") crash) live on branch alek/raw-wire-f32-guard — they patch code that doesn't exist on main, so they ride separately when the stack rolls up.

Verified

• Full suite 2,244 passed; ruff / format / ty clean (no new diagnostics); node js/build.mjs.
• New test pins both halves: drill wire ships 1 B/pt u8 within half a quantization step of the exact unit value; build payload stays f32 with no dtype marker.
• Live end-to-end on the 100M-point FastAPI drilldown: identical rendering before/after for the same view window, buffers on-GPU as normalized UNSIGNED_BYTE, x/y byte-identical to the pre-change capture, all three u8 channels within the 0.5/255 bound, zero console errors.

[codspeed-hq]

Merging this PR will not alter performance

✅ 102 untouched benchmarks
⏩ 1 skipped benchmark¹

---

_{Comparing alek/u8-drill-wire-v7 (87da83e) with main (31d3959)}

Footnotes

1. 1 benchmark was skipped, so the baseline result was used instead. If it was deleted from the codebase, click here and archive it to remove it from the performance reports. ↩

[greptile-apps]

Greptile Summary

This PR reduces live drill and sample-overlay payload sizes with u8 channel coordinates. The main changes are:

• Quantized continuous color, size, and density values on live interaction paths.
• Normalized u8 WebGL attributes while retaining categorical byte semantics.
• Preserved f32 continuous channels in build payloads for tooltip readback.
• Invalidated cached VAOs when attribute type or normalization changes.
• Added wire-protocol documentation and encoding tests.

Confidence Score: 5/5

This looks safe to merge.

The VAO cache now reacts to both element-type and normalization changes. Live continuous channels are tagged and bound as normalized u8 values. Build payloads retain f32 values for client-side readback.

No blocking issues were found in the changed code.

T-Rex Logs

What T-Rex did

• Compared the pre-change live drill run and confirmed it failed with KeyError: 'dtype' at the u8 assertion.
• Compared the post-change live drill run and confirmed the regression now passes on HEAD, with 1 test passing.
• Verified the build/import pipeline after changes completed successfully, with Vite generation succeeding and import/export verification passing (exit code 0).

_{Ran code and verified through T-Rex}

Important Files Changed

Filename	Overview
js/src/45_lod.ts	Reads marked u8 drill channels and binds continuous values as normalized WebGL attributes.
js/src/50_chartview.ts	Adds u8 sample-channel handling and rebuilds cached VAOs when attribute normalization changes.
python/xy/_payload.py	Threads opt-in continuous-channel quantization through the payload helper.
python/xy/channels.py	Adds unit-range u8 quantization while keeping the build path on f32.
python/xy/interaction.py	Enables u8 encoding for live color, size, and density channels.
spec/design/wire-protocol.md	Documents u8 live-channel encoding and the f32 build-payload contract.
tests/test_scatter.py	Covers live u8 encoding, quantization bounds, and retained f32 build channels.

_{Reviews (2): Last reviewed commit: "Re-id channel buffers on normalization c..." | Re-trigger Greptile}