An interchangeable-lens digital camera whose computing lives on a phone. The camera carries the optics, the sensor, the controls and its own WiFi radio; the phone does live view, RAW processing, storage and editing. The phone is the camera's operating system, not the camera.
| server/ | Mock camera. Emits byte-for-byte what real firmware must emit. |
| app/ | Expo React Native app. Finds the camera, renders live view, applies the look. |
| docs/protocol.md | The wire contract, and the specification for real firmware. |
Anywhere a shortcut would be easy in Node but wasteful on an embedded SoC, the mock takes the harder option — frames go over the wire binary, never base64, because base64 would cost a camera a third of its WiFi bandwidth.
Proved the riskiest part of the architecture before any hardware existed: a phone receives frames off the camera over WiFi and renders them as a viewfinder, automatically, with usable latency.
Preview and capture are independent paths, mirroring a real mirrorless camera — the EVF never sees RAW. Live view is a downscaled JPEG stream; RAW is reserved for the shutter press.
A configurable colour-science pipeline on the phone: neutral frame in, styled frame out, live in the viewfinder, with the same transform baked into real bytes on capture.
camera ──JPEG bytes over WiFi──► Skia.Image.MakeImageFromEncoded ← no base64
│
┌───────▼────────┐
Recipe ──bake──► LUT │ style shader │ ← one GPU pass
(JSON) (texture)└───────┬────────┘
▼
screen (live)
│
shutter ──► same shader, offscreen surface
▼
encodeToBytes() → real JPEG on disk
A file exported from the app contains genuinely stylized pixels. Open it anywhere, on any device, and it shows what the viewfinder showed — nothing about the recipe is needed to read it.
But the recipe, not the pixels, is the document:
| Input | Output | Destructive? | |
|---|---|---|---|
| Preview | JPEG frame | GPU surface, discarded next frame | no — nothing is kept |
| Capture | same frame | real JPEG bytes, plus a .recipe.json sidecar |
yes, deliberately |
| Re-export | source + sidecar | new bytes, any recipe | no — source is preserved |
The preview stream is not a candidate for baking, and not for convenience reasons: it is a 720p JPEG the camera produced by discarding most of the sensor data and lossily compressing the rest. Stylizing those bytes would be grading a JPEG of a JPEG. This is how a Fuji behaves — the JPEG has the simulation burned in, the RAF keeps the option open.
White balance, levels, master and per-channel tone curves, highlight rolloff, shadow lift, saturation, both hue wheels, skin protection, the chrome effect and split toning are each a stateless function of RGB alone. Composed, they are still one function of RGB.
So they are evaluated once across a colour cube whenever a recipe changes — off the frame path —
and cost one texture fetch per pixel at frame time, however many are switched on. Adding a new
colour control costs nothing per frame; it only has to be implemented in applyRecipeCpu.
Only genuinely spatial effects need their own pass. Grain and vignette are functions of position, so they ride along in the same shader for free.
33³ is the film-industry default and it is not enough here — a counter-intuitive result worth recording. Against the reference implementation, sampled at inter-node midpoints:
| Cube | Interpolation | Worst error, a heavily graded recipe |
|---|---|---|
| 33³ | trilinear | 8.19 / 255 |
| 33³ | tetrahedral | 5.50 / 255 |
| 65³ | trilinear | 4.07 / 255 |
Not because any one operation is unstable: saturation, both hue wheels, the chrome effect and split toning each contribute curvature and they compound. Moving the per-channel work into a separate 1D shaper barely helped — the residue is genuinely three-dimensional.
What is left is not a resolution problem, and half of it turned out to be avoidable. Every
hue-gated operation here is a function of HSL hue, which misbehaves in two places: the hexagon
vertices where r == g, where hue is continuous but its derivative is not, and the achromatic
axis, where hue is barely defined at all and swings 15° between adjacent cube nodes. Neither
reconstructs by linear interpolation, which is why error scales with node spacing rather than its
square, and why 33 → 65 only halved it. A bigger cube is the wrong lever.
The near-axis half is avoidable, and finding it was worth more than the cube size was: a hue-gated effect must fade out before hue stops being well-defined. Colour Chrome FX Blue originally acted on anything above saturation 0.05 and was on its own responsible for 6.4 / 255 of error — against 1.4 for the entire rest of the pipeline — landing on pale near-white blues, which is to say skies and walls, which is where banding is most visible. Gating it to fade out below saturation 0.3 removed it outright and cost nothing: a real sky sits at saturation 0.58 whether it is deep zenith blue or near-white haze.
Shipping presets now reconstruct within 4 / 255 — Classic Chrome measures 3.2, down from 6.4 before the gate was widened. A deliberately pathological fixture, with ±16° of rotation on every hue spoke, still reaches ~21 / 255 near the grey axis — the honest ceiling of doing hue work in HSL at all. The fix for the remaining half is an opponent-colour space, where rotating and scaling the chroma vector is smooth everywhere except the axis. That is a colour-science change with its own retuning, and it is not in this milestone.
Slices are tiled 9 × 8 rather than laid end to end. In a single row a 65-cube would be 4225 texels wide, past the 4096 that older mobile GPUs guarantee — the texture would fail to allocate on exactly the cheap hardware most likely to run this.
Neutral is the control case: it must be bit-for-bit identical to no pipeline at all, and if it is
not then every other measurement here is suspect. Classic Chrome is the one that has to justify
the milestone.
An earlier set — Velvia, Classic, Mono — was hand-tuned by eye and has been deleted. Not because it looked bad, but because "tuned by eye" is a dead end: there is no way to tell whether a change made a look more like the film or merely more like something. Everything now comes from a written parameter reference, so every value has a source and a confidence tag, and a wrong one is a wrong number that can be corrected rather than a matter of taste.
style/fuji.ts is the only file that knows Fujifilm exists. Settings go in — Color +1, Highlight
+1, Grain Weak/Small — and a Recipe comes out. Keeping that boundary is what stops one
manufacturer's UI conventions ending up inside the document format.
Four things in that reference are counterintuitive enough to be worth restating:
The sign convention on Highlight and Shadow is backwards from the obvious guess. Positive means more contrast at that end, at both ends: +1 Highlight brightens highlights and +1 Shadow darkens shadows. Read as "positive brightens" the shadow control comes out inverted, which is invisible to any test that only checks that something changed.
Negative Clarity is not an inverted unsharp mask. It is a soft-focus filter — it blooms highlights outward. Two different operations sharing one slider, so the negative half drives the shader's highlight gather and the Clarity uniform clamps at zero.
Halation, vignetting and chromatic aberration are all off, and must stay off. Halation is a property of physical film; Fuji's simulations do not reproduce it. Vignetting and lateral CA are lens defects Fuji corrects. Every one of them is a good effect, and every one of them moves the result further from a Fuji. They stay in the pipeline for original recipes, where they earn their place — and there is a test asserting they are zero in every Fuji preset, because "it looks nicer with a bit of halation" is exactly the change someone will make.
Noise reduction is asymmetric. Chroma reduction is strong at every setting and barely moves across the range; luma reduction is what the slider actually controls. Colour noise carries no detail so there is no reason to leave much in; luma noise carries texture so removing it costs something.
The reference is emphatic that a film-simulation cube expects an image in a known state — linear contrast, from a calibrated profile — and that applying one to an already-rendered JPEG produces something wrong in a way that is hard to attribute. The detail settings have exactly the same problem and it does not get flagged, because they look like ordinary sliders rather than a table with a domain.
Fuji's Sharpness 0 means "the standard amount of sharpening for demosaiced sensor data". Our input is a JPEG a webcam has already sharpened and already denoised, with settings we do not control and cannot see. Applying the number on top does it twice: it amplifies the ISP's own halos and its compression noise. The result reads as a grainy, crunchy image while every individual control sits at a value the reference endorses — which is precisely what shipped first, and what it looked like.
So FujiSettings carries a source, and the detail stage scales by how much work has already been
done for it. Settings keep their meaning — Sharpness +4 still does more than Sharpness 0 — and
against a rendered JPEG the whole stage runs at a quarter strength. Flip it to sensor the day the
RAW path exists and it stops holding back.
The conventional settings for Classic Chrome call for weak/small grain, and it is the one place a shipping preset departs from them.
Fuji's grain is roughly pixel-scale on a 40MP file. Matched as a fraction of the frame — which is what makes a setting mean the same thing at any resolution — that is 0.25px on a 1280-wide preview, below what a pixel can represent. The shader floors the cell at one pixel, which turns a fine emulsion texture into one-pixel coloured noise at 35% strength: not film, a bad sensor, and louder than the thing it is approximating rather than quieter.
Off is closer to the target than a wrong-scale approximation of it, and Fuji's weak/small grain is nearly invisible anyway. One word turns it back on the day the frame is big enough to hold it; nothing else changes, because the size is already a fraction of the frame.
The reference quotes sharpening radii and grain sizes in pixels, measured on a 40MP body. Ours are fractions of the frame, so converting through that body's long edge is what makes a setting mean the same thing at any resolution — and it produces a number worth knowing before judging the preview.
A 0.7px sharpening radius on a 7728px frame is 0.12px on a 1280px one. Fuji's grain is finer still. Both land below one pixel, and the shader floors them at about one. That floor is correct rather than a workaround — an acutance radius is tied to the pixel grid, not to the scene — but it means these controls are sitting at their floor rather than at their setting, and grain at preview resolution reads closer to sensor noise than to emulsion. Nothing here needs to change when the sensor lands; the numbers simply start tracking.
Provia is built and tested but deliberately not in the strip, because it should look like almost nothing — and that is the point.
The film-simulation cubes published for these bodies expect an image in a specific state: linear contrast, from a calibrated camera profile. Our input is not that. It is a JPEG a camera already rendered, white-balanced and tone-mapped by an ISP we do not control. A correct Provia on top of that is close to a no-op, because the standard rendering has already happened once; layering another would apply it twice.
So Provia is the recipe that must look like nothing. If it ever visibly changes the image, the fault is upstream of every look rather than in that one — and it would be wrong underneath all of them. There is a test holding it under 2/255 across the whole colour cube.
This is also the honest limit of the current colour work. The identity of a film simulation is a 3D
table, not a set of slider values, and SIMULATIONS in fuji.ts is a hand-authored impression
rather than a match. Substituting a measured cube is a drop-in change — the pipeline already
collapses all of colour and tone into one 65³ texture, so an external table is the same texture
from a different producer — but it is worth little until there is a calibrated linear pipeline to
put it at the end of, which means the RAW ISP, which means the sensor.
app/src/style/ is plain TypeScript with no React, Skia or react-native imports — only
render.ts touches Skia. That boundary buys npm run test:style, which runs the whole colour
pipeline on the laptop under node --test.
cd app && npm run test:styleNumeric colour code fails as "the picture looks a bit wrong", which is close to undebuggable through a viewfinder and trivially bisectable here. Two of these tests earned their keep during the build:
- The LUT accuracy test sampled on a 17-step grid, and 17 steps over a 33-node cube puts every probe exactly on a node — where the table is exact by construction. It passed comfortably while the real interpolation error was 8/255. A LUT test that samples on nodes measures the bake and calls it interpolation. It now probes inter-node midpoints.
- The shader's tiled-cube arithmetic is re-implemented in the test and checked against the reference sampler, including the hardware bilinear filter it leans on. That is the riskiest twenty lines in the pipeline and the least observable: a wrong tile origin does not crash, it just tints the picture.
The highlight rolloff was rewritten because of this too. The original rescaled the compressed segment so white stayed white, which put a slope discontinuity at the knee — invisible to a monotonicity check, plainly visible in a sky as a hard contour, and measurably the largest single contributor to reconstruction error. A concave curve cannot both hold unit slope at the knee and preserve white; rolloff is the one worth keeping.
One pipeline, two resolutions. drawStyled renders the viewfinder and renderToBytes renders
the export, and the second calls the first. They differ only in size and in where pixels land.
This is why the viewfinder can be trusted, and it is structural rather than asserted.
Every spatial parameter is a fraction of the frame, never pixels. Grain size, vignette radius. Grain sized in pixels looks right at preview resolution and vanishes at full resolution, which would silently break the promise above the moment real sensor hardware lands. For the same reason the shader is handed the size of the fitted frame rather than the screen, so grain and vignette sit relative to the picture and not to the device it happens to be shown on.
The viewfinder does not crop. It letterboxes: a 16:9 frame on a 19.5:9 phone gets narrow black bars at the sides. Cover-fit would fill the screen but hide ~18% of the frame off the top and bottom while the captured file still held all of it — composing one picture and saving another. A viewfinder that crops is a viewfinder that lies, which is why real cameras show bars too.
The grain seed is a constant, not a clock. A time-varying seed would make the viewfinder shimmer and — worse — would mean the exported still could not match the frame you saw.
Neutral is a true no-op. The renderer skips the lookup entirely when a recipe's colour and tone modules are inert, so the neutral preset is bit-for-bit the untouched frame rather than the frame round-tripped through an 8-bit cube.
No development build, no Xcode, no Apple Developer account. It also deleted the per-frame base64
encode: Skia.Image.MakeImageFromEncoded takes the JPEG bytes as they came off the socket,
where <Image> needed them turned into a data: URI first. The style layer arrived cheaper than
the code it replaced.
Skia's
<Canvas>requires react-native-reanimated, which is not obvious from anything in this codebase — nothing here animates. It fails at runtime with "react-native-reanimated is not installed!" rather than at build time.npx expo install react-native-reanimatedpullsreact-native-workletswith it, andbabel-preset-expowires the worklets Babel plugin automatically once that package is present, so nobabel.config.jsis needed. If a future change adds one, the worklets plugin must be listed last.
That also removed the old Viewfinder's double-buffering and decode watchdog, which existed only to
work around <Image> refusing to repaint a changed URI. In their place is one real obligation: an
SkImage is native memory, and at 20fps not releasing it is roughly 24 MB/s of pressure — but it
cannot be released the moment it is replaced either, because the picture on screen still
references it. Each generation is retired when the one after it arrives.
Separable from the colour science and independently toggleable, as its own module:
| Grain | size, intensity, roughness, colour. Two octaves of value noise mixed by roughness — one octave reads as sensor noise, not emulsion. Luminance-weighted to peak in the midtones (4L(1−L)), because that is where developed silver actually lives. |
| Halation | strength, threshold, radius, hue. Threshold the highlights → tint → blur → screen-blend back. |
| Bloom | the same mechanism, neutral and broader — light in the air rather than inside the emulsion. |
| Vignette | strength, radius, softness. Elliptical to the frame, so it is resolution-independent. |
| Chromatic aberration | signed. Modelled as a magnification difference per channel about the optical axis, so fringing grows toward the corners and vanishes at the centre on its own. Positive adds the defect, negative corrects it. |
Halation is the one that most reads as "shot on film", and it is the thing a digital sensor never
does: light passes through the emulsion, scatters off the film base, and comes back up to expose
the layers a second time. It is red because the anti-halation backing absorbs shorter wavelengths
first. Measured on a white square against black at strength 0.6 — 4px outside the edge the glow is
r=52 g=31 b=23, by 20px it is gone.
Two things worth knowing about how it is built:
The glow pass is the same shader, selected by a ShaderMode uniform, not a second shader that
re-implements exposure and the colour lookup. Two copies of a colour pipeline is exactly the kind
of duplication that drifts silently — the halation would slowly stop matching the image it is
supposed to be scattering off.
Recipes that do not scatter light never allocate anything. recipeHasGlow gates the extra
passes, so the neutral path is one pass and one quantisation, exactly as it was before halation
existed. The glow itself renders at quarter scale into a cached surface: it is heavily blurred by
definition, so resolution buys nothing and the blur gets four times cheaper per axis.
Screen-blend rather than add, because r = s + d − s·d cannot exceed white — a glow over an
already-clipped highlight brightens its surroundings instead of blowing a hole in the frame.
Sharpening, clarity and noise reduction need neighbouring pixels. The obvious design renders the graded frame to an intermediate texture and runs a second pass over it — and that design is impossible here, for a reason worth writing down because it costs an evening to rediscover:
// react-native-skia/apple/MetalContext.h
static thread_local MetalContext instance;Skia's GPU context is thread-local. Surface.MakeOffscreen runs on the JS thread and produces
a texture owned by that thread's Metal context, while the picture we record is replayed on the
render thread with a different one. The texture is simply not usable there, so the frame draws
black — with no exception and nothing in the log, because nothing failed. The single-pass
constraint is not a simplification; it is the only thing that works.
So everything happens in one shader invocation, and the neighbourhood taps read the source frame rather than a graded intermediate:
source → CA → noise reduction → clarity → sharpen → output sharpen
→ exposure → colour cube
→ halation, bloom (17-tap gather)
→ grain → vignette → dust, scratches, light leak
That order is not a compromise. Noise reduction and sharpening are ISP operations on sensor data, not grading operations on a graded image — doing them before the look is the more faithful order, and it is where the reference pipeline puts them too.
Halation and bloom gather 17 taps over two rings instead of blurring across passes. The inner ring is rotated half a step off the outer one, because a single ring reads as an eight-pointed star around a bright point.
Within the detail stage the order is fixed: noise reduction → clarity → sharpen. Sharpening before denoising amplifies exactly the noise you are about to remove, and sharpening after grain is what makes an image look crunchy rather than filmic. A test asserts the whole ordering, because reordering those calls is a one-line change nothing else would notice.
Noise reduction filters luma edge-aware — neighbours weighted by brightness similarity, so edges survive — and chroma flat, then recombines colour from the heavily smoothed version with brightness from the careful one. That is why chroma NR at 0.6 still looks sharp.
Dust, scratches and light leaks are procedural, not composited from scanned overlays. No assets to ship, but the real reason is that a tiled dust texture puts the same speck in the same place on every frame of a sequence — which reads as a dirty lens, the one thing the effect must not look like.
Grain is ISO-linked. The camera's reported ISO reaches the shader as live context rather than as part of the recipe, because a recipe describes a portable look while ISO belongs to the exposure that was actually made. At full link the intensity roughly doubles every two stops above ISO 200, matching how emulsions are specified.
Not in this milestone: deconvolution sharpening, bokeh simulation (you have real lenses), and the RAW ISP. See Where this goes next.
Known issue: the colour cube bake takes ~4.5s on device, against 144ms on a laptop — Hermes has no JIT, and 275k samples of allocating colour maths is the worst case for it. It is synchronous, so switching preset freezes the viewfinder for that long. The fix is an allocation-free inner loop plus chunking the bake across frames; not yet done.
Both devices must be on the same WiFi network.
# terminal 1 — the camera
cd server && npm install && npm run dev
# terminal 2 — the app
cd app && npm install && npx expo startThe server prints its LAN address on boot. Scan the Expo QR code with Expo Go; the app finds the camera by itself.
Using the app: the viewfinder is bare by design — the image edge to edge and one status label.
Tap anywhere to reveal the preset strip and the shutter; both hide themselves again after a few
seconds. Captures land in the app's document directory as DIY_<timestamp>.jpg alongside a
matching .recipe.json.
Server flags: --port --fps --width --height --quality --source --caption --device --capture-size --capture-fps --raw-width --raw-height.
Expo Go on the App Store and Play Store is stuck at SDK 54 — the SDK 55 build was submitted to Apple in May 2026 and never approved, and SDK 56+ ships only via TestFlight or a custom build. A project on any newer SDK fails on a real phone with "Project is incompatible with this version of Expo Go", and no amount of updating Expo Go fixes it, because the store already has the newest one Apple ships.
create-expo-appdefaults to the latest SDK, so a fresh scaffold will reintroduce this. Install native modules withnpx expo install, never plainnpm install— Expo Go ships fixed builds of them and a version mismatch fails on the device, not in the terminal.
server/src/framesource/ is a registry, selected with --source. Replacing the mock with real
hardware means adding one file here — the protocol, and therefore the whole app, stays untouched.
npm run dev # testpattern (default)
npm run dev -- --source=webcam # live, from the Mac's camera
npm run dev -- --source=webcam --device=1 # a different camera
npm run dev -- --source=webcam --width=1920 --height=1080
npm run dev -- --source=webcam --capture-size=1920x1080 # force the device's capture mode
npm run dev -- --caption # burn seq + clock into the frameThe test pattern carries SMPTE bars, a grey ramp, four skin tones and six vivid colour patches. The patch rows are there for the style pipeline rather than the transport: a film simulation is judged on what it does to skin and to already-vivid colour, and neither is answerable by pointing a webcam at a room. You need known values, side by side, that do not move.
--caption burns the frame counter and camera clock into the image. Off by default because it is
not what a viewfinder should show, but kept because reading that counter against the app's own is
the only thing that distinguishes the picture is stale from the app is not repainting.
The webcam source shells out to ffmpeg (brew install ffmpeg) and asks for raw RGB, not
MJPEG, so real frames go through the same encode path as the test pattern.
List devices with ffmpeg -f avfoundation -list_devices true -i "". If ffmpeg exits immediately,
the usual cause is the capture rate: it defaults to 29.97fps and Mac cameras accept only 15 or 30,
so we request 30 explicitly — --capture-fps=15 if a device disagrees.
The output rate must be pinned too (
-fps_mode cfr -r), and this is not optional. Left alone, ffmpeg free-ran at roughly 600fps on a stock Mac — 550 MB/s of raw RGB into a 64 KB pipe. Node cannot drain that, so ffmpeg blocked on write and every frame that arrived was enormously stale. Because the caption is composited after capture, the frame counter kept climbing over a photograph that never changed: the picture froze while every statistic on the phone stayed perfect. Measured frame-to-frame change was0.02— the sensor noise floor — against3.5–10.6once pinned.The rate is set to 1.5× the broadcast rate, capped at the camera's own. Matching it exactly costs ~15% of frames, because the two clocks drift and any tick landing before the next capture finds nothing new. A warning fires if ffmpeg ever exceeds twice the requested rate again — a flooded pipe is close to undiagnosable from the phone, so it has to announce itself here.
Always request an explicit capture mode. Left to itself, ffmpeg opens the device in whatever mode it defaults to, and on a modern Mac that turned out to be square — a 1552×1552 Continuity Camera mode. Scaled into the old 640×480 preview that left a small square picture with black pillarbox either side, and the visible image was roughly 360 real lines stretched across a 1179-pixel-tall phone screen. It looked like a soft, low-resolution preview. It was a perfectly sharp preview of the wrong capture mode, and nothing anywhere reported it, because nothing failed.
So the source now passes -video_size, defaulting to the preview size, overridable with
--capture-size=1920x1080. If a device does not offer that mode ffmpeg refuses and lists the ones
it does — failing loudly beats silently capturing a square.
The preview default is 1280×720: the native aspect of essentially every webcam and phone sensor, and the aspect a phone held in landscape actually wants. Nothing is spent on letterbox bars, and the capture path saves at that resolution too.
As a backstop, the source inspects the first real frame for uniformly black bands at the edges and reports the exact percentage wasted. It measures the symptom rather than interrogating the device, so it catches any cause — and it is what found the square-capture bug.
Measured at quality 70, 20fps, sustained, zero dropped frames:
| Preview | Frame | Bandwidth | Link |
|---|---|---|---|
| 640×480 (old default) | 15.1 kB | 300 kB/s | 2.46 Mbps |
| 1280×720 (default) | 34.1 kB | 674 kB/s | 5.52 Mbps |
| 1920×1080 | 61.3 kB | 1210 kB/s | 9.91 Mbps |
| 1280×720 @ quality 80 | 41.0 kB | 809 kB/s | 6.63 Mbps |
The server sustained 19.7–19.8fps with zero drops at every row, including 1080p — so on this hardware the mock is not the constraint. For real hardware the 720p row is the number that matters: a camera SoC needs roughly 5.5 Mbps, comfortably inside what 2.4GHz 802.11n does in practice. Bandwidth is not the constraint; encode throughput on the SoC is the thing to verify.
1080p is a real option and measurably works server-side. The open question is the phone: JPEG decode is synchronous on the JS thread, and 1080p is 2.25× the pixels of 720p per frame.
One caveat: frame size varies with scene entropy, and these were measured on a static subject against a plain wall. Foliage, texture and motion will push frame sizes well above this, so treat the table as a floor rather than a worst case.
curl -s localhost:8080/status | jq # identity and state
open http://localhost:8080/preview.jpg # one frame — is the generator OK?
curl -sD- -o /tmp/f.raw -X POST localhost:8080/capture | head # RAW + metadata header/preview.jpg exists purely to separate "the generator is broken" from "the transport is broken".
The single most important behaviour in this system, and it has now been the answer in three separate places — a different-looking bug every time:
| Queue | Producer → consumer | Symptom |
|---|---|---|
| WebSocket send buffer | camera → network | viewfinder drifts seconds into the past |
| Image decode | network → screen | picture freezes, network stats perfect |
| ffmpeg stdout pipe | capture → encoder | picture freezes, frame counter still climbing |
The third was the nastiest, because the caption is composited after capture — so the instrument that measured staleness was itself fresh, and the failure looked exactly like a UI bug from the phone. Worth reaching for this rule first anywhere a fast producer feeds a slower consumer, and worth checking that whatever measures freshness sits on the producer side of the queue.
The second one is now free rather than engineered: Skia decodes synchronously, so each arriving frame simply replaces the last and intermediate frames are never looked at.
The status label shows painted frames per second, not received. Absolute latency is not
computable from the phone — camera timestamps are boot-relative and we deliberately do not clock
sync — so read it off the burned-in counter with --caption, by holding the phone next to the
laptop.
docs/protocol.md the camera ⇄ phone contract
server/src/
index.js HTTP + WS wiring, broadcast loop, backpressure
framing.js binary frame headers ─┐ these three must
capture.js RAW Bayer synthesis │ change together
framesource/testpattern.js bars, ramp, skin + colour patches
app/src/
camera/framing.ts binary frame headers ─┘
camera/connection.ts WS lifecycle, reconnect, link measurement
camera/discovery.ts remembered address → subnet scan → manual entry
style/recipe.ts what a look is, as data — the document
style/curves.ts tone shaping ─┐
style/color.ts colour maths │ pure TS, tested in Node
style/bakeLut.ts the cube + CPU reference
style/fuji.ts Fuji's settings → physical parameters
style/presets.ts the two looks that ship, plus a test fixture
style/shader.ts the SkSL source ─┘
style/render.ts the only file that imports Skia
style/save.ts bake a styled JPEG + recipe sidecar
ui/Viewfinder.tsx Skia canvas, one styled pass
ui/PresetStrip.tsx hidden until tapped
server/src/framing.js, app/src/camera/framing.ts and docs/protocol.md describe the same bytes
from three directions. Change one, change all three.
- Bake performance — see the known issue above. The inner loop allocates roughly ten arrays per sample across 275k samples, which Hermes handles far worse than V8 does.
- Deconvolution sharpening — reverses lens and sensor blur rather than boosting edge contrast. More expensive and more real than unsharp mask.
- The RAW ISP — black level, lens shading, demosaic, auto white balance, colour matrix, linearize. Buildable now against the mock's synthetic Bayer frame, which was written to be genuinely demosaicable, and required the moment real sensor hardware exists. This is what makes exports full resolution; 640×480 is currently all the hardware produces.
- Opponent-colour hue work — would take the LUT residue from 4/255 to the storage floor and
remove a real (if subtle) crease from gradients crossing the
r == gplane. - Real hardware — Raspberry Pi Zero 2 W + IMX477. Replaces the frame source; the protocol and the entire app stay untouched.
- Open decision — does the real camera join your home WiFi (onboarding UX, mDNS) or run its own SoftAP the phone joins (works anywhere, no internet while shooting)? Most action cameras do SoftAP. Nothing here forecloses the choice.