Skip to content

feat(ios): the speech pair closes the last deferral — ratchet 28 -> 26 - #101

Merged
glennmichael123 merged 1 commit into
mainfrom
feat/ios-speech-pair
Sep 2, 2026
Merged

feat(ios): the speech pair closes the last deferral — ratchet 28 -> 26#101
glennmichael123 merged 1 commit into
mainfrom
feat/ios-speech-pair

Conversation

@glennmichael123

Copy link
Copy Markdown
Contributor

startListening and stopListening move from the Swift shim into bridge_mobile_speech.zig, which until now served nothing on purpose. An earlier round researched both actions in full, found four blockers, and wrote down five preconditions rather than shipping. All five are met here, so the file goes from a record of a decision to the implementation it described.

80 of 106 actions in Zig. Ratchet 28 → 26.

The realtime tap

-[AVAudioNode installTapOnBus:bufferSize:format:block:] runs its block on the audio I/O thread, which nothing else in this tree touches. Swift's recognitionRequest?.append(buffer) gets a free ARC retain for the duration of the call; Zig gets nothing, so releasing the request while a tap call is in flight is a use-after-free on a realtime thread.

The handoff is a published pointer plus an in-flight counter, and every access is .seq_cst rather than acquire/release. That is not caution — the dangerous interleaving is Dekker's: the tap increments then loads the pointer while teardown clears the pointer then loads the counter, and acquire/release permits both sides to see the other's old value. That case is precisely "teardown believes no tap is running while a tap holds a pointer it is about to free". One xchg per audio buffer (~86/sec) is not worth reasoning around.

Teardown stops the engine, removes the tap, clears the pointer, then drains for a bounded 20ms. Engine-then-pointer in that order, not the reverse: clearing first would silently drop buffers the tap had already delivered. If the drain ever fails, the request is leaked rather than released, and the fact is logged.

Two crashes that were waiting

  • With no input route, -outputFormatForBus:0 answers a format whose sampleRate and channelCount are both 0, and installTapOnBus: then raises an NSException — an uncatchable SIGABRT. Swift aborts there today (CraftApp.swift:2535-2538); formatCarriesInput refuses instead, using the check AVAudioEngine.h:459-461 spells out.
  • The recognition handler fires on an internal Speech queue, so it hops to the main queue before anything touches AVAudioEngine.

Verified on a simulator, not asserted from the headers

The fixture drives both actions and asserts two new events. craftSpeechStart is emitted on the far side of the whole chain — plist guard, requestAuthorization:'s block on an arbitrary queue, the main-queue hop, the audio session, a live SFSpeechRecognizer, a non-zero input format, the recognition task, the tap installed on the audio I/O thread, and AVAudioEngine actually starting. Any link short of the last emits craftSpeechError instead.

Both assertions were mutation-tested: forcing startAndReturnError: to fail produces the i=74 refusal note and fails on i=68; removing the end emit fails on i=72. The second mutation initially reported the wrong cause — openSettings is chained off craftSpeechEnd and its assertion ran first — so the speech block now sits ahead of it and the failure names the real thing.

One finding worth recording. simctl privacy documents nine services and none is speech, but it does not validate the argument: raw kTCCServiceSpeechRecognition is accepted where speech-recognition is not. Without it the run stopped dead at a "would like to access Speech Recognition" alert no script can answer — which a screenshot of the simulator showed, and the logs did not.

Divergences, each in the module header

  • The plist key stands in for the config flag. NSSpeechRecognitionUsageDescription is written from enableSpeechRecognition and nothing else (no ||), so a missing key means the flag was off — and the message emitted is the one Swift's nil-recognizer guard would have emitted. The check runs before requestAuthorization:, where Swift's runs after, because asking without the key terminates the process rather than failing.
  • A zero format reports "Audio engine failed" rather than inventing a fifth message Swift has never sent.
  • A second startListening while running stops first. Swift cancels only the task and then installs a second tap on bus 0, which raises.
  • A stop that stops nothing still emits craftSpeechEnd. Swift's stopSpeechRecognition() has no early return, and that event is what a page's listening indicator binds to.
  • A recognition error still emits no craftSpeechError — Swift discards the NSError, Android does not, and inventing a string here would prejudge a contract question that belongs to the JS surface.

triggerHapticChecked becomes pub for one caller: CraftApp.swift:2544 and :2558 fire a light impact directly, bypassing the enableHaptics gate the dispatcher applies to case "haptic". That bypass is the spec's, not a choice this port makes.

Verification

  • zig build test, zig fmt --check, zig build, build-ios, build-ios-simulator — all green
  • packages/ios/fixtures/zig-slice/build-and-run.shPASS, 39 assertions, from a clean uninstall/install

@github-actions

github-actions Bot commented Sep 2, 2026

Copy link
Copy Markdown

✅ Binary load time

rounds:    25 interleaved
base:      p50 159.7ms   p95 172.3ms   (151.9–172.3ms)
head:      p50 30.9ms   p95 37.4ms   (26.0–44.1ms)
delta:     -80.7%  (fails above +20.0%)

No binary load time regression.
What this measures

craft --help: process spawn, dynamic linking and argument parsing.
It never opens a window, so it cannot see a change in window or
webview startup — real startup is benchmarks/startup.bench.ts, which
needs a display.

Both binaries are measured interleaved on this runner and compared by
p50, rather than against a number recorded on another machine. On
byte-identical binaries that method reads within ~3.5%; the old one
swung 45%.

@github-actions

github-actions Bot commented Sep 2, 2026

Copy link
Copy Markdown

✅ Binary Size Report

Metric Value
Current Size 14245KB (13.91MB)
Change 0KB (0%) unchanged
Size limits
  • Warning: 14.50MB
  • Maximum: 16.00MB

`startListening` and `stopListening` move from the Swift shim into
`bridge_mobile_speech.zig`, which until now served nothing on purpose: an
earlier round researched both actions in full, found four blockers, and wrote
down five preconditions instead of shipping. All five are met here.

The load-bearing one is the realtime tap. `installTapOnBus:` runs its block on
the audio I/O thread, which nothing else in this tree touches, and Swift's
`recognitionRequest?.append(buffer)` gets a free ARC retain that Zig does not.
Releasing the request while a tap call is in flight is a use-after-free on a
realtime thread. The handoff is a published pointer plus an in-flight counter,
both `.seq_cst` — the dangerous interleaving is Dekker's, and acquire/release
permits exactly the case where teardown believes no tap is running while one
holds a pointer it is about to free. Teardown stops the engine, removes the
tap, clears the pointer, then drains for a bounded 20ms; if it ever fails to
drain it leaks the request rather than racing, and says so.

Two other preconditions were crashes waiting to happen. With no input route
`-outputFormatForBus:0` answers a zero format and `installTapOnBus:` raises an
NSException, which is an uncatchable SIGABRT — Swift aborts there today, and
`formatCarriesInput` refuses instead. And the recognition handler fires on an
internal Speech queue, so it hops to the main queue before touching
AVAudioEngine.

Verified on a simulator, not asserted from the headers. The fixture drives
both actions and asserts `craftSpeechStart` and `craftSpeechEnd`; the start
event is emitted on the far side of the plist guard, the authorization block,
the main-queue hop, the audio session, a live recognizer, a non-zero input
format, the recognition task, the tap, and the engine actually starting.

Getting there needed a finding worth recording: `simctl privacy` documents
nine services and none is speech, but it does not validate the argument — the
raw `kTCCServiceSpeechRecognition` is accepted where `speech-recognition` is
not. Without it the run stopped dead at a system alert no script can answer,
which is what a screenshot of the simulator showed rather than what the logs
said.

Divergences, all documented in the module header: the plist key stands in for
`config.enableSpeechRecognition` (it is written from that flag and nothing
else); a zero format reports "Audio engine failed" rather than inventing a
fifth message Swift never sends; a second `startListening` while running stops
first, where Swift installs a second tap and raises; and a stop that stops
nothing still emits `craftSpeechEnd`, because that is what a page's listening
indicator binds to.

`triggerHapticChecked` becomes `pub` for one caller: Swift fires a light impact
on start and stop directly, bypassing the `enableHaptics` gate its own
dispatcher applies to `case "haptic"`. That bypass is the spec's.
@glennmichael123
glennmichael123 merged commit 14fe431 into main Sep 2, 2026
16 of 18 checks passed
@glennmichael123
glennmichael123 deleted the feat/ios-speech-pair branch September 2, 2026 12:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant