feat(plugins): plugin system v1 + voice-driven meme generator (10-iteration spike, productionized) - #244
Merged
Merged
Conversation
Carries the owner-tested spike (PR #243, 10 iterations) onto main. This commit is the spike content verbatim; the production hygiene, the shipping build defaults, the MAK-100 manifest contracts, CI, and the docs follow as their own commits on top, so the diff between "what was tested" and "what productionizing changed" stays readable. THE SYSTEM - `PluginManifest` — the host/plugin contract: id, name, version, SF Symbol, entry kind, `networkHosts` disclosure, `keyEquivalent` request, and `voiceTriggers`. Id validation is strict because the id becomes a path component under Application Support, so traversal-shaped ids are refused before they are ever joined onto a URL. - `PluginDiscovery` — merges an ORDERED provider list. Providers are passed in DESCENDING trust order and earlier wins, so a writable directory can never shadow a reviewed plugin. The compile-time registry is one entry in that list, which is the seam a real loader plugs into rather than replaces. - `PluginEnablement` — the enabled set, DEFAULT-OFF, with pruning. - `PluginRegistry` — the compile-time list of in-repo plugins. - `PluginHost` / `PluginsPane` / menu-bar submenu — the app-side surface, none of it on AppState (the MAK-32 ratchet is at zero headroom). THE PLUGIN A voice-driven meme generator: dictate a description, and it picks a template, writes the captions, renders them locally, and lets you edit, export, or share. Three merged template providers (imgflip, memegen.link, and the user's own imported library, which is the only one that works offline and in any language), scored search over names + keywords, per-template caption slots, schema- constrained decoding, a normalized box model whose preview IS the export, a busy-state machine whose `finish` is idempotent and ticket-guarded, a rebuildable URLSession, and a learned template affinity that is bounded so it can never resurrect the confident-wrong-template bug this plugin was built to kill. Every DECISION lives in `OpenWhisp/Services` and is covered by `swift test`; `plugins/MemeGenerator/` holds only AppKit/SwiftUI and IO. Co-Authored-By: Claude <noreply@anthropic.com>
…ion-only probes, MAK-100 manifest contracts Productionizes the carried spike. Three groups of change, all verified against real builds rather than by reading. 1. PLUGINS SHIPS BY DEFAULT `PLUGINS` now defaults to 1, matching WHISPERKIT / PARAKEET / SPARKLE; `PLUGINS=0` stays the lean escape hatch. Two things this deliberately does NOT change: plugins remain DISABLED at runtime until the user enables one per-plugin in Settings → Plugins (`PluginEnablement` defaults to the empty set), and the pure plugin core is compiled and tested either way. The source list moved into `scripts/plugin-source-args.sh`, sourced by BOTH build.sh and build-dmg.sh. build-dmg.sh is the RELEASE path and previously had no plugin support at all — left as it was, the shipped DMG would have been the only build with an empty Plugins pane. Sharing the helper is the same pattern whisperkit-link-args.sh / sparkle-link-args.sh already use, for the same reason. `scripts/verify-plugins-binary.sh` guards package.sh and build-dmg.sh the way verify-whisperkit-binary.sh does. Plugins live OUTSIDE build.sh's OpenWhisp/ glob, so a broken source list drops every one of them with NO compile error — the app builds and runs perfectly with an empty pane. Verified non-vacuous: the guard fails on a PLUGINS=0 binary and skips when PLUGINS=0 is intentional. 2. DEVELOPER SURFACES ARE COMPILED OUT OF CONSUMER BUILDS `MemeTrace`'s emission and the two launch-gated probes in AppMain (plus the window controller's probe hooks) are now behind `OPENWHISP_INSTRUMENTATION`, the gate `LLMBenchRunner` and `LLMLabView` already use. An env var that opens a window and drives a generate is a fine debugging tool and a poor thing to leave reachable in a signed, notarized app holding Accessibility and mic grants. Verified in both directions rather than asserted: `[MemeGen]` appears 0 times in a default binary's strings and 1 time in an INSTRUMENTATION=1 one. The pure LINE BUILDERS stay compiled — they are `swift test`-covered functions and the core target has no instrumentation define. Only the side effect is conditional. `scripts/meme-runtime-proof.sh` was referenced by a doc comment but never existed; it now does, alongside the voice-command harness it mirrors. 3. MAK-100 CONTRACTS (all additive, all forward-compatible) - `clipboardAccess` (bool, default false) — WIRED, not just declared. The host reads the pasteboard only for a plugin that declared it, guarded by `needsPasteboard` BEFORE the read, so an undeclared plugin causes no NSPasteboard access at all. The rule lives in the pure, tested `PluginInvocationContext`; the pane discloses it beside the network hosts. The meme plugin declines it deliberately. - `destination` (enum, default ownWindow) — `cursor` / `outputTarget` are RESERVED: validated and reported, never fatal, and `effectiveDestination` falls back in one place so a declared-but-unimplemented route is refused honestly instead of silently doing something else. - `appAffinity` ([String]) — reserved router metadata. A HINT, never a priority: a test pins that declaring it buys a plugin no ranking advantage, because MAK-100's ~15-tool cap makes trigger surface a host-arbitrated resource. An unknown `destination` decodes to the default rather than throwing, so a manifest written for a future host degrades instead of vanishing from the list. Also swept spike-era provenance markers from doc comments and the user-facing "this prototype" strings, and corrected the `PLUGINS=1` references that inverted meaning when the flag became default-on. Gates: `swift test` 2685 passing (2670 carried + 15 new contract tests) · ratchet 7024/7024 · default / PLUGINS=0 / lean / INSTRUMENTATION=1 builds all compile · zero warnings in any plugin-owned file. Co-Authored-By: Claude <noreply@anthropic.com>
DOCS `docs/PLUGINS.md` — the reference the plugin system didn't have: architecture (core/app split, the provider seam, the three window seams), a field-by-field manifest schema including the new contract fields, how to write an in-repo plugin today, the security and trust model, and the path to hot-swappable. Two sections carry decisions rather than description: - **Security and trust.** Why plugins are runtime-opt-in, why providers are ordered by descending trust (a user-writable directory must never shadow a reviewed plugin in an app holding Accessibility + mic + clipboard rights), and the honest limits — `networkHosts` is disclosure not a sandbox, and the clipboard gate is real at the host but cannot bind an in-process plugin. Enforcement only becomes meaningful at a process boundary. - **Path to hot-swappable.** The owner's shipping requirement is installing a plugin without a rebuild, so this is written as a committed roadmap rather than an open question: manifest/script-driven plugins FIRST (cheapest, and delivers install-without-rebuild on its own — `ScriptPostProcessor` and `ConfigPack` are already this shape), out-of-process executables NEXT (real isolation, and it reuses two precedents this app already ships: helper binaries at Contents/Helpers and the Agent Bridge's local-socket protocol), WKWebView as an optional UI layer, and dylibs permanently rejected per ROADMAP §6. Also documents the testing trap the system is shaped around: `plugins/` is outside the `swift test` target, so a test that re-implements the app's sequence proves nothing about the chain — extract the decision into one core function and have both the app and the test call it. README gains a plugin line and a docs link; ARCHITECTURE gains the `plugins/` row; ROADMAP §6 gets a status banner and two table rows (v1 shipped, out-of-process next), with the dylib rejection left exactly as it was. CI Two jobs now pin OPPOSITE sides of the PLUGINS flag, which is the point: - `build-app` is pinned to `PLUGINS=0`. Without the flag it would have silently started building WITH plugins when the default flipped, and the lean escape hatch would have rotted untested. - `build-app-plugins` (new) builds the DEFAULT configuration users actually get, then runs `verify-plugins-binary.sh`. The build succeeding is not evidence the plugins linked — a broken source list yields a working app with an empty Plugins pane and no error anywhere — so CI asserts the symbols instead. The nightly full-fat `package.sh` job picks up the plugins and the new verify guard automatically; its comment now says so. Co-Authored-By: Claude <noreply@anthropic.com>
Comment on lines
67
to
+95
| @@ -82,8 +89,51 @@ jobs: | |||
| with: | |||
| path: build/sparkle | |||
| key: sparkle-${{ runner.os }}-${{ hashFiles('scripts/fetch-sparkle.sh') }} | |||
| - name: Build mac app (lean, WHISPERKIT=0 PARAKEET=0) | |||
| - name: Build mac app (lean, WHISPERKIT=0 PARAKEET=0 PLUGINS=0) | |||
| run: WHISPERKIT=0 PARAKEET=0 PLUGINS=0 ./build.sh | |||
|
|
|||
| build-app-plugins: | |||
| # Compiles the app WITH the in-repo plugins — the DEFAULT configuration, and | |||
| # therefore the one users actually get (docs/PLUGINS.md). | |||
| # | |||
| # Separate from `build-app` rather than folded into it because the two jobs | |||
| # pin opposite sides of the same flag: that job proves PLUGINS=0 still | |||
| # compiles (the lean escape hatch), this one proves the shipped default does. | |||
| # Collapsing them into one build would leave whichever side wasn't chosen | |||
| # untested, and the plugin surface is exactly the kind of code that breaks | |||
| # without anyone noticing — it lives outside build.sh's OpenWhisp/ glob, so | |||
| # dropping it is not a compile error. | |||
| # | |||
| # Stays lean on the ENGINES (WHISPERKIT=0 PARAKEET=0) for the same reason | |||
| # build-app does: the engines are irrelevant to whether the plugin layer | |||
| # type-checks, and skipping them keeps the job to minutes instead of an hour. | |||
| name: build mac app (with plugins) | |||
| runs-on: macos-14 | |||
| timeout-minutes: 15 | |||
initcore0
added a commit
that referenced
this pull request
Aug 4, 2026
…cratchpad editor (#245) Adds the 2026-08 release entry covering everything shipped after the 2026-07 entry (v1.0.19 → v1.1.0+177), and re-renders changelog.html from the JSON. 18 entries — 9 features, 6 fixes, 3 improvements — verified against `git log v1.0.18+173..v1.1.0+177`, docs/PLUGINS.md, and the manifest in PluginRegistry.swift rather than from PR intent. The lead story is the plugin system (#244 / MAK-100) split into three entries, because "there are plugins now", "the first one makes memes", and "you can drive it hands-free from refine" are three different things a reader needs: - Plugins are opt-in twice over — the pane defaults to nothing enabled, and the entry says so, along with the networkHosts disclosure being a label rather than a sandbox (the honest limit docs/PLUGINS.md states). - The meme entry discloses the only network use: api.imgflip.com, i.imgflip.com and api.memegen.link serve the blank template image; captioning and rendering are local, and imported templates need no network at all. memegen.link's server-side captioning is deliberately unused, and the entry doesn't claim capability the plugin declines. - The voice-command entry spells out both flows (selection vs. spoken remainder), the EN + RU triggers, and the strict prefix/word-boundary matching — the reason "create a memo about Q3" is never eaten. Remaining entries cover the Scratchpad quality editor (MAK-95/96/97), in-note find (#240), AI actions + file-transcript import (MAK-98/99), model-readiness UX (MAK-94), the stream-overlay counter (#232), and the fixes: vocabulary substitutions before translation (#238), ru/uk detection (#235/#238), app-wide ⌘V (#242), per-keystroke Scratchpad saves, "Model: Not checked", and the "in 0s" timestamp. Every `howTo.availability` is "live" — each was checked against the shipping code path, per the truth contract in docs/changelog/README.md. stats.testsPassing = 2685 is the measured `swift test` count on this tree, not a carried-forward number. changelog.html renders only the latest release (its established behavior), so July's entries move out of the reference render and stay canonical in the JSON. The <style> block and page skeleton are byte-identical; only the content under div.wrap is regenerated. Gates: swift test 2685 passing, 0 failures (docs-only change — untouched). Co-authored-by: Claude <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Ships the plugin system and the voice-driven meme generator from the
10-iteration spike in #243, brought to production quality.
Every behavior the owner tested in #243 is carried byte-identical. What
changed is everything around it: the build defaults, the developer-only
surfaces, the manifest contracts, CI, and the docs. #243 stays open for you to
close; nothing here is marked do-not-merge.
Provenance
The spike ran ten owner-tested rounds, each fixing something reported against a
running build — the confident-wrong-template fallback, the stuck spinner, the
day-long-uptime download failure, the four-items-became-two-captions bug, and
finally the voice-command route. The full iteration log, including the root
cause of each defect and what was deliberately not done, lives in #243 and
is worth keeping as the design record.
The spike branched from what is still
maintoday, so this is a cleancarry — no rebase conflicts, and nothing had to be dropped or reworked to
land it.
What's in it
Plugin system
Pure rules in
OpenWhispCore(allswift test-covered), IO and AppKit in theapp.
PluginManifest— the host/plugin contract. Id validation is strict becausethe id becomes a path component under Application Support.
PluginDiscovery— merges an ordered provider list; providers are passedin descending trust order and earlier wins, so a user-writable directory
can never shadow a reviewed plugin. The compile-time registry is one entry in
that list — the seam a real loader plugs into rather than replaces.
PluginEnablement— default-off, with pruning.PluginHost/PluginsPane/ menu-bar submenu — none of it onAppState.PluginWindowLifecycle,PluginDictationSink,PluginVoiceCommandSink.The meme plugin
Three merged template providers (imgflip, memegen.link, and the user's own
imported library — the only one that works offline and in any language), scored
search over names and keywords, per-template caption slots, schema-constrained
decoding, a normalized box model where the preview IS the export, an
idempotent ticket-guarded state machine, a rebuildable
URLSession, and abounded learned affinity.
Voice-command routing
With refine armed, an instruction starting with a manifest-declared phrase goes
to the plugin instead of the refine LLM. Both flows work: selection-as-material
and spoken-remainder-as-material. Matching is prefix-only, word-boundary, exact,
no fuzzy — a match redirects a dictation away from the user's editor, so a
false positive costs them text. Non-match, disabled plugin, or a window that
can't take it all fall through to the untouched normal refine.
What productionizing changed vs the spike
PLUGINS ships on by default
PLUGINSnow defaults to 1, matchingWHISPERKIT/PARAKEET/SPARKLE;PLUGINS=0is the lean escape hatch. Compiled in != enabled — plugins staydisabled at runtime until you enable one per-plugin in the pane.
The source list moved into
scripts/plugin-source-args.sh, shared bybuild.shand
build-dmg.sh.build-dmg.shis the release path and had no pluginsupport at all in the spike — left alone, the shipped DMG would have been the
only build with an empty Plugins pane.
scripts/verify-plugins-binary.shguardspackage.shandbuild-dmg.shtheway the WhisperKit/Parakeet guards do. Plugins live outside
build.sh'sOpenWhisp/glob, so a broken source list drops every one of them with nocompile error — the app builds and runs fine with an empty pane. Verified
non-vacuous: it fails on a
PLUGINS=0binary and skips when that wasintentional.
Developer surfaces compiled out of consumer builds
MemeTrace's emission and the launch-gated probes are now behindOPENWHISP_INSTRUMENTATION— the gateLLMBenchRunnerandLLMLabViewalreadyuse. An env var that opens a window and drives a generate is a fine debugging
tool and a poor thing to leave reachable in a signed, notarized app holding
Accessibility and microphone grants.
Verified in both directions rather than asserted:
[MemeGen]appears 0 timesin a default binary's strings, 1 time with
INSTRUMENTATION=1.The pure line-builders stay compiled — they're
swift test-covered and the coretarget has no instrumentation define. Only the side effect is conditional.
scripts/meme-runtime-proof.shwas referenced by a doc comment but neverexisted. It does now, mirroring the voice-command harness.
MAK-100 contracts — all additive, all forward-compatible
clipboardAccess(bool, default false)destination(enum, defaultownWindow)cursor/outputTargetreserved + validatedappAffinity([String])voiceTriggersClipboard is genuinely gated. The host reads the pasteboard only for a
plugin that declared it, and the check runs before the read — an undeclared
plugin causes no
NSPasteboardaccess at all. The rule lives in the pure,tested
PluginInvocationContextrather than at the AppKit call site where aforgotten
ifwould be invisible. The pane discloses it beside the networkhosts. The meme plugin declines it deliberately (its Cmd-V import is for
template images, an explicit user action).
Reserved routes are refused honestly. A manifest declaring
.cursorreports.unsupportedDestination, stays valid and runnable, andeffectiveDestinationfalls back in one place — rather than silently doing something the author didn't
ask for. An unknown
destinationdecodes to the default instead of throwing,so a manifest written for a future host degrades rather than vanishing.
appAffinityis a hint, never a priority. A test pins that declaring it buysno ranking advantage, because MAK-100's ~15-tool cap makes trigger surface a
host-arbitrated resource — a manifest must never self-assign priority, for the
same reason
networkHostsis a label.Docs
New
docs/PLUGINS.md: architecture, full manifest schema reference, how towrite an in-repo plugin, the security/trust model, and — as a committed
roadmap rather than an open question — the path to hot-swappable.
Hot-swap roadmap. The shipping requirement is installing a plugin without
rebuilding. In order: (1) manifest/script-driven plugins first, because it's
cheapest and delivers install-without-rebuild on its own (
ScriptPostProcessorand
ConfigPackare already this shape); (2) out-of-process executablesnext, the only option with real isolation, reusing two precedents this app
already ships — helper binaries at
Contents/Helpers/and the Agent Bridge'slocal-socket protocol; (3) WKWebView as an optional UI layer; (4) dylibs
permanently ruled out, per ROADMAP section 6, which this PR leaves exactly as
it was.
README gains a plugin line and a docs link, ARCHITECTURE gains the
plugins/row, and ROADMAP section 6 gains a status banner plus two table rows.
CI
Two jobs pin opposite sides of the same flag, which is the point:
build-appis now pinned toPLUGINS=0. Without the flag it would havesilently started building with plugins when the default flipped, and the
lean escape hatch would have rotted untested.
build-app-plugins(new) builds the default configuration users actuallyget, then runs the symbol verify. A green build is not evidence the plugins
linked.
The nightly full-fat
package.shjob picks up plugins and the new guardautomatically.
Testing
plugins/compiles only into the app and sits outside theswift testtarget, so the split matters. Which pure core covers what:
PluginManifestPluginDiscoveryPluginEnablementPluginKeyEquivalentPluginVoiceCommandRouterPluginInvocationContext+PluginManifestMemeCaptionSeedingMemeCaptionExtractionMemeAIMemeTemplateCatalog/MemeTemplateMatcherMemeGenerationStateMemeCatalogCacheMemeCaptionLayoutMemeTemplateAffinityVerified none regressed to the app layer during the carry. The decision
logic the spike moved into core is still in core:
MemeCaptionSeeding.resolvestill owns the whole captions-to-boxes decision and
applyRankedis still acall to it plus UI glue.
What remains app-layer and untestable by
swift test: window lifecycle, thenetwork round-trip, AppKit rendering, and the async refit. Those are what the
two runtime-proof harnesses exist for.
Gates
swift test— 2685 passing, 0 failures (2670 carried + 15 new contract tests)./build.sh(default, now with plugins) — passPLUGINS=0 ./build.sh— passWHISPERKIT=0 PARAKEET=0 SPARKLE=0— passPLUGINS=0(the CI lean job's exact config) — passINSTRUMENTATION=1 ./build.sh— passPLUGINS=0build vs 112 with plugins — the plugin sources add none)Not verified
Same honest caveat the spike carried: the Mac app can't be screenshotted in
this environment (the name resolves to the iOS bundle), so no UI was visually
confirmed, and no live LLM round-trip was run against this build. The runtime
proofs in #243 were run against the spike binary; the code they exercise is
carried unchanged here, but they were not re-run after the instrumentation
gating — re-running them now requires
INSTRUMENTATION=1 ./build.sh, which isthe one behavioral change to that path.
🤖 Generated with Claude Code