Skip to content

Keynote transition + build reverse-engineering and write backend - #1

Merged
leogdion merged 29 commits into
v0.1.xfrom
prototype
Jul 28, 2026
Merged

Keynote transition + build reverse-engineering and write backend#1
leogdion merged 29 commits into
v0.1.xfrom
prototype

Conversation

@leogdion

@leogdion leogdion commented Jul 28, 2026

Copy link
Copy Markdown
Member

Brings the full keynote-format-lab prototype onto the v0.1.x release line: 27 commits reverse-engineering how Keynote's .key format represents slide transitions and object builds, plus a working write backend.

What's here

Phase 0 — toolchain. Validated unpack, and fixed pack, which the stock keynote-parser could not do for Keynote 15.3. Option A pairs the installed 14.4 TSPRegistryMapping with regenerated 15.3 protos; type IDs proved stable, so the LLDB runtime dump was never needed. Vendored as a reproducible schema (mise run prepare-keynote-parser, 631/631 registry entries).

Phase 1 — transitions. Established that transitions live at KN.SlideArchive.transition.attributes.animationAttributes; effect: none means off. Catalogued and verified all 43 effects, plus their per-effect custom* option bags. Magic Move correspondence is not persisted — it's a runtime heuristic, so magic-id must be an authoring-time abstraction (confirmed from two angles).

Phase 2 — object builds (Exps 5–11). Builds are a KN.BuildArchive + KN.BuildChunkArchive pair referenced from the slide; order is list position, with no explicit order field. In/Out/Action are one animationType enum, Action carrying a distinct motion-path payload. The character effect suffix is not object-type-qualified. Builds also serialize direction — the same slot transitions omit at default.

Backends. Transitions are fully AppleScript-scriptable (deckkit.py + build_deck.py). Builds and direction are not scriptable, so they go through YAML archive surgery (archive_backend.py) on the repacked deck.

The crash fix

Authored builds crashed Keynote 15.3 with EXC_BREAKPOINT (SIGTRAP) in -[__NSSetM addObject:]. A four-deck bisect isolated it to build authoring, and comparison against human-authored fixtures found two missing document-level invariants in Metadata.iwa.yaml:

  1. Every KN.BuildArchive id must be registered in TSP.PackageMetadata → the slide's component → objectUuidMapEntries, with the entry's uuid equal to the chunk's buildId (8/8 human-authored builds register it; 0/2 of ours did).
  2. lastObjectIdentifier must stay above every minted archive id (5/5 fixtures hold this; ours inverted it).

Neither is visible in the decoded slide YAML, which is why structural verification passed while Keynote kept crashing. _verify_uuid_map now enforces both from inside write_back.

Verification

  • mise run test → 18 tests, no Keynote required; both crash fixes mutation-tested
  • mise run selftest, mise run verify-pack → pass
  • Human reopen confirmed: all four isolated decks and the combined acceptance deck open in Keynote 15.3 with no crash and no repair warning. Newly authored In, Out, Action, multiple ordered builds, and transition direction all survive the round trip.

Known gaps

Build triggers other than on_click (after_previous / with_previous need fixtures), the apple:sidezoom alias reconciliation flagged in build_catalog.md, Exp 9's catalog rows not yet wired into deckkit.py, the full direction enum (two values known), and item:N targets only. All tracked in findings/write_backend.md and findings/HANDOFF.md.

Note on scope

This is the research prototype — Python tooling, findings docs, and fixtures. findings/HANDOFF.md is the entry point for picking the work back up.

🤖 Generated with Claude Code

leogdion and others added 27 commits July 17, 2026 16:44
@leogdion @cursoragent
Set up harness runtime and record Phase 0 + Phase 1 transition findings
87045b4 
Phase 0: mise Python 3.12 + venv, keynote-parser 1.14.4.0 (pinned), mise tasks,
and toolchain validation against Keynote 15.3 (unpack works; pack does not
round-trip due to the 15.3-vs-14.4 mapping gap — documented, not blocking).

Phase 1 (transitions, fully scripted): transition attaches to the slide
archive's animationAttributes; effect is a string enum matching the sdef Cocoa
values; duration/delay are verbatim seconds; auto-advance is just isAutomatic;
and Magic Move object correspondence is NOT persisted (runtime heuristic), so
magic-id must be a compile-time abstraction (confirmed via shapes and
duplicate-and-edit).

Adds normalizer object-id canonicalization, transition generators, Phase 2
builds base/control fixtures + steps, a Deck model synthesis, and HANDOFF.md.

Co-authored-by: Cursor <cursoragent@cursor.com>
@leogdion @cursoragent
Sweep transition effect matrix; model per-effect custom attributes
5dfc16e 
Built one deck per representative effect (dissolve/wipe/move-in/iris/object-cube/
object-flip/switch) and extracted each transition block. All effect strings
match the sdef Cocoa values. Effect-specific knobs are sparse custom* siblings
of animationAttributes: only object flip emits customBounce; magic move emits
its three; the rest emit none. No direction is serialized at default.

Updates effect_type.md, deck_model_notes.md (transition options = extensible
per-effect bag), and HANDOFF.md. Adds effect_matrix + one_effect generators.

Co-authored-by: Cursor <cursoragent@cursor.com>
@leogdion @cursoragent
Verify per-slide transition independence
54b2423 
A 3-slide deck with a different effect per slide stores each effect in its own
slide archive (no document-level list, no coupling), and slide 1 carries a
transition like any other slide. Confirms Transition is a per-Slide field.
Adds multi_slide generator + findings; updates deck_model_notes.

Co-authored-by: Cursor <cursoragent@cursor.com>
@leogdion @cursoragent
Add transition backend: Deck IR -> AppleScript -> .key, with verify
b29a2b2 
Implements the scriptable half of the backend end-to-end:
- tools/deckkit.py: Deck/Slide/TextItem/Transition IR, JSON spec loader, an
  EFFECTS map (name -> archive string + AppleScript enumerator), AppleScript
  code-gen that drives Keynote to build the .key (per-slide transitions incl.
  slide 1), and a round-trip verifier.
- tools/build_deck.py: CLI (build + optional --verify via keynote-parser unpack).
- examples/deck_example.json: 4-slide sample. Verified end-to-end (VERIFY PASS).

No pack dependency. Builds/direction (non-scriptable) remain future work.
Updates deck_model_notes.md and HANDOFF.md.

Co-authored-by: Cursor <cursoragent@cursor.com>
@leogdion @cursoragent
Stop tracking __pycache__; ignore *.pyc
bc5ffc5 
Untrack the accidentally-committed compiled module and add __pycache__/ + *.pyc
to .gitignore.

Co-authored-by: Cursor <cursoragent@cursor.com>
@leogdion @cursoragent
Add no-Keynote unit tests for the Deck backend + mise tasks
05a4444 
tools/test_deckkit.py exercises spec parsing, AppleScript code-gen (terms,
quote-escaping, per-slide transition count), and the verify/extract logic on a
synthetic unpacked deck (positive + negative). Adds mise tasks `test` and
`build-example`. Both `mise run test` and `mise run selftest` pass.

Co-authored-by: Cursor <cursoragent@cursor.com>
@leogdion @cursoragent
Catalog all 43 transition effects; expand backend to full effect set
84a12a5 
Extends deckkit.EFFECTS to the complete Keynote 15.3 transition enum (43
effects + none) and verifies every archive effect string round-trips by
building a 43-slide catalog deck (examples/effect_catalog.json) and running
build_deck --verify (43/43 PASS).

Findings from the sweep:
- radial wipe's archive string is `apple:radial wipe` (contains a space; the
  raw sdef value is misleading). Fixed the verifier's effect regex to accept
  spaces.
- Per-effect custom* options catalogued: customBounce (bool) on object flip/
  object revolve/cube/flip/scale/revolving door; customTravelDistance (fade and
  move); customTwist (twist); magic move's three. All others emit none, and none
  are AppleScript-settable.

Updates effect_type.md, deck_model_notes.md, HANDOFF.md, and extends the
no-Keynote unit test (EFFECTS completeness + space-in-effect regression).

Co-authored-by: Cursor <cursoragent@cursor.com>
@leogdion @cursoragent
Add Phase 2 build experiment fixtures (Exps 5-7)
6ae546d 
Human-authored golden pairs for build-in vs none, build order, and build
effect/timing sweeps, ready for differential diffing.

Co-authored-by: Cursor <cursoragent@cursor.com>
@leogdion @claude
Reverse-engineer object builds (Phase 2, Exps 5-7)
4a79ab4 
Diff the hand-authored build fixtures (build_in/order/fx) and document how
Keynote's .key stores object builds:

- A build = KN.BuildArchive (effect in a transition-shaped animationAttributes,
  targeting its object by drawable.identifier) + KN.BuildChunkArchive (timing),
  referenced from new builds/buildChunks lists on KN.SlideArchive. Document
  flips hasExplicitBuilds -> true (derived cache).
- Order is list position; no explicit order field (Exp 6).
- Build effects: apple:dissolve character / apple:move in character (distinct
  from transition strings). Move In carries animationAttributes.direction (int)
  -- the same field transitions omit at default.

deckkit.py: add the Build IR (builds on Slide), BUILD_EFFECTS, spec loading,
and the read/verify half (extract_builds/verify_builds). No build write path --
builds are not AppleScript-scriptable (byte-surgery, future). extract_transitions
now filters by animationType so builds aren't miscounted; transition round-trip
still VERIFY PASS.

findings: build_in.md (Exp 5), build_order.md (Exp 6), build_fx.md (Exp 7);
deck_model_notes.md gains a Build {} section; HANDOFF.md marks Phase 2 done.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@leogdion @claude
Add agent-skills config (AGENTS.md + docs/agents)
d164de1 
Scaffold per-repo config for the Matt Pocock engineering skills:
- AGENTS.md with an "Agent skills" section
- docs/agents/issue-tracker.md — GitHub issues via gh CLI
- docs/agents/domain.md — single-context domain-doc consumer rules

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@leogdion @claude
Track corrections log in-repo
ef7f820 
Move the append-only corrections log and its loader instruction from global
~/.claude into the repo so they are versioned and committed:
- CORRECTIONS.md at the repo root (append-only source of truth)
- AGENTS.md gains a "Corrections log" instruction pointing at it

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@leogdion @claude
Move corrections log to .claude/
82cf088 
Relocate CORRECTIONS.md from the repo root to .claude/ (preserves history via
git mv); update the AGENTS.md pointer to .claude/CORRECTIONS.md.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@leogdion @claude
Add Phase 2b fixture instructions + generators (Exps 8-11)
01f4f10 
Stage the next human-in-the-loop fixtures for features Keynote won't script:
- findings/builds_setup_2.md — per-experiment steps for Exp 8 (non-text/shape
  build, settles the "character"-suffix question), Exp 9 (build-effect catalog),
  Exp 10 (Build Out / Action), Exp 11 (transition direction).
- generators/base_shape_build.applescript — base with one shape (verified: emits
  a Shape archive, no builds).
- generators/direction_base.applescript — 2-slide deck with a scripted Move In
  transition at default direction (verified: slide 2 = apple:slide, Transition).

Downstream diff/analysis stays unattended once the fixtures are saved.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@leogdion @claude
Test the build read/verify path in deckkit
e6c7598 
Add dependency-free unit tests (synthetic unpacked decks, no Keynote/keynote-parser):
- test_parse_builds / test_unknown_build_effect_and_kind — Build spec loading + validation
- test_build_effects_map — BUILD_EFFECTS archive strings; no AppleScript term
- test_extract_and_verify_builds — extract_builds fields (incl. direction/drawable)
  + verify_builds positive/negative
- test_extract_transitions_excludes_builds — the animationType filter routes
  Transition vs In blocks correctly

mise run test -> DECKKIT TEST PASS (11 tests); mise run selftest -> SELFTEST PASS.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@leogdion @claude
Add pack-mapping feasibility spike + refresh HANDOFF
b3f87c5 
findings/pack_mappings.md: bounded spike on regenerating keynote-parser's 15.3
pack mappings. The proto-definition half runs unattended today (33 15.3 protos
extracted via the source's pure-Python protodump.py — no C++ proto-dump needed —
and compiled with protoc; TSKArchives.sos dropped vs 14.4, proving a real schema
shift). Blocked on the TSPRegistryMapping (archive-type-ID -> message-name), which
is a runtime LLDB dump of [TSPRegistry sharedRegistry] and needs human/toolchain
setup (Homebrew LLVM w/ matching Python + debuggable re-sign of the sandboxed MAS
app). Nothing installed/committed beyond the note; experiments stayed in scratch.

HANDOFF.md: point build follow-ups at builds_setup_2.md, note the unit tests, and
record the pack-spike outcome; refresh the findings index + generator list.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@leogdion @claude
Document the pack write-backend approach + glossary
cfa105a 
Decision-support note (companion to pack_mappings.md/versions.md) explaining why a
byte-surgery backend is needed for builds/direction, how pack works and why it's
broken for 15.3, the TSPRegistryMapping blocker, and three options (A: reuse the
14.4 registry + round-trip test — cheap/mostly unattended; B: full LLDB runtime
dump — heavy, needs SIP/re-sign; C: defer) with an at-a-glance comparison. Includes
a full glossary of the IWA/protobuf/codesign/LLDB terminology.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@leogdion @claude
Add read_builds CLI + capture build options (Exp 9 prep)
549921f 
Prep tooling for the incoming build experiments (esp. the Exp 9 effect catalog):
- deckkit.extract_builds now also returns an `options` bag per build
  (customBounce / customTravelDistance / customTextDelivery /
  customDeliveryOption / delivery / eventTrigger), the sparse bag Build.options
  models; scalars coerced (bool/int/float/str), space-containing `delivery`
  preserved, BuildArchive.duration excluded.
- tools/read_builds.py: CLI that unpacks one+ .key files and prints their builds
  (kind/effect/timing/direction/drawable + options), --json supported.
- mise task `read-builds`.
- test_extract_build_options covers the options capture.

Verified against committed fixtures: build_fx_B -> apple:move in character,
direction 13, customBounce, delivery By Paragraph; build_in_B -> the dissolve
build. mise run test -> DECKKIT TEST PASS (12); selftest PASS.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@leogdion @claude
Log corrections: in-repo .claude location + unattended-pieces meaning
e35640d 
Append two entries to the append-only corrections log.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@leogdion @claude
Add Exp 8-11 human-in-the-loop fixtures (builds + direction)
25b02cb 
Adds the completed .key fixtures for builds_setup_2.md Exps 8-11:
- Exp 8: build_shape_A/B (shape Dissolve build)
- Exp 9: build_cat control + 9 Build-In effect variants
  (appear, dissolve, movein=Fly In, fadein=Fade and Scale,
   fademove, scale, shiftscale=Shift and Scale, blur, flip)
- Exp 10: build_out_A/B, build_action_B
- Exp 11: direction_A/B

Also updates the Exp 9 effect table to match the actual Keynote
Build In menu (Fly In, Fade and Scale, added Shift and Scale row).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@leogdion @claude
Exp 8-11 findings + Option A pack success; mirror memories into repo
5330fd3 
Phase 2b experiments (fixtures authored by hand, analyzed unattended):
- Exp 8 (build_shape): the " character" effect suffix is NOT
  object-type-qualified — a shape's Dissolve = apple:dissolve character too.
- Exp 9 (build_catalog): 8-effect catalog across three naming families;
  direction/customBounce/customTravelDistance options captured.
- Exp 10 (build_out): In/Out/Action are one animationType enum on the same
  KN.BuildArchive; Action is a distinct payload (motion path, no text delivery).
- Exp 11 (direction): transition direction = animationAttributes.direction (int),
  the same slot builds use.

Option A (pack write-backend): ACCEPTED. Pairing the installed 14.4
TSPRegistryMapping with regenerated 15.3 protos packs a .key Keynote 15.3 opens
(regression_check.key, 4 slides). Type IDs are stable 14.4->15.3, so the LLDB
runtime dump (Option B) is not needed. See findings/pack_option_a.md.

Rolled findings into deck_model_notes.md + HANDOFF.md; marked
pack_backend_approach.md section 5 RESOLVED.

Also mirror the auto-memory set into .claude/memory/ so memories are
version-controlled with the repo. Raw findings/<exp>/ diff dirs left untracked,
per existing convention (regenerable).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@leogdion
Document build backend authorization and handoff
b71089a
@leogdion
Vendor reproducible Keynote 15.3 parser schema
3115a86
@leogdion
Implement Keynote build archive write backend
dac1816
@leogdion
Document build backend verification blocker
ec96b4c
@leogdion
Record Keynote animation crash evidence
56f6216
@leogdion
Update build backend status and next steps
dd0ff04
@leogdion @claude
Fix authored-build crash: register builds in the package UUID map
7cce82b 
Authored decks containing any object build crashed Keynote 15.3 with
EXC_BREAKPOINT (SIGTRAP) in -[__NSSetM addObject:]. A 4-deck bisect
(examples/bisect_*.json) isolated it: a direction-only deck reopens, while
one Build In, one Build Out, and one Action each crash — so the pack path,
write_back surgery, and hybrid parser were all sound.

The decoded slide YAML was equivalent to the human-authored fixtures
throughout; the missing invariants were document-level, in Metadata.iwa.yaml:

1. Every KN.BuildArchive id must be registered in TSP.PackageMetadata ->
   the slide's component -> objectUuidMapEntries, with the entry's uuid
   equal to the KN.BuildChunkArchive's buildId. Measured 8/8 human-authored
   builds registered, 0/2 of ours; chunk ids are registered in 0/8.
2. lastObjectIdentifier must stay strictly above every minted archive id.
   5/5 fixtures hold this; ours inverted it.

Both fit the crash frame — a set insertion failing on an identity Keynote
cannot resolve while rebuilding its animation registry on load — and both
are absent from the direction-only deck, which allocates no ids.

archive_backend now registers one entry per emitted build and bumps the
high-water mark only when ids were allocated, leaving the working
direction-only path untouched. _verify_uuid_map enforces both from inside
write_back, since deckkit.extract_builds reads only Index/Slide*.iwa.yaml
and structurally cannot observe them.

Human reopening confirms all four bisect decks and the combined acceptance
deck now open with no crash and no repair warning: newly authored In, Out,
Action, multiple ordered builds, and transition direction all survive.

Also generalizes authored_build_smoke.py to take spec/output arguments with
a spec-derived reopen assertion (no-arg behavior unchanged), and adds
--no-reopen for generating diagnostic decks without driving Keynote.

Tests: 18 pass (4 new, over a synthetic unpacked tree); both fixes were
mutation-tested. mise run test / selftest / verify-pack all pass.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Jul 28, 2026 •
edited
Loading

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: dff35aea-2176-40ab-b349-90941f6eface

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch prototype

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

leogdion and others added 2 commits July 28, 2026 15:15
@leogdion @claude
Move research content under research/ for Swift Package layout
c31276a 
Frees the repo root for Package.swift, Sources/, and Tests/ by relocating
all reverse-engineering content into research/: tools/, findings/,
fixtures/, generators/, examples/, vendor/, plus the gitignored working
directories samples/, unpacked/, and build/.

Pure relocation — git detects all 282 files as renames with no content
change. The Python tools resolve their paths from __file__ via
Path(...).parents[1], so moving the tree as a unit keeps every internal
reference correct with no code change.

Updated:
- mise.toml task paths, including the two tasks passing relative paths as
  arguments (build-example, transitions)
- .gitignore, anchoring research/samples/, research/unpacked/, and
  research/build/keynote-parser/ (the last was root-anchored and would
  otherwise have stopped matching)
- README.md layout section and tool references

Verified after the move: the hybrid parser cache rebuilds from scratch to
the identical hash (86ee8d81), confirming the vendored schema inputs are
byte-identical; mise run test (18) / selftest / verify-pack all pass; and
end-to-end authoring regenerates a build deck through the full
unpack -> author -> pack -> verify path.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@leogdion @cursoragent
Replace CORRECTIONS.md with agent-notes and track installed agent ski…
778ff58 
…lls.

Consolidate standing directives into a supersedable in-repo log and document the convention in AGENTS.md so future sessions read one source of truth.

Co-authored-by: Cursor <cursoragent@cursor.com>
@leogdion
leogdion merged commit ab23b45 into v0.1.x Jul 28, 2026
1 check passed
@leogdion
leogdion deleted the prototype branch July 28, 2026 20:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@leogdion