refactor: extract pdf-to-latex-swift + ocr-swift to PsychQuant repos; commit Tests/WordToMDTests fixtures (continues #78 pattern)

[kiki830621]

Problem

From verification of #78 (#78 verify report, finding #3):
「Macdoc 的 clean-clone swift build 失敗 on HEAD because packages/pdf-to-latex-swift/ and Tests/WordToMDTests/ are gitignored/untracked (same pattern #78 fixed for note-* packages, applied to other paths).」
— Source: codex independent verifier, 2026-04-18

#78 successfully extracted the 3 note-* packages to independent PsychQuant/* repos so they could be properly version-controlled. That fix did not generalize to other gitignored/untracked paths in macdoc:

• packages/pdf-to-latex-swift/ — declared as path: in Package.swift but matched by .gitignore blanket packages/ rule (no whitelist entry). Source files exist only on the developer's filesystem.
• packages/ocr-swift/ — same pattern (just discovered while reviewing refactor: extract NoteCore / NoteToPDF as separate PsychQuant repos (packages 應該要是獨立套件) #78's PageOCRRunner.swift workstream)
• Tests/WordToMDTests/ — referenced by macdoc's Package.swift .testTarget, but not tracked in git (git ls-tree HEAD Tests/ shows only MacDocCLITests and OCRTests)

A fresh git clone PsychQuant/macdoc.git && swift build would fail or silently use stale packages/note-*-swift/ directories that an existing developer machine already has.

Type

refactor

Expected

• Each currently-gitignored package under packages/* either:
  • (a) extracted to its own PsychQuant/* repo with a v0.1.0 tag (same as refactor: extract NoteCore / NoteToPDF as separate PsychQuant repos (packages 應該要是獨立套件) #78's approach for note-*), or
  • (b) explicitly whitelisted in .gitignore and committed in-tree (same as srt-to-html-swift, md-to-html-swift, etc).
• Tests/WordToMDTests/Fixtures/* either added to git or moved out of the macdoc repo.
• A clean git clone … && swift build succeeds without any local-disk pre-state.

Actual

git -C macdoc ls-tree HEAD packages/  # missing pdf-to-latex-swift, ocr-swift, note-* (note-* is intentional after #78)
git -C macdoc ls-tree HEAD Tests/     # missing WordToMDTests
git -C macdoc check-ignore -v packages/pdf-to-latex-swift  # → matched by `packages/` rule

Strategy

Mirror #78's phased approach:

• Phase 1 — pdf-to-latex-swift → new PsychQuant/pdf-to-latex-swift repo, tag v0.1.0, swap path: → url: in macdoc Package.swift — Track A; shipped in 9170093 (part of PR refactor: extract pdf-to-latex-swift + ocr-swift + flip gitignore strategy (#79) #89 squash)
• Phase 2 — ocr-swift → new PsychQuant/ocr-swift repo, tag v0.1.0 — Track B; shipped in 3a2806a (part of PR refactor: extract pdf-to-latex-swift + ocr-swift + flip gitignore strategy (#79) #89 squash); PR fix: PageOCRRunner uses OCRCore.backend API (unblocks main build) #84 PageOCRRunner compiles against new url dep
• Phase 3 — Tests/WordToMDTests/Fixtures/* decision: delete (dir was completely empty scaffolding) — Track C; shipped in 6fd4880 (part of PR refactor: extract pdf-to-latex-swift + ocr-swift + flip gitignore strategy (#79) #89 squash)
• Phase 4 — Clean-clone verification executed: fresh git clone && swift build succeeds in 95.80s; swift test runs 26 Swift Testing tests clean (2 XCTest NoteConvertTests XCTSkip per test: add NoteConvertTests smoke coverage (.note → pdf/html) to MacDocCLITests #81 Option B, expected)
• Phase 5 — Audit surfaced 6 more packages with missing sources (md-to-word, word-to-html, html-to-word, pdf-to-md, pdf-to-docx, bib-apa-swift + 3 bib-apa-to-*); all committed in-tree via .gitignore whitelist expansion in a380cfc (part of PR refactor: extract pdf-to-latex-swift + ocr-swift + flip gitignore strategy (#79) #89 squash); see PR refactor: extract pdf-to-latex-swift + ocr-swift + flip gitignore strategy (#79) #89 body for full list

Related

• Refs refactor: extract NoteCore / NoteToPDF as separate PsychQuant repos (packages 應該要是獨立套件) #78 (sibling refactor, completed for note-* only)
• Blocks any future "new contributor onboarding" since clean-clone build currently doesn't succeed

---

Current Status

Phase: closed
Last updated: 2026-04-20 by idd-close (retroactive)

Key Decisions

• Closed retroactively 2026-04-20 — PR refactor: extract pdf-to-latex-swift + ocr-swift + flip gitignore strategy (#79) #89 used Closes #79. trailer which auto-closed on merge, bypassing idd-close gate check. Retroactive Closing Summary posted to preserve audit trail.
• 35/35 Spectra tasks done; all 5 body Strategy phases done; clean-clone git clone && swift build verified 95.80s (from zero pre-state)
• Scope expansion: Track D audit surfaced 6 MORE packages with missing sources — fixed in-scope via extended .gitignore whitelist (48 files / 9,059 insertions in commit a380cfc)
• Track A found PsychQuant/pdf-to-latex-swift already existed from earlier aborted extraction; diff -rq confirmed byte-identical → reconciliation collapsed to tag v0.1.0 + swap dep
• Track B created PsychQuant/ocr-swift v0.1.0; verified PR fix: PageOCRRunner uses OCRCore.backend API (unblocks main build) #84's Qwen3-VL/OCRBackend wiring still compiles
• Track C deleted empty WordToMDTests testTarget (0 Swift files, dead scaffolding)
• All pre-existing dep pins preserved (note-core@0.1.3, note-to-pdf@0.1.2, word-builder@0.9.0, ooxml@0.7.0)
• Complexity: SDD-warranted → discuss (5 assumptions correct after diff-verify) → propose → apply → retroactive close
• Spectra change extract-remaining-packages ready for /spectra-archive

Scope Changes

• Original scope: 2 packages + WordToMDTests decision
• Actual scope: 2 remote extractions + testTarget delete + 6-package systemic in-tree rescue + CLAUDE.md docs = much larger than diagnosed
• .gitignore "flip strategy" attempted mid-apply but didn't reach final merged commit — functionally equivalent, just verbose

Blocking

• (none — closed)
• Known follow-up (not blocking): .gitignore per-file whitelist is now 80+ lines; could simplify with packages/*/.build/ exclude-only pattern

Commits (squashed into d181090 via PR #89)

• 9170093 Track A — pdf-to-latex-swift url dep
• 3a2806a Track B — ocr-swift extraction to PsychQuant/ocr-swift
• 6fd4880 Track C — delete empty WordToMDTests testTarget
• a380cfc Track D — extend .gitignore whitelist + track 48 previously-missing package sources
• fe1f2b9 Track E — CLAUDE.md Sub-Repositories table update

New Repos

• https://github.com/PsychQuant/ocr-swift v0.1.0 (Track B)
• https://github.com/PsychQuant/pdf-to-latex-swift v0.1.0 (Track A)

[kiki830621]

Diagnosis

Type

refactor (systemic — continues the extraction pattern #78 opened)

Current state (verified 2026-04-19)

Path	Local state	Remote state	Gap
packages/pdf-to-latex-swift/	556 KB source (10 .swift files in PDFToLaTeXCore), no .git, mtime 2026-03-15	PsychQuant/pdf-to-latex-swift already exists, pushed 2026-04-16, has both PDFToLaTeXCore + PDFToLaTeXCLI targets, no tags	Partial extraction already started; remote is newer; macdoc Package.swift:15 still path: dep
packages/ocr-swift/	32 KB source (5 .swift files: OCRPipeline, OCRBackend, OllamaBackend, MLXBackend, PDFKitExtractor), no .git	No remote repo	Clean slate (like note-* was)
Tests/WordToMDTests/	Empty directory (Fixtures/ exists but 0 files)	—	Referenced by Package.swift:60-64 testTarget but has no test files

Analysis — refactor goals

Each of the three has different character; diagnosis needs to treat them separately.

(1) pdf-to-latex-swift — partially extracted

• The PsychQuant/pdf-to-latex-swift repo already exists (pushed 2026-04-16, presumably from an earlier aborted extraction attempt)
• Remote has an extra PDFToLaTeXCLI target that local doesn't — suggests the remote is the more-complete source-of-truth
• Local files haven't been touched since 2026-03-15 — stale
• No tag yet → macdoc can't switch to url: dep
• Consumers in macdoc: MacDoc+PDF+Phase2.swift, MacDoc+OCR.swift, (indirect via MacDoc+Config)

Resolution: reclaim remote as SoT (use remote sources, delete local, clone if needed for active dev), tag v0.1.0, switch macdoc to .package(url:). Diff local vs remote first to ensure no loss.

(2) ocr-swift — fresh extraction (#78-style)

• No remote repo exists
• Local source is small (5 files, ~24 KB) but high-consumer-web (4+ macdoc CLI files + ooxml-swift dependencies via OCRPipeline)
• Recent consumer added in PR fix: PageOCRRunner uses OCRCore.backend API (unblocks main build) #84 (897682b on main) — PageOCRRunner uses MLXBackend + OllamaBackend, which live in this package
• Coordination needed: the Qwen3-VL refactor (fix: PageOCRRunner uses OCRCore.backend API (unblocks main build) #84) is already merged and depends on this path

Resolution: classic #78 playbook — git init + create repo + push + tag + swap macdoc dep. Zero surprises expected.

(3) Tests/WordToMDTests/ — decision, not mechanical work

• Directory exists but is completely empty (no .swift files, no fixtures)
• Package.swift:60-64 declares it as a testTarget with resources: [.copy("Fixtures")]
• Two valid resolutions:
  • (3a) Delete: remove the testTarget declaration from macdoc Package.swift since no tests actually exist. Simplest. Matches reality.
  • (3b) Populate: write minimal placeholder tests + fixtures so the testTarget has substance. Higher-effort, lower-value (tests without purpose).

Impact

• Affected: macdoc/Package.swift (2 deps swapped path→url + 1 testTarget resolved), packages/pdf-to-latex-swift/ (reclaimed or discarded), packages/ocr-swift/ (git init + push), Tests/WordToMDTests/ (deleted or populated)
• Risk: fix: PageOCRRunner uses OCRCore.backend API (unblocks main build) #84's OCRCore API changes live in this repo's source; if we extract to a new repo, we need to preserve the Qwen3-VL refactor state that just landed on main
• Unblocks: clean-clone build for new contributors (fixes the systemic gap codex flagged in refactor: extract NoteCore / NoteToPDF as separate PsychQuant repos (packages 應該要是獨立套件) #78 verify)

Strategy

Mirror #78's proven 6-phase approach, split into 3 mostly-independent tracks:

Track A — pdf-to-latex-swift (medium risk; remote already exists)

• A.1 Clone PsychQuant/pdf-to-latex-swift to a temp dir; diff remote HEAD against local packages/pdf-to-latex-swift/ source. Document divergences.
• A.2 Decide resolution: (a) remote is newer, discard local; (b) merge if truly different work; (c) push local on top if local has critical unmerged changes.
• A.3 Tag v0.1.0 on the reconciled remote
• A.4 Update macdoc Package.swift:15 from path: → .package(url: "https://github.com/PsychQuant/pdf-to-latex-swift.git", from: "0.1.0")
• A.5 swift package update pdf-to-latex-swift + confirm swift build green
• A.6 Delete local packages/pdf-to-latex-swift/ directory (now redundant)

Track B — ocr-swift (low risk; clean extraction)

• B.1 cd packages/ocr-swift && git init && git add . && git commit -m "init: OCRCore v0.1.0 extracted from macdoc (#79)"
• B.2 gh repo create PsychQuant/ocr-swift --public --source=. --remote=origin --push --description "OCR pipeline — MLX + Ollama backends, PDFKit extractor"
• B.3 git tag v0.1.0 && git push origin v0.1.0
• B.4 Update macdoc Package.swift:31 from path: → .package(url: "https://github.com/PsychQuant/ocr-swift.git", from: "0.1.0")
• B.5 swift build on macdoc — confirm the Qwen3-VL/OCRCore work from PR fix: PageOCRRunner uses OCRCore.backend API (unblocks main build) #84 still links correctly through the new url: dep
• B.6 Delete local packages/ocr-swift/ directory

Track C — Tests/WordToMDTests (decision → small code change)

• C.1 Confirm directory really has no intended tests (grep WordToMDTests across macdoc sources + docs to make sure nothing references specific test names)
• C.2 Choose resolution: delete testTarget from macdoc Package.swift:60-64 (recommended; matches reality), OR populate with a placeholder "package loads" test similar to test: add NoteConvertTests smoke coverage (.note → pdf/html) to MacDocCLITests #81's strategy
• C.3 Apply chosen resolution + run swift test to confirm no regression
• C.4 Delete Tests/WordToMDTests/ directory if going with C.2-delete

Track D — End-to-end verification (after A + B + C)

• D.1 On a clean scratch dir: git clone https://github.com/PsychQuant/macdoc.git && cd macdoc && swift build — must succeed without any local pre-state (the goal issue explicitly calls out)
• D.2 swift test on the cloned copy — 28+ tests still green (NotePDFConvertTests and NoteHTMLConvertTests will XCTSkip per test: add NoteConvertTests smoke coverage (.note → pdf/html) to MacDocCLITests #81's Option B, but shouldn't crash)
• D.3 Close refactor: extract pdf-to-latex-swift + ocr-swift to PsychQuant repos; commit Tests/WordToMDTests fixtures (continues #78 pattern) #79 with a summary that documents the final resolution for each of (1), (2), (3)

Complexity

SDD-warranted (same as #78). Reasons:

• ✅ Cross-cutting change: 3 tracks across packages/*, macdoc Package.swift, and Tests/* — with ordering dependencies (Track D needs A + B + C done first)
• ✅ Architectural decision required for Track A (how to reconcile local vs existing remote — diff-first workflow needs design)
• ✅ Architectural decision required for Track C (delete vs populate)
• ✅ New external dependencies: 2 new .package(url:) entries in macdoc
• ✅ 5+ sequenced phases per track; multiple version-pin coordination points

Opt-out of spectra-discuss is not defensible here: Track A in particular has a genuinely open decision (what exactly is on remote vs local, and what to do with the divergence). Needs explicit alignment.

Risks

1. Remote pdf-to-latex-swift has divergent code from local → Track A.1/A.2 must explicitly audit diff. If remote has an PDFToLaTeXCLI target that local lacks, we might be reclaiming someone's half-finished experiment. Mitigation: document divergence before reconciliation; if user doesn't remember creating remote, consider moving it to PsychQuant/pdf-to-latex-swift-legacy and starting fresh.

2. PR #84 (Qwen3-VL/OCRCore) just merged — extracting ocr-swift mid-flight could dangle → Track B.5 is the critical check. The refactor on PageOCRRunner.swift relies on OCRBackend protocol from this package. After extracting, we must verify the compiled macdoc still links correctly.

3. swift package update mid-refactor might re-resolve other packages → Same class of risk we caught in #78 verify (note-to-pdf-swift silent downgrade). Pin Package.resolved carefully. Check after each update that note-core-swift@0.1.3, note-to-pdf-swift@0.1.2, word-builder-swift@0.9.0, ooxml-swift@0.7.0 don't accidentally regress.

4. Large .build/ directories (762 MB for pdf-to-latex + 2.7 GB for ocr-swift) accidentally get committed → If git init is run without a .gitignore excluding .build/, these could push to GitHub as multi-gig repos. Mitigation: verify .gitignore excludes .build/ before git add .. The note-* extraction got this right; repeat the pattern.

5. Tests/WordToMDTests might actually be expected to have tests by someone → Track C.1 audit before delete. Worst case: create a minimal placeholder test and defer the real fixtures to a future issue (matches #81's Option B spirit).

Open questions for spectra-discuss

1. Track A reconciliation strategy — what's the authoritative source for pdf-to-latex-swift? The remote (2026-04-16) vs local (2026-03-15)? Hand-audit diff first?
2. Version numbering — start both at v0.1.0 (like note-*) or bump based on prior commit history if any exists?
3. Track C decision — delete WordToMDTests testTarget or populate? Matches what intent?
4. Serial vs parallel — A and B can run in parallel (different repos), but A has remote-divergence risk. Do A first to de-risk?
5. Track D "clean clone" verification depth — only swift build, or also swift test on all 28 tests? XCTSkip semantics mean tests "pass" even without fixture.
6. Release version for macdoc after — does this warrant a v0.5.1 tag since Package.swift changes are user-visible? (Cf. release: note packages extraction + word-builder-swift wiring + #77 bump (#71 #77 #78) #82 which cut a release-style commit.)
7. Parallel branch strategy — 3 tracks → 3 feature branches merging to main in sequence? Or one branch refactor/extract-remaining-packages-79?

Next

Complexity = SDD-warranted with Track A divergence = genuinely unresolved. Recommend /spectra-discuss before proposing.