Skip to content

B3: derived doc generators (timeline, file-structure, tree, rebase) - #119

Merged
thrillmot merged 2 commits into
v1-go-rewritefrom
feat/go-b3-derived-docs
Jun 2, 2026
Merged

B3: derived doc generators (timeline, file-structure, tree, rebase)#119
thrillmot merged 2 commits into
v1-go-rewritefrom
feat/go-b3-derived-docs

Conversation

@thrillmot

Copy link
Copy Markdown
Collaborator

Summary

Wave B3 of the logmind Go rewrite — ports the derived-doc generators and the rebase wrapper.

  • 4 new subcommands: timeline, file-structure, tree, rebase
  • 4 new internal packages: internal/config, internal/decisions, internal/timeline, internal/tree
  • 2 new helpers on internal/gitcli: DefaultBranch (5-step search mirror), RunCaptured (stderr-exposing variant for rebase)
  • Makefile SNAPSHOT_PKGS extended with internal/timeline/... + internal/tree/...

Built on top of B1 + B2 (PRs #116 + #117). Targets v1-go-rewrite.

Byte-identical output vs Python v0.6.14

Verified against the actual logmind repo (112 decisions across 6 months, ~120KB unbounded tree):

Command Result
logmind timeline (brief, default) BYTE-IDENTICAL (1647 bytes)
logmind timeline --full BYTE-IDENTICAL (27407 bytes)
logmind file-structure (depth 2) BYTE-IDENTICAL (4180 bytes)
logmind file-structure --max-depth 0 (unbounded) BYTE-IDENTICAL (121729 bytes)
logmind file-structure --max-depth 1 BYTE-IDENTICAL (975 bytes)
logmind file-structure --max-depth 3 BYTE-IDENTICAL
logmind timeline --check (clean) BYTE-IDENTICAL
logmind file-structure --check (clean) BYTE-IDENTICAL
logmind tree BYTE-IDENTICAL
logmind timeline --check (stale) BYTE-IDENTICAL (exit 1)
logmind rebase (failure path on dirty WT) BYTE-IDENTICAL

Confirmation method: diff <(venv/bin/logmind <cmd>) <(bin/logmind <cmd>) against /Users/ludlow/logmind itself — the largest real fixture available.

Snapshot tests (49 total + 7 golden fixtures)

  • internal/timeline/ — 4 goldens covering brief/full × elision-vs-not, plus the empty/single-newline property tests
    • brief-mixed.golden — 3 months (4-entry elision, 2-entry verbatim, 1-entry singleton)
    • brief-singular-elision.golden — exactly 3 entries → ... 1 more decision ... singular noun
    • brief-two-elided-months.golden — two elided months back-to-back for inter-month spacing
    • full-mixed.golden — full mode pins lack of count suffix on month headers
  • internal/tree/generate-file-structure.golden pins the template head + tail bytes
  • internal/cli/timeline_stdout_brief.golden, timeline_stdout_full.golden for end-to-end stdout shape
  • 23 cli tests + 26 package tests, all green via make test

Brief-mode byte-identical proof

Brief-mode month grouping is the highest-risk algorithm — Python uses lines = [HEADER, ""] then "\\n".join(lines), which produces three newlines between the --- footer of the header and the first ## YYYY-MM. Go's strings.Builder byte-level approach would emit only two unless you mirror the join semantics precisely.

Resolution: assemble a []string mirror of Python's lines and use strings.Join(lines, "\\n") rather than emitting bytes directly. The trailing lines.append("") gives the final \\n after the last entry. See internal/timeline/timeline.go comments for the trace.

Verified against the actual docs/timeline.md which carries the brief-mode elision lines for 6 months of real history.

Tree-walk byte-identical proof

Tree walk uses pure Go (filepath.WalkDir not invoked — explicit os.ReadDir per frame for sort control). Sort key matches Python: (not is_dir, name.lower()) → directories first then case-insensitive alphabetical. Pattern matching is path-aware (matches full relative path, components, and basename).

One known semantic divergence: filepath.Match's * does NOT cross /, but Python's fnmatch.fnmatchcase's * DOES. For the patterns logmind actually ships (DEFAULT_IGNORES + typical .gitignore), this is invisible — patterns are either segment-level (*.pyc, __pycache__) or fully literal (site/.next). Documented in patternSetMatches.

Verified at depth 0/1/2/3 against the logmind repo itself which is the most realistic fixture I have (it has site/.next/, venv/, __pycache__/, .git/, mixed casing, et al).

Known divergence vs Python v0.6.14

--check without --write:

  • Python: prints error + exits 2
  • Go: prints byte-identical error + exits 1

The stdout message is unchanged so consumers diffing output see no difference. The exit-code divergence is a documented known issue. Closing it would require an ErrSilentExit2 sentinel in cmd/logmind/main.go (deferred to a coordinated cross-wave change — B4 lands first and is a candidate).

Open questions for human reviewer

  1. Exit-code divergence: keep as documented divergence, or land the ErrSilentExit2 sentinel as a coordinated patch alongside B4?
  2. Glob semantics divergence: the *-vs-/ divergence is theoretical but real. Want a custom fnmatch port that's *=anything-including-/ strict, or accept the practical equivalence?
  3. Rebase --no-fetch: Python exposes --no-fetch so tests can rebase against a fixture without an origin/. I ported it for parity; any objection?
  4. B4 coexistence: this PR is independent of B4 work happening in parallel on feat/go-b4-agents. The B4 branch will need to add root.AddCommand(newAgentsCmd()) next to the B3 wires.

Design decisions

  • Brief-mode algorithm: assemble []string mirror + strings.Join rather than byte-level emission. See comment in internal/timeline/timeline.go:106-119.
  • Tree walker is pure Go: never shell out to system tree(1) even when available. Consuming repos get identical output across OSes. See internal/tree/tree.go:1-22.
  • Config loader strategy: typed Config struct with yaml: tags; user keys overlay defaults leaf-by-leaf via yaml.Unmarshal into a pre-populated default struct. Mirrors Python's _deep_update shape without recursive Map traversal in Go. See internal/config/config.go:118-140.
  • -1 sentinel for unbounded depth: CLI's --max-depth 0 translates to internal -1 so Render can use a simple maxDepth >= 0 && depth >= maxDepth guard. Documented at the function boundary.
  • DefaultBranch 5-step search: extends internal/gitcli (not duplicated in internal/rebase) because B5 (init) and B4 (agents in some paths) will also want it.

Test plan

  • make build && make test — all packages green
  • make snapshot regenerates 7 golden files without drift
  • Byte-identical diff vs Python v0.6.14 across 8+ invocation shapes on the logmind repo itself
  • --check stale + clean paths verified
  • rebase failure paths verified (not-a-repo, detached HEAD, refusing-self, fetch fail, rebase fail)
  • CI matrix (Go 1.22+) — will run on push
  • Manual sanity check by reviewer on a consuming repo (e.g. tokenomics) to confirm the merge driver still converges

🤖 Generated with Claude Code

@vercel

vercel Bot commented Jun 2, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
logmind-site Ready Ready Preview, Comment Jun 2, 2026 5:36am

@claude

claude Bot commented Jun 2, 2026

Copy link
Copy Markdown

Claude finished @thrillmot's task in 7m 8s —— View job


🐛 Clud Bug review

This round: 1 critical · 0 minor · 0 resolved from prior · 0 still open

Found: 1 🔴 / 0 🟡 / 0 🟣

Per-skill scan

  • [critical-issues-only]: scanned all 29 files. 1 critical finding.
  • [evidence-based-review]: applied to all findings. ✓ anchored to specific line.
  • [respect-existing-conventions]: scanned for pattern fights. 0 findings.

Critical findings

🔴 [critical-issues-only]: Sort-order divergence from Python for symlinks-to-directories (internal/tree/tree.go:275)

Reasoning

os.ReadDir uses lstat, so e.IsDir() returns false for symlinks-to-dirs. Python's scandir().is_dir() follows symlinks and returns true, sorting symlinks-to-dirs before files. In any repo with unignored symlinks-to-directories the rendered tree diverges from Python v0.6.14, silently breaking the byte-identity contract. The explicit symlink guard at lines 309–311 is also unreachable dead code for the same reason (e.IsDir() is already false for symlinks, so line 301 already continues before reaching line 309).

Fix: use os.Stat for the sort-key dir-check when e.Type()&os.ModeSymlink != 0.


Skills referenced: [critical-issues-only, evidence-based-review, respect-existing-conventions]

Comment thread internal/tree/tree.go
@github-actions

github-actions Bot commented Jun 2, 2026

Copy link
Copy Markdown
Contributor

🐛 Clud Bug review

This round: 1 critical · 0 minor · 0 resolved from prior · 0 still open

Found: 1 🔴 / 0 🟡 / 0 🟣

Per-skill scan

  • [critical-issues-only]: scanned all 29 files. 1 critical finding.
  • [evidence-based-review]: applied to all findings. ✓ all anchored to specific lines.
  • [respect-existing-conventions]: scanned for pattern fights. 0 findings.

Critical findings

🔴 [critical-issues-only]: sort-order diverges from Python for symlinks-to-directories, breaking byte-identity contract (internal/tree/tree.go:275).

Reasoning

os.ReadDir uses lstat so e.IsDir() returns false for symlinks; Python's scandir().is_dir() follows symlinks (returns true), sorting symlinks-to-dirs before files. Any repo with unignored symlinks-to-directories produces non-byte-identical output vs Python v0.6.14. The symlink guard at lines 309–311 is also unreachable dead code (line 301 already continues for symlinks). Fix: probe with os.Stat for the sort key when e.Type()&os.ModeSymlink != 0.

Skills referenced: [critical-issues-only, evidence-based-review, respect-existing-conventions]

thrillmot added 2 commits June 2, 2026 01:46
…nternal config/decisions/timeline/tree packages with byte-identical Python v0.6.14 output
@thrillmot
thrillmot force-pushed the feat/go-b3-derived-docs branch from 4d2fb9b to 4a66f4b Compare June 2, 2026 05:47
@vercel

vercel Bot commented Jun 2, 2026

Copy link
Copy Markdown

Deployment failed with the following error:

Resource is limited - try again in 24 hours (more than 100, code: "api-deployments-free-per-day").

Learn More: https://vercel.com/thrillmot-s-projects?upgradeToPro=build-rate-limit

@claude

claude Bot commented Jun 2, 2026

Copy link
Copy Markdown

Claude finished @thrillmot's task in 2m 43s —— View job


Re-review: B3 derived doc generators (fix-push)

  • Load tools
  • Fetch incremental diff (commits 50bea65→4a66f4b)
  • Check prior thread (symlink sort fix) — thread already resolved ✓
  • Review new code for correctness
  • Emit structured summary

Fix verified: isDirFollow at internal/tree/tree.go:397 correctly uses os.Stat to follow symlinks for sort ordering, exactly matching Python's pathlib.Path.is_dir() behavior. Loop safety is preserved via the existing !e.IsDir() guard at line 310 (symlinks can't be real dirs via lstat, so they render as leaf nodes and never recurse).

@github-actions

github-actions Bot commented Jun 2, 2026

Copy link
Copy Markdown
Contributor

🐛 Clud Bug review

This round: 0 critical · 0 minor · 0 resolved from prior · 0 still open

Found: 0 🔴 / 0 🟡 / 0 🟣

Per-skill scan

  • [critical-issues-only]: scanned fix-push delta (4 files). prior critical resolved. 0 new findings.
  • [evidence-based-review]: not applicable — no new findings to anchor.
  • [respect-existing-conventions]: not applicable — fix-push only adds isDirFollow helper matching existing patterns.

Skills referenced: [critical-issues-only, evidence-based-review, respect-existing-conventions]

@thrillmot
thrillmot merged commit f220d1d into v1-go-rewrite Jun 2, 2026
15 of 16 checks passed
@thrillmot
thrillmot deleted the feat/go-b3-derived-docs branch June 2, 2026 05:51
@thrillmot thrillmot mentioned this pull request Jun 3, 2026
7 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant