Skip to content

Add leadtype doctor to explain resolved config and project health - #163

Open
KayleeWilliams wants to merge 3 commits into
dx/154-docs-project-runtimefrom
dx/155-doctor
Open

Add leadtype doctor to explain resolved config and project health#163
KayleeWilliams wants to merge 3 commits into
dx/154-docs-project-runtimefrom
dx/155-doctor

Conversation

@KayleeWilliams

@KayleeWilliams KayleeWilliams commented Aug 3, 2026

Copy link
Copy Markdown
Collaborator

Closes #155. Stacked on #162, part of #157.

generate, sync, lint, and score each answer a question about a project by doing something to it. None answered the one you need first: what is this project, and why? Which config was discovered, whether it resolved single-source or multi-repo, which values were authored versus inherited from a source repo versus inferred, which collections share one clone, what routes will exist, and which command fixes what is currently wrong.

Config
  docs/docs.config.ts  (single-source)

Sources
  local  local  → docs

Collections
  docs  /docs  52 page(s)  docs

Navigation
  explicit  52 routed page(s)
  sections: Concepts, Docs Pipeline, AEO & Agent Readability, Reference, Changelog

Output
  public  6 present, 0 missing, 0 stale

Integrations
  framework: Astro
  surfaces:  mcp, nlweb, skills, agent-card, robots:balanced, redirects, feeds:1

No findings.

Read-only by construction

Doctor never clones, refreshes, writes, or generates, so an unsynced remote is a finding naming leadtype sync rather than a fetch — which is what makes it safe to run first, in CI, or against a production config you are debugging. Everything it reports comes from the same config loader and resolvers the other commands use, so a clean doctor run and a clean generate run cannot disagree about the project.

Every finding carries a stable id, the config field or file that owns it, and a concrete next command. --json keeps those ids and adds provenance so an agent can act without parsing prose; warnings stay on stderr so the report stays a clean machine record. Exit 0 when nothing is an error, 1 when a required input is missing or invalid.

It found a real bug on its first run

/docs/pipeline/redirects was listed in llms.txt's starting points but absent from navigation, so the page agents were pointed at had no place in the sidebar. Fixed in this PR.

One premise corrected while building it

Pages missing from a curated tree are not unreachable — they fall back to the root of ungrouped. The finding reports that accurately (present, but placed by default rather than by decision) and skips configs whose root entries are include globs rather than guessing at an expansion it would have to re-derive.

Two bugs the c15t migration then exposed

Fixture tests could not catch either; pointing doctor at a real pinned-source project did.

--out resolved against --src, while leadtype generate resolves it against the current directory. doctor --src apps/c15t-example --out apps/c15t-example/public therefore looked in apps/c15t-example/apps/c15t-example/public and reported the output missing. Two commands reading the same flag differently is its own bug.

Navigation reported as inferred for a project whose navigation is inherited. Config loading does not apply source-owned inheritance — generation does, later in its pipeline — so doctor was describing a pre-inheritance project and showing a filesystem-derived tree the real build never uses. For a pinned-source project that is precisely the question doctor exists to answer, answered wrongly.

It now applies inheritance through the same shared implementation generation uses, and reports inherited as a distinct origin, which is the more useful answer anyway: "this tree belongs to the source repo" is not the same as "you wrote this tree". A source that cannot be read degrades to a source.inherit-failed finding rather than throwing, because doctor must keep reporting when a source is unsynced.

Against real c15t content it now reports inherited, the seven sections c15t actually publishes, and 63 pages absent from that curated tree.

@coderabbitai

coderabbitai Bot commented Aug 3, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: c82750ed-febc-4ff9-a4c7-fa7f6029580f

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Comment @coderabbitai help to get the list of available commands.

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Important

Two findings worth fixing before merge: the sources loop recomputes a default cacheDir differently than sync does, so doctor reports "not synced" for a synced source (reproduced by running it); and inspectNavigation resolves navigation from inputs that diverge from what generate stages, which can produce a false nav.unknown-group error and exit 1 on a project that generates cleanly. Both undercut the headline claim that a clean doctor run and a clean generate run cannot disagree.

Reviewed changes — the whole PR at 7e8909b: the new command, its tests, the CLI wiring, and the docs/nav/changeset that ship with it.

  • New leadtype doctor commandpackages/leadtype/src/cli/doctor.ts loads the config through loadDocsConfig, then inspects collections, the deduped source graph, navigation, output artifacts, and agent surfaces, emitting findings that each carry a stable id, level, owner config field, and a fix command.
  • Read-only by construction — no clone, refresh, write, or generate; an unsynced remote becomes a source.not-synced finding naming leadtype sync. Exit 0 unless some finding is level: "error", 2 on usage error.
  • CLI wiringcli.ts gains the doctor dispatch branch, the commandUsage branch for leadtype help doctor, and a line in the command list.
  • Tests — 18 tests over the healthy path, provenance, config discovery/deprecation, remote sync states, shared acquisitions, glob misses, unrepresented pages, artifacts, and framework/surface detection. All pass locally (npx vitest run src/cli/doctor.test.ts).
  • Docs — new docs/reference/doctor.mdx (finding-id table, --json shape), a doctor section in cli.mdx, nav + paths.lock.json entries, and SKILL.md step 1 now tells agents to run doctor first.
  • Pre-existing nav gap fixedpipeline/redirects was reachable and listed in llms.txt starting points but absent from navigation; docs/docs.config.ts now places it. Confirmed: running doctor on this repo at head reports no nav.unrepresented-page finding.

⚠️ The error-level findings that gate exit 1 are the least tested

source.not-synced and source.cache-stale have tests, but source.cache-unverifiable, source.dir-missing, and nav.unknown-group have none — and those three are the ones that flip the exit code that CI is invited to gate on. The cacheDir bug below is a direct consequence of the same gap: every git fixture in doctor.test.ts sets an explicit cacheDir, so the default-path branch is never executed.

Technical details
# Cover the exit-1 paths

## Affected sites
- `packages/leadtype/src/cli/doctor.test.ts` — no fixture omits `cacheDir`; no fixture reaches `source.cache-unverifiable` (checkout present, manifest absent), `source.dir-missing` (collection `dir` pointing nowhere), or `nav.unknown-group` (page `group:` absent from a collection's declared `groups`).

## Required outcome
- Each finding id that can produce exit `1` is exercised by at least one test that asserts the id, its `level`, and the exit code.
- At least one remote-collection fixture relies on the default `cacheDir` rather than authoring one, so the `defaultCacheDir` path is covered.

## Suggested approach
- The existing `fixture()` + `runJson()` helpers are enough; `writeSyncManifest` is already imported for seeding cache state, and omitting it is what produces `source.cache-unverifiable`.

ℹ️ Nitpicks

  • packages/leadtype/README.md:72 still reads "The leadtype binary wraps init, generate, sync, lint, mcp, and score" — doctor is missing, and that README is the npm landing page.
  • EXPECTED_SITE_ARTIFACTS describes site mode only, so after a successful leadtype generate --bundle doctor reports output.not-generated and suggests leadtype generate. Bundle mode is a CLI flag with no config trace, so detecting it is awkward — a one-line caveat in doctor.mdx may be the honest fix.
  • inspectNavigation collapses disagreeing per-collection origins to "explicit" (origins.size !== 1), which reports a specific provenance for a project that doesn't have one — in a command whose purpose is explaining where values came from.
  • renderHuman's surfaces: (defaults) branch is unreachable: enabledSurfaces pushes skills and agent-card on !== false, which holds when agents is undefined, so the list is never empty.

Pullfrog  | Fix all ➔Fix 👍s ➔View workflow run | Using Claude Opus𝕏

Comment on lines +460 to +464
const cacheDir = path.resolve(
configDir,
source.cacheDir ??
path.join(".leadtype", "sources", `${source.repository}@${source.ref}`)
);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This recomputes the default cache path with the raw repository URL, but sync derives it through repositorySlug() in defaultCacheDir().leadtype/sources/acme-acme@main, not .leadtype/sources/https:/github.com/acme/acme.git@main. For any git source without an authored cacheDir the reported cacheDir is a path that can never exist and syncedCommit is always null, so the human report prints "not synced" while the collection-level checks (which go through resolveCollection, hence the correct path) emit no source.not-synced finding — one report contradicting itself.

Technical details
# Use the shared default cache path

## Reproduced
A fixture with `collections.docs = { repository: "https://github.com/acme/acme.git", ref: "main", dir: "docs" }` (no `cacheDir`) and a real checkout + sync manifest at `.leadtype/sources/acme-acme@main` reports:

```json
{ "cacheDir": "…/.leadtype/sources/https:/github.com/acme/acme.git@main", "syncedCommit": null }
```

with no `source.not-synced` finding — `Sources` says not synced, `Findings` says nothing is wrong.

## Affected sites
- `packages/leadtype/src/cli/doctor.ts:460-464` — the `resolved.sources` loop's `cacheDir` fallback.
- Downstream: `syncedCommit` (JSON) and the `not synced` column in `renderHuman`.

## Required outcome
- Doctor resolves a source's cache directory to exactly the path `sync` would use, so `cacheDir` and `syncedCommit` agree with the collection-level `source.*` findings in the same report.

## Suggested approach
- `defaultCacheDir(repository, ref)` is already exported from `../sync/sync` (it is what `resolveCollection` uses); import it alongside `readSyncManifest` and drop the hand-rolled `path.join`.
Suggested change
const cacheDir = path.resolve(
configDir,
source.cacheDir ??
path.join(".leadtype", "sources", `${source.repository}@${source.ref}`)
);
const cacheDir = path.resolve(
configDir,
source.cacheDir ?? defaultCacheDir(source.repository, source.ref)
);

mounts,
};

const manifest = await resolveDocsNavigation({

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This call reuses resolveDocsNavigation, but it assembles the inputs itself rather than the way generate does, and three inputs go missing: collection include/exclude (generate applies them while staging the source mirror), i18n/locale (without them every translation file becomes its own page), and globally merged groups (generate merges across collections via mergeCollectionGroups; here each collection sees only its own). The result is false nav.unrepresented-page warnings on filtered and localized projects, and a false nav.unknown-group — which is level: "error", so doctor exits 1 on a project that generates cleanly.

Technical details
# Resolve navigation from the same inputs `generate` resolves it from

## Affected sites
- `packages/leadtype/src/cli/doctor.ts:580-590``navigationOptions` + the `resolveDocsNavigation` call; no `include`/`exclude`, no `i18n`/`locale`, `groups: authoredGroups ?? []`.
- `packages/leadtype/src/cli/doctor.ts:592-600``nav.unknown-group`, `level: "error"`, so these divergences change the exit code.
- `packages/leadtype/src/cli/doctor.ts:631-641``unrepresentedPages` compares `manifest.ungrouped` against literal root entries.

## The three divergences
- **Filters.** `resolveDocsSources` in `cli/generate.ts` turns a collection's `include`/`exclude` into `source.filters`, and `copySourceFiles` stages with `ignore: exclude` before nav is resolved over the mirror. Doctor's own `inspectCollection` honors them for `pageCount` (`ignore: authored?.exclude ?? []`), so `pageCount` and `routedPages` in one report are computed over different file sets, and an excluded page is reported as absent from the curated tree. (The single-default-`docs`-collection shape takes `createSourceMirror`'s no-staging fast path, so that one shape agrees by accident.)
- **i18n.** `generate` passes `i18n` and loops locales; `readSourceDocs` then collapses translations to one page per logical path. With neither, `docs/es/guide.mdx` resolves as its own page `/docs/es/guide`, lands in `manifest.ungrouped`, and is reported unrepresented on every curated i18n site. `routedPages` is inflated the same way.
- **Groups.** `mergeCollectionGroups` builds one globally-unique group list before nav; doctor passes each collection only its own `groups`, so a page whose `group:` is declared by a sibling collection errors here and resolves there.

## Required outcome
- Either doctor resolves navigation over the same file set and group list `generate` does, or the checks that depend on the difference (`nav.unrepresented-page`, `nav.unknown-group`) are skipped for the configs where doctor cannot reproduce it — the way the `include`-glob root-entry case is already deliberately skipped just below.
- No `level: "error"` finding on a config that `generate` resolves without complaint.

## Open questions for the human
- Is mirroring generate's input assembly the intended long-term shape, or should the staging/filter/locale resolution move behind a shared helper both commands call? The stated contract ("a clean doctor run and a clean generate run cannot disagree") is a standing invariant, and every future generate-side change to staging will silently re-break it as long as the inputs are assembled twice.

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Important

The --out fix is right (it matches generate, verified at generate.ts:2630). The inheritance fix is right in direction but only half-plumbed: navigation and groups now come from the inherited collection while the mounts used to compute every page's urlPath still come from the un-inherited one, which re-opens the same class of false nav.* finding the commit set out to close. Separately, source.inherit-failed is warn, so doctor exits 0 on the exact fixture the new test builds — a synced source whose config can't be read — while generate hard-fails on it.

Reviewed changes — the delta since the prior pullfrog review at 7e8909b. The branch was rebased, so the original commit is now 3683f11; the substantive new work is 2ab82ba. The sparse git-source support that appears in a raw commit-range diff arrived from the base branch and is out of scope here.

  • Fixed --out resolutiondoctor.ts:374 now resolves the flag against the current directory rather than --src, matching leadtype generate; the artifacts test passes an absolute path accordingly.
  • Applied source-owned inheritance before reporting — a new applySourceInheritance calls the shared inheritCollectionSourceConfigs on the normalized collections map (correct input: it is what expands sources: {} and folds the sourceConfig alias), threads the merged collections into inspectNavigation, and adds "inherited" to navigation.origin.
  • Degraded rather than thrown — a source config that can't be read becomes a source.inherit-failed finding, keeping the report intact.
  • Tests — two new cases covering the inherited-navigation and unreadable-source-config paths; 20 tests pass (npx vitest run src/cli/doctor.test.ts).
  • Docsdoctor.mdx gains the inherited origin, the source.inherit-failed row, and the --out resolution note; paths.lock.json hash refreshed.

ℹ️ Nitpicks

  • docs/reference/cli.mdx:51 and DOCTOR_USAGE (doctor.ts:119) both still describe --out without the cwd-resolution note that doctor.mdx now carries, and they sit directly beneath --docs-dir, which is --src-relative — the two flags differing is exactly what needs saying in the place a reader hits first.
  • source.inherit-failed uses owner: "inheritConfig" where every other finding names a keyed path (collections.<key>.inheritConfig), and because inheritCollectionSourceConfigs throws on the first failing collection, one unreadable source silently drops inherited navigation for every healthy collection in the same project too.

Pullfrog  | Fix all ➔Fix 👍s ➔View workflow run | Using Claude Opus𝕏

// resolved with its own mount — the same mapping the runtime project uses.
const mounts = [
{ pathPrefix: "", urlPrefix: collection?.routePrefix ?? "/docs" },
...(collection?.mounts ?? []),

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mounts is in DEFAULT_SOURCE_CONFIG_INHERIT, so a source repo's mounts are inherited exactly like its navigation — but this reads the resolved (un-inherited) collection while line 635 now reads the merged one. resolveDocsNavigation computes every page's urlPath from these mounts, so for a pinned source that declares mounts doctor resolves the inherited tree against un-mounted paths: the curated entries miss, the pages land in ungrouped, and doctor emits nav.unrepresented-page on a project generate routes cleanly.

Technical details
# Resolve navigation with the inherited mounts too

## Affected sites
- `packages/leadtype/src/cli/doctor.ts:653``...(collection?.mounts ?? [])` reads `loaded.resolved.collections`, which `normalizeDocsConfig` built before inheritance ran.
- `packages/leadtype/src/cli/doctor.ts:710``toDocsUrlPath(entry, mounts)` uses the same list to compare root entries, so the `unrepresented` comparison is wrong on both sides.

## Why generate disagrees
- `generate` replaces `loadedConfig.config.collections` with the inherited map (`cli/generate.ts:2751`) *before* `resolveDocsSources`, so `entry.collection.mounts` at `cli/generate.ts:2013` is post-inheritance and composes into the mount list at `cli/generate.ts:1955`.

## Required outcome
- The mount list doctor resolves navigation against is the one generate would use for the same collection, including mounts inherited from the source repo.
- A fixture whose source `docs.config.ts` declares `mounts` (and whose project config does not) reports no `nav.unrepresented-page` — no test currently exercises inherited mounts at all.
Suggested change
...(collection?.mounts ?? []),
...(merged?.mounts ?? collection?.mounts ?? []),

} catch (error) {
issues.push({
id: "source.inherit-failed",
level: "warn",

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warn means exit 0, and the new test above builds the case where that is wrong: a checkout that is present and manifest-verified but has no readable source config. generate throws there (nothing catches inheritCollectionSourceConfigs at generate.ts:2751), so CI gating on doctor passes and the build then fails — while doctor.mdx promises exit 1 when "a required input is missing or invalid". The unsynced case doesn't need warn to stay reportable; it already exits 1 through source.not-synced.

Technical details
# Level `source.inherit-failed` by what it actually blocks

## Affected sites
- `packages/leadtype/src/cli/doctor.ts:596-602` — the finding, `level: "warn"`.
- `packages/leadtype/src/cli/doctor.test.ts` — "keeps reporting when the source config cannot be read" asserts `code` is `0` for a fixture with a valid `.git`, a matching sync manifest, and no `docs/docs.config.ts` in the cache. That is a synced source, and it is a hard `generate` failure.

## Required outcome
- When a collection declares `inheritConfig` and the checkout exists but its config cannot be read or throws, doctor reports `level: "error"` and exits `1`, because `generate` cannot run on that project.
- Keeping the report itself intact (no throw) is right and should not change.
- A test asserts the exit code for both shapes: cache absent (already `error` via `source.not-synced`) and cache present but config unreadable.

## Open questions for the human
- Is there a case where a synced source legitimately ships no config while `inheritConfig` is on? If so it wants a distinct id, not a softer level on this one — `inheritConfig` is opt-in, so "enabled and unsatisfiable" reads as a config error either way.

// pinned-source project, so doctor has to apply it before reporting — the
// un-inherited config would say "inferred" for a project whose tree comes
// from the source repo. Uses the same shared implementation generation does.
const inherited = await applySourceInheritance({

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Inheritance runs here, after loadDocsConfig already normalized, so resolveCollectionEntry's recordExplicit never saw the inherited fields — report.collections[].provenance.navigation is absent for a project whose navigation did come from a source repo, in the same report where navigation.origin says "inherited". generate (generate.ts:2751) and project/index.ts:207 both inherit before normalizing, which is what keeps their provenance honest.

Technical details
# Make provenance see what inheritance supplied

## Affected sites
- `packages/leadtype/src/cli/doctor.ts:508-512` — inheritance applied after `loadDocsConfig`, and its result is threaded only into `inspectNavigation`.
- `packages/leadtype/src/cli/doctor.ts:449``provenance: collection.provenance`, computed by `config/normalize.ts:151-163` over the un-inherited collection.
- Inheritable fields with no provenance signal at all today: `groups`, `mounts`, `frontmatterSchema`, `flatteners`.

## Required outcome
- For a collection whose `navigation`/`groups`/`mounts`/`frontmatterSchema` came from its source repository, the report says so — `provenance` and `navigation.origin` cannot contradict each other in one run. Explaining where values came from is the command's stated purpose.

## Suggested approach
- Inherit first, then normalize, the way `project/index.ts:204-219` does it, and let `recordExplicit` record the merged values. That also removes the standing hazard of two orderings drifting.

## Open questions for the human
- Is "inherited" meant to be a `FieldProvenance` origin of its own rather than folded into `explicit`? A reader debugging a pinned-source project wants "authored here" and "came from the source repo" distinguishable per field, not only for the navigation tree as a whole.

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Important

The gate itself is sound — I ran leadtype doctor --src . --docs-dir docs at head and it exits 0 (output.not-generated is warn, so the absent public/ doesn't fail the job). What needs fixing is the guidance shipped alongside it: both the new doc section and the ci.yml comment explain the multi-repo hazard through the wrong mechanism, and they tell a reader that pinning a commit makes the gate safe when it doesn't.

Reviewed changes — the delta since the prior pullfrog review at 2ab82ba. The branch was rebased again, so that commit is now 827e697; the only new work is 80f4df4, which touches three files. doctor.ts is unchanged, so all five prior inline threads remain open.

  • Gated CI on doctor.github/workflows/ci.yml gains a Diagnose project (leadtype doctor) step running bun x leadtype doctor --src . --docs-dir docs, placed after the existing leadtype lint step, with a comment explaining why the command is safe to gate on and why the example apps are excluded.
  • Documented gating for usersdocs/pipeline/validate-in-ci.mdx gains a "Gate on the resolved project too" section, and its "Run before generate" snippet is reordered to doctor → lint → generate with a new rationale for that order.
  • Refreshed the lockfile hashdocs/paths.lock.json picks up the new validate-in-ci body.

ℹ️ Gating on a command whose two known divergence-from-generate findings are still open

nav.unknown-group is error-level (doctor.ts:667-675), and the still-open thread on inspectNavigation's inputs is precisely a claim that doctor can emit it for a project generate routes cleanly. Until that closes, this step converts a latent false positive into a red build on main. This repo doesn't trip it today — one default collection, no include/exclude, no i18n, no mounts — so this is a sequencing question, not a present breakage.

Technical details
# Decide whether the gate lands before or after the open divergence threads

## Affected sites
- `.github/workflows/ci.yml:55-56` — the new gate.
- `packages/leadtype/src/cli/doctor.ts:655-665` — nav resolution inputs (open thread on review 4851890790's predecessor).
- `packages/leadtype/src/cli/doctor.ts:653` — un-inherited `mounts` (open thread).
- `packages/leadtype/src/cli/doctor.ts:667-675``nav.unknown-group`, `level: "error"`.

## Required outcome
- Either the two nav-input threads are closed before this gate merges, or there is a deliberate decision that the gate's blast radius on this repo's single-collection shape is acceptable for now.

## Open questions for the human
- Is the intent to gate every repo in this monorepo eventually, or only the root docs permanently? The `ci.yml` comment reads as the latter, but the doc section invites users to do the former.
- `source.cache-unverifiable`, `source.dir-missing`, and `nav.unknown-group` are the three findings that flip the exit code and none has a test (raised in review 1, still open). Gating CI on them raises the cost of that gap.

ℹ️ Nitpicks

  • ci.yml runs doctor after leadtype lint, while the doc it dogfoods now says to order these "cheapest-and-most-specific first" and puts doctor above lint in its snippet, on the rationale that "Doctor fails on a project that can't resolve at all, before anything reads content." Swapping the two steps would make the repo's own CI match the sequence it recommends.
  • docs/reference/doctor.mdx:156 describes --json as "Machine-readable report on stdout. Warnings stay on stderr", but finish (doctor.ts:902-914) writes only to stdout in both modes and nothing reaches stderr except the usage error — findings at level: "warn" live in the JSON issues array. Pre-existing to this delta, but it is the one claim in the flag table that a --json consumer would act on.

Pullfrog  | Fix all ➔Fix 👍s ➔View workflow run | Using Claude Opus𝕏


It is read-only by construction: it never clones, refreshes, writes, or generates, which is what makes it safe to run first in a job. Exit `1` means a required input is missing or invalid; warnings still print but do not fail the build.

Worth knowing before you gate a multi-repo project on it: a config that pins a **mutable ref** — a branch rather than a commit — has a cache that is stale whenever CI hasn't synced, and doctor reports that. On a pinned commit it is a real signal; on a branch it mostly reports the CI environment. Either run `leadtype sync` first, or gate only on the repo whose docs you own.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doctor's exit code does not depend on ref kind, so this tells a reader the opposite of what will happen. On a fresh CI checkout the cache is absent, not stale, which is source.not-synced at level: "error" (doctor.ts:276-284) — for a pinned SHA exactly as much as for a branch. source.mutable-ref is only warn (doctor.ts:471-483), so the ref kind never moves the exit code on its own, and source.cache-stale needs an existing manifest to compare against. Someone who follows this, pins a commit, and gates gets a red build.

Technical details
# Describe the gate hazard as "no cache", not "mutable ref"

## Affected sites
- `docs/pipeline/validate-in-ci.mdx:138` — attributes gate failure to mutable refs and cache staleness, and asserts a pinned commit makes it "a real signal".
- `.github/workflows/ci.yml:51-54` — the same misconception in the comment justifying the example-app exclusion.

## Why the code disagrees
- `packages/leadtype/src/cli/doctor.ts:276` tests `existsSync(path.join(cacheDir, ".git"))` with no reference to `refKind`, pushes `source.not-synced` at `level: "error"`, and returns before any manifest read.
- `packages/leadtype/src/cli/doctor.ts:297-305` (`source.cache-stale`) is only reachable once that check passes and a manifest exists, so it cannot fire on a fresh checkout.
- `packages/leadtype/src/cli/doctor.ts:471-483` (`source.mutable-ref`) is `level: "warn"`; `report.ok` is `!issues.some(i => i.level === "error")` (`:525`) and `finish` returns `report.ok ? 0 : 1` (`:913`).
- Source caches are gitignored by this project's own convention — root `.gitignore:38` (`apps/*/.leadtype/`) and `apps/c15t-example/.gitignore:3` — so "no cache in CI" is the normal state for every remote-source project.

## Required outcome
- The paragraph says that any project with a remote source exits `1` in CI until `leadtype sync` has run, regardless of whether `ref` is a branch or a commit, and names `source.not-synced` as what fires.
- The two remedies already given (`leadtype sync` first, or gate only on the repo whose docs you own) stay — they are correct. Only the mechanism needs rewriting.
- Pinning a commit is still worth recommending, but for reproducibility, not for making this gate pass.

Comment thread .github/workflows/ci.yml
# declares, a collection pointing at a directory that isn't there.
#
# Only this repo's own docs are gated. The example apps pin the current
# git branch as their source ref, so their caches are stale by design on

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is right to exclude the example apps but wrong about why, in a way that would mislead whoever revisits the decision. apps/next-example does track the current branch (resolveExampleSourceRef() shells out to git branch --show-current), but apps/c15t-example/leadtype.config.ts:51 pins a commit SHA and its own comment says it does so deliberately. And neither would report a stale cache: both cache dirs are gitignored, so in CI they have no checkout at all and produce source.not-synced at level: "error". The accurate reason is "no source is synced in CI, so any remote collection is an error until leadtype sync runs".

generate, sync, lint, and score each answer a question about a project by
doing something to it. None answered the one you need first: what *is* this
project, and why? Which config was discovered, whether it resolved
single-source or multi-repo, which values were authored versus inherited from
a source repo versus inferred, which collections share one clone, what routes
will exist, and which command fixes what is currently wrong.

Read-only by construction. Doctor never clones, refreshes, writes, or
generates, so an unsynced remote is a finding naming `leadtype sync` rather
than a fetch — which is what makes it safe to run first, in CI, or against a
production config you are debugging. Everything it reports comes from the same
config loader and resolvers the other commands use, so a clean doctor run and a
clean generate run cannot disagree about the project.

Every finding carries a stable id, the config field or file that owns it, and a
concrete next command. `--json` keeps those ids and adds provenance so an agent
can act without parsing prose; warnings stay on stderr so the report stays a
clean machine record. Exit 0 when nothing is an error, 1 when a required input
is missing or invalid.

Dogfooding it on this repo immediately found a real bug: `/docs/pipeline/redirects`
was listed in llms.txt's starting points but absent from `navigation`, so the
page agents were pointed at had no place in the sidebar. Fixed here.

One premise had to be corrected while building it: pages missing from a curated
tree are not unreachable — they fall back to the root of `ungrouped`. The
finding reports that accurately (present, but placed by default rather than by
decision) and skips configs whose root entries are include globs rather than
guessing at an expansion it would have to re-derive.
Pointing doctor at a real pinned-source project — the migrated c15t example —
surfaced two things the fixture tests could not.

`--out` resolved against `--src`, while `leadtype generate` resolves it
against the current directory. Running `doctor --src apps/c15t-example --out
apps/c15t-example/public` therefore looked for artifacts in
`apps/c15t-example/apps/c15t-example/public` and reported the output as
missing. Two commands reading the same flag differently is its own bug, so
doctor now matches generate.

More seriously, doctor reported navigation as `inferred` for a project whose
navigation is inherited from its source repository. Config loading does not
apply source-owned inheritance — generation does, later in its pipeline — so
doctor was describing a pre-inheritance project and showing a
filesystem-derived tree that the real build never uses. For a pinned-source
project that is precisely the question doctor exists to answer, answered
wrongly.

It now applies inheritance through the same shared implementation generation
uses, and reports `inherited` as a distinct origin, which is the more useful
answer anyway: "this tree belongs to the source repo" is different from "you
wrote this tree". Because doctor is read-only and must keep reporting when a
source is unsynced, a failure to read source config degrades to a
`source.inherit-failed` finding rather than throwing.
CI already dogfoods `leadtype lint` on this repo's docs. Doctor is the same
argument one level up: lint checks content, doctor checks that the project
resolves at all — config loads, every collection's directory exists, no page
declares a group nothing declares.

It is read-only by construction, so gating on it is safe: it never clones,
refreshes, writes, or generates.

Only this repo's own docs are gated. The example apps pin the current git
branch as their source ref, so their caches are stale by design on any branch
that hasn't synced — gating there reports the CI environment rather than the
config. The docs page says so, because anyone with a branch-pinned multi-repo
config will hit exactly that.

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ℹ️

This push is a rebase onto dx/154-docs-project-runtime at 0d78b7cdoctor.ts and doctor.test.ts are byte-identical to the last review, and all five open threads on them still describe the code as written. The rebase absorbed four base-branch files correctly and mis-resolved one: two hashes in docs/paths.lock.json are the pre-rebase values for content the base branch has since edited. Separately, the base moved generate's side of three seams doctor is supposed to agree with — worth reading before addressing the open threads, because one of them is now a working in-repo reference implementation.

Reviewed changes — the delta since the prior pullfrog review at 80f4df4. git diff-tree -r --name-status 0d30b51 0d78b7c (old base → new base) returns the same 16 files as the head-to-head delta, so there is no new hand-written work in this push.

  • SKILL.md — the writing.mdwrite-for-agents.md reference rename absorbed from the base; the doctor-specific content is unchanged.
  • .github/workflows/ci.yml — the base's Test evals step absorbed and the PR's Diagnose project (leadtype doctor) gate survived intact at L55-56. Verified the gate is green at head: running runDoctorCommand(["--src", ".", "--docs-dir", "docs"]) from workspace source exits 0 (52 pages, one local collection, only output.not-generated at warn), so the base's config-loading changes did not break it.
  • docs/reference/cli.mdx — the base's routePrefixprefix prose correction absorbed alongside the PR's doctor section.
  • docs/paths.lock.json — mis-resolved (inline, two comments). The base-to-base diff changes three hashes; head-to-head only one landed.

The base branch moved generate's side of three seams

The whole argument for doctor is that it reports what generate will do. Three of the five open threads are about doctor and generate disagreeing — and on this rebase the base branch changed generate at all three sites while doctor.ts stood still. None of this is a new defect in the PR; it changes what "fix the open thread" means.

  • generate.ts:2765-2788 now re-normalizes after inheritance. That is precisely what the thread at doctor.ts:508 asks for — inherit, then normalizeDocsConfig over the merged collections so recordExplicit sees the inherited fields and provenance stops contradicting navigation.origin. There is now a concrete in-repo implementation to mirror rather than a design to invent, including the detail of preserving resolved.sources and resolved.deprecations across the re-normalize.
  • generate.ts:2939-2951 gained an i18n guard on content-derived navigation (metadata.i18n === undefined before deriving), with a comment spelling out why: derivation keys sections off the first path segment, which for docs/en/… is the locale, while navigation resolves per locale over locale-stripped paths — so no derived section can ever match. doctor.ts:639-647 derives unconditionally. The thread at doctor.ts:661 already names i18n/locale as one of the missing inputs, so this is the same concern with a sharper counterexample now checked in, not a separate finding.
  • generate.ts:2765-2788 also widens the inherited-mounts gap flagged at doctor.ts:653: because generate re-normalizes over the merged collections, entry.collection.mounts is post-inheritance everywhere downstream, so the distance between the two mount lists grew rather than shrank.

ℹ️ Nitpicks

  • A stale hash on a stable path is inert today — computeDocsRedirects consults hashes only for paths that disappeared or appeared, and any regenerate silently rewrites the file. The cost is deferred: the next rename of /docs/pipeline/collections or /docs/reference/cli fails rename detection and generate throws docs page(s) disappeared without a redirect. CI has no lockfile-drift check to catch the divergence in between, which is what makes a rebase conflict here easy to miss.

Pullfrog  | Fix all ➔Fix 👍s ➔View workflow run | Using Claude Opus𝕏

Comment thread docs/paths.lock.json
{
"path": "/docs/pipeline/collections",
"hash": "8114273c620aa826"
"hash": "c0385584dc467a04"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rebase conflict mis-resolution: this is the pre-rebase hash for /docs/pipeline/collections, whose content the base branch edited. Regenerating on this branch (runGenerateCommand with --base-url) produces 8114273c620aa826 — the value this line held before the push. Every other entry in the file matches a fresh regenerate.

hashRedirectContent (redirects/node.ts:21-27) hashes the trimmed body of the generated <outDir>/docs/<page>.md mirror, so this drifts whenever the page content changes and the lockfile isn't refreshed.

Suggested change
"hash": "c0385584dc467a04"
"hash": "8114273c620aa826"

Comment thread docs/paths.lock.json
{
"path": "/docs/reference/cli",
"hash": "7dcd173e5a973493"
"hash": "73e07d8bd071d8d0"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same mis-resolution for /docs/reference/cli: a fresh regenerate yields ce55b7b6c5c42916, not 73e07d8bd071d8d0. This one is expected to move on this branch — the PR adds a doctor section to cli.mdx and the base branch corrected the routePrefixprefix prose in the same file — but the recorded value is neither the old content's hash nor the new one's.

The adjacent /docs/pipeline/validate-in-ci hash (d85e9261d6493e0a) and the new /docs/reference/doctor entry (f1ca98d7b80d093e) both verified correct, so regenerating and committing the lockfile is a two-line change.

Suggested change
"hash": "73e07d8bd071d8d0"
"hash": "ce55b7b6c5c42916"

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add leadtype doctor to explain resolved config and project health

1 participant