Skip to content

feat(pyresolve): CLI for inspecting a PyPI RSF - #6

Merged
jonyoder merged 2 commits into
mainfrom
cli
Aug 4, 2026
Merged

feat(pyresolve): CLI for inspecting a PyPI RSF#6
jonyoder merged 2 commits into
mainfrom
cli

Conversation

@jonyoder

@jonyoder jonyoder commented Aug 4, 2026

Copy link
Copy Markdown
Collaborator

What

cmd/pyresolve — a stdlib-only CLI over a local RSF. This makes the metadata layer something you can actually run.

Command Does
stats package count, dependency-dictionary size, file size
versions <pkg> captured versions, sorted by PEP 440 order (not string order)
deps <pkg> [version] requires-python, requirements, provided extras; defaults to the highest version
walk <pkg> [--depth N] breadth-first over the highest-version-of-each-package graph

All accept --rsf <path> (falling back to $PYRESOLVE_RSF) and --json. No CLI framework dependency, and no net/http anywhere — there's a test asserting that, since it's the property that makes the tool honest about reading a local file.

walk is not resolution, and says so

It doesn't solve version constraints — it takes the highest captured version of each package and follows its Requires-Dist edges, ignoring markers, extras conditions, and specifiers. Every --help and every actual invocation prints that, and the JSON carries it in a note field. Shipping something that looked like resolution but wasn't would be worse than not shipping it.

Real-file verification

Against the production RSF (932,861 packages, 936.3 MiB), each command ~0.4–0.6s:

$ pyresolve deps flask
flask 3.1.3
Requires-Python: >=3.9
Requires-Dist:
  asgiref>=3.2; extra == "async"
  blinker>=1.9.0
  click>=8.1.3
  ...
Provides-Extra: async, dotenv

That matches real Flask 3.1.3 metadata. walk flask --depth 3 reaches 131 packages.

A bug I found reviewing against real data

walk collapsed three outcomes into one bucket and reported all as "referenced but not found in this RSF":

  1. ErrPackageNotFound — genuinely absent
  2. zero captured versions — present, nothing captured
  3. ErrMetadataUnavailable — present, but not this version

Only (1) is "not found". Walking flask reaches big-o and curio, and both are present in the RSF with zero captured versions — pyresolve versions big-o correctly reports that, while walk claimed the package didn't exist. That sends someone hunting for a typo in a name that's present and spelled correctly, and the case is common and expected: a package with no built distribution has no captured dependency metadata.

Now two separate lists in both text and JSON — absent for (1), no_dependency_data for (2) and (3), which share a practical outcome and are equally not absent. The CLI had been contradicting the distinction the index layer draws deliberately. Regression test included with a fixture containing one of each.

This also corrects something I reported earlier. An index-level walk that treated only Versions() errors as unresolvable reported "zero unresolvable names" for this same closure. That check couldn't see a package that exists with an empty version list — which is exactly what these two are. The honest figure is 131 reachable with 2 carrying no usable dependency data.

Judgment calls worth reviewing

  • Exit codes: 0 success, 1 usage/file error, 2 not-found — including "known package, uncaptured version", on the grounds that both are "what you asked for isn't in this RSF" rather than a usage mistake.
  • reorderArgs: stdlib flag stops parsing at the first positional, so walk flask --depth 3 silently ignored --depth. Flags are now moved ahead of positionals while preserving positional order. That shape is the documented example, so it had to work.
  • SetEscapeHTML(false) on JSON output, or >=3.9 renders as >=3.9.
  • walk surfaces names it couldn't traverse rather than silently dropping them.

Testing

83.6% coverage on cmd/pyresolve; -race -count=1 clean across all three packages; gofmt -l . silent; golangci-lint run ./... at CI's pinned v2.11.2 → 0 issues from the module root.

Built largely by a delegated agent; I reviewed the diff, independently re-ran every verification, and exercised it against the production file — which is how the walk bug surfaced.

🤖 Generated with Claude Code

jonyoder and others added 2 commits August 4, 2026 12:42
Adds cmd/pyresolve, a standalone binary for inspecting a PyPI Repository
Snapshot Format file directly, built only on the stdlib flag package and
the existing pypirsf/index packages.

- stats: package count, dependency-dictionary size, file size
- versions: every captured version of a package, sorted by PEP 440 order
- deps: requires-python/requires-dist/provides-extra for one version,
  defaulting to the highest captured version
- walk: breadth-first traversal of the highest-version dependency graph,
  explicitly documented (in --help and in its own output) as NOT dependency
  resolution -- it does not solve version constraints

Every command accepts --rsf (or $PYRESOLVE_RSF) and --json, and flags may
appear before or after positional arguments since the stdlib flag package
alone would otherwise miss `walk flask --depth 3`. Exit codes: 0 success,
1 usage/file error, 2 package not found. No net/http import anywhere.

Core command logic is factored to take an io.Writer and tested against a
real RSF fixture built with rsf.NewWriter, mirroring index/rsfindex_test.go.
Manually verified against the real ~936MB/932,861-package production RSF.
…as absent

Found while reviewing against the real production RSF rather than the
fixtures.

walk collapsed three distinct outcomes into one "unresolved" bucket and
reported all of them as "referenced but not found in this RSF":

  1. ErrPackageNotFound -- genuinely absent from the file
  2. zero captured versions -- present, but nothing captured
  3. ErrMetadataUnavailable -- present with versions, but not this one

Only (1) is "not found". Walking flask on the production file reaches big-o
and curio, and BOTH are present in the RSF with zero captured versions --
`pyresolve versions big-o` correctly says "no versions with captured
dependency data", while walk claimed the package did not exist. Reporting it
that way sends someone hunting for a typo in a name that is present and
spelled correctly, and the case is common and expected: a package with no
built distribution has no captured dependency metadata.

Now reported as two separate lists, in text and in JSON:
`absent` for (1), `no_dependency_data` for (2) and (3), which share a
practical outcome (the walk cannot continue through them) and are equally not
absent.

This is the same not-found-versus-unavailable distinction the index layer
already draws deliberately, so the CLI was contradicting the library it sits
on.

Adds a regression test with a fixture containing one of each, asserting the
JSON fields and that the text output says two different things. Verified the
real case now reads "2 present in this RSF but with no captured dependency
data, so the walk stopped there: [big-o curio]".

Also corrects a claim I made earlier from a weaker measurement: an index-level
walk that only treated Versions() ERRORS as unresolvable reported "zero
unresolvable names" for this same closure. That check could not see a package
that exists with an empty version list, which is exactly what these two are.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@jonyoder
jonyoder merged commit b72c0f4 into main Aug 4, 2026
2 checks passed
@jonyoder
jonyoder deleted the cli branch August 4, 2026 19:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant