Skip to content

fix(index): the "lenient" Requires-Python policy was implemented as its exact opposite - #8

Open
jonyoder wants to merge 1 commit into
mainfrom
requires-python-leniency
Open

fix(index): the "lenient" Requires-Python policy was implemented as its exact opposite#8
jonyoder wants to merge 1 commit into
mainfrom
requires-python-leniency

Conversation

@jonyoder

@jonyoder jonyoder commented Aug 4, 2026

Copy link
Copy Markdown
Collaborator

What

An unparseable — or absent — Requires-Python now actually admits every interpreter, via a
new PackageMetadata.SupportsPython accessor. The field carries a warning not to call
Check on it directly.

The defect

The policy was right and the implementation was its exact inverse.

An unparseable Requires-Python is deliberately non-fatal — the comment said so, and pip
agrees (it catches InvalidSpecifier and treats the candidate as compatible). The value
stored to express that was version.Specifiers{}. But Specifiers.Check iterates its
specifier groups and returns false when there are none, so the empty set admits
nothing
. "Over-admits a candidate" was in fact "rejects every interpreter", and
types.go's "Zero value means unconstrained" was simply false.

Measured on the production snapshot: RequiresPython.Check(3.11) == false for
aiohttp-compress 0.2.1 (unparseable constraint).

⚠️ The absent case is the far bigger blast radius

The parsing branch is guarded by raw.RequiresPython != "", so a version declaring no
Requires-Python never enters it and keeps the same empty zero value. In the production
snapshot that is 2,027,153 versions — a consumer trusting the documented meaning and
calling Check directly rejects roughly a quarter of the corpus as incompatible with every
interpreter. Measured: falcon 0.1.8rc1, which declares none, also returns false for 3.11.

⚠️ The existing test defended the defect

TestRSFIndexUnparseableRequiresPythonIsLenient asserted RequiresPython.String() == "" and
stopped. It asked what the value renders as and never what it admits — so an inverted
policy shipped with a passing test named after the property it violated.

This is the fourth time in this family of repositories that an implementation and a test
written from the same misunderstanding have agreed with each other.

Both directions are now pinned, and I verified each fails independently:

Test Fails when
leniency, across 5 interpreters the empty-set guard is removed
the absent case, direct and via lookup the empty-set guard is removed
a real >=3.9 constraint still excludes 3.8 leniency is bought by returning true always

The third one matters: without it, "fix the leniency" could be satisfied by an accessor that
never rejects anything, which would be a worse bug than the one being fixed.

Why this is fixed in the caller and not in the library

An empty specifier set admitting everything is what the reference implementation does —
measured, not assumed:

SpecifierSet('') repr: <SpecifierSet('')>   len: 0
  '3.11' in SpecifierSet('') -> True
  '2.7'  in SpecifierSet('') -> True

And go-python-packaging's own andCheck already returns true for an empty group, so the
library is inconsistent with itself: one empty group is vacuously true, zero groups is false.
By that reading the library is wrong.

But flipping Check from reject-all to admit-all is a fail-open change to a published
library
, and that is not a call to make unattended — a consumer using it as a gate would
silently start admitting everything. Filed upstream for a human decision. This fix does not
depend on the outcome; if the library changes, SupportsPython becomes redundant rather than
wrong.

Worth noting: PPM never hits the trap, because src/requirements/filter.go guards
specs == "" and returns true before calling Check. That workaround existing in the only
other consumer is itself evidence
that "empty means unconstrained" is what callers expect
and currently each have to implement themselves.

Scope

This is F1 of 15 findings from the review. The rest are filed separately — including a
nondeterministic dependency answer for versions whose keys are PEP 440-equal but spelled
differently, pre-release candidates selected as "highest" against PEP 440's default for 13,335
packages, and a transitive parse failure destroying an entire walk. None are fixed here.

go test -race ./... green, gofmt clean, golangci-lint 0 issues.

…ts exact opposite

Found by an independent correctness review against the production PyPI snapshot,
and verified here before fixing.

THE DEFECT. An unparseable Requires-Python is deliberately non-fatal: the comment
said so, pip agrees (it catches InvalidSpecifier and treats the candidate as
compatible), and the value stored to express it was `version.Specifiers{}`. But
`Specifiers.Check` iterates its specifier groups and returns false when there are
none — so the empty set admits NOTHING. "Over-admits a candidate" was in fact
"rejects every interpreter." types.go's "Zero value means unconstrained" was
simply false.

⚠️ THE FAR BIGGER BLAST RADIUS IS THE ABSENT CASE, not the unparseable one. The
parsing branch is guarded by `raw.RequiresPython != ""`, so a version that
declares no Requires-Python at all never enters it and keeps the same empty
zero value. In a production PyPI snapshot that is 2,027,153 versions. A consumer
trusting the documented meaning and calling Check directly would reject roughly a
quarter of the corpus as incompatible with every interpreter.

⚠️ THE EXISTING TEST DEFENDED THE DEFECT. TestRSFIndexUnparseableRequiresPythonIsLenient
asserted `RequiresPython.String() == ""` and stopped there — it asked what the
value RENDERS AS and never what it ADMITS. So the inverted policy shipped with a
passing test named after the property it violated. This is the fourth time in this
family of repositories that an implementation and a test written from the same
misunderstanding have agreed with each other. The test now asserts the policy
across five interpreters, and I verified it fails without the fix; a separate test
asserts the opposite direction (a real, parseable constraint must still EXCLUDE an
interpreter outside it) and I verified that one fails if leniency is bought by
making the accessor unconditionally true. Neither is vacuous.

THE FIX is a PackageMetadata.SupportsPython accessor that treats an empty
specifier set as unconstrained, plus a warning on the field itself not to call
Check directly. Deliberately NOT fixed by changing the library: an empty specifier
set admitting everything is what the reference implementation does
(`Version("3.11") in SpecifierSet("")` is True, measured) and `andCheck` already
returns true for an empty GROUP, so go-python-packaging is inconsistent with
itself and arguably wrong — but flipping `Check` from reject-all to admit-all is a
FAIL-OPEN change to a published library, which wants a human decision rather than
an unattended one. Filed there; this fix does not depend on the outcome.

Also verified while confirming this, and NOT changed: PPM never hits the trap,
because src/requirements/filter.go guards `specs == ""` and returns true before
calling Check. That workaround existing in the only other consumer is itself
evidence that "empty means unconstrained" is the semantics callers expect and
currently have to implement themselves.

Refs rstudio/package-manager#19437

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant