Skip to content

The offline link gate never checks absolute self-links #82

Description

@lesnik512

Follow-up to #66, which adopted lychee --offline as a blocking gate in every non-archived repo
but the two templates.

The gap

--offline is what makes the gate deterministic: it blocks network requests and excludes
external URLs rather than erroring on them. But an absolute
https://github.com/modern-python/<this repo>/blob/main/<path> link pointing back at your own
repo is an external URL by that rule, so it is never checked. A relative link to a deleted file
fails the gate; the same link written absolutely passes forever.

This is not theoretical. The #67 migration deleted planning/ and architecture/ in 21 repos and
found two live instances that the gate would have waved through:

  • httpware — 7 links in docs/ pointing into architecture/, all about to 404.
  • faststream-concurrent-aiokafka — one inside middleware.py's nack guard, pointing at
    planning/decisions/…. It was deliberately absolute so PyPI users without a checkout could open
    it, which makes it user-visible on the package page.

Both are fixed. Neither was caught by the gate; both were caught by a human reading the diff.

Online mode is not the fix

Dropping --offline would check them, but it re-introduces exactly what #66 rejected: network
non-determinism in a blocking gate, plus unauthenticated github.com rate limits in CI. #66's
decision to keep the external check weekly and .github-only still stands.

Proposed fix: --remap, still fully offline

A self-link's target is in the repo, so it needs no network — just resolution against the working
tree. lychee --remap does this. Measured against a synthetic case (one valid self-link, one
broken self-link, one broken relative link, one tag-pinned URL, one cross-repo URL):

--offline                          → 1 error   (broken relative only; broken self-link EXCLUDED)
--offline + --remap '…/(?:blob|tree)/main/(.*) file:///input/$1'
                                   → 2 errors  (both, with the remap shown in the message)

The remap line for a repo would be:

            --remap
            'https://github.com/modern-python/<repo>/(?:blob|tree)/main/(.*) file://$PWD/$1'

Two behaviours worth keeping, both confirmed in that test:

  • Tag-pinned URLs stay excluded. …/blob/0.6.1/planning/releases/0.6.1.md does not match
    blob/main/, so it is left alone — correct, since it deliberately points at a historical tree
    that still resolves. (faststream-concurrent-aiokafkaOffer the link checker to the other repos as a reusable workflow #64 is exactly this case.)
  • Cross-repo links stay external. …/modern-di/blob/main/README.md from another repo is not
    remapped, which is right — it is genuinely someone else's tree.

The alternative, and why it is not enough on its own

Rewriting absolute self-links to relative removes the class of problem, and where possible it is
the better fix. But it cannot cover every case: a URL in a docstring rendered on PyPI has no
checkout to be relative to, which is precisely how the faststream-concurrent-aiokafka instance
arose. So the gate still needs to cover what remains.

One thing to decide

#66 justified copying the gate into each repo rather than sharing a reusable workflow on the
grounds that "the config is 5 static lines with no parameters, so there is nothing to drift." A
remap rule contains the repo name, so that argument weakens: the config becomes parameterised
and a copy-paste error (wrong repo name in the pattern) would silently disable the new check
without failing anything. Worth deciding whether that changes the reusable-workflow call, or
whether a cheap guard is enough — e.g. a test asserting the remap pattern matches the repo's own
origin URL.

Scope

The 25 repos carrying the gate. Also worth a sweep for existing absolute self-links at adoption
time, since this gate has never checked them: grep -rn "github.com/modern-python/<repo>/\(blob\|tree\)/main/".

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions