Harden linkcheck.yml, declare known false positives, and pin the action to v1 - #592
Merged
Conversation
Closes #566. Hardening, as filed ------------------- The three improvements from #566, already proven in lecture-python-intro #783: - `contents: read` in permissions. The block only granted `issues: write`, and because `permissions:` resets every unlisted scope to none, the `gh api .../releases/latest` call was running with `contents: none`. - `set -euo pipefail` and single-asset selection. The old jq emitted one line per matching asset with nothing to stop an empty or multi-line result flowing into curl; it now selects the first match and fails with a clear message when there is none. - `curl -fsSL` rather than `-sL`, so an HTTP error page is a download failure rather than a cryptic tar extraction error. Verified the new jq against this repo's current latest release: it returns the publish-2026aug03 tarball, same URL as the old form. Known false positives --------------------- action-link-checker v1.1.0 adds an `ignore-patterns` input (QuantEcon/action-link-checker#2, shipped in #3 there). This declares the same exemptions the repo already keeps in `lectures/_config.yml` under `linkcheck_ignore`, which Sphinx's built-in checker honours but this checker never sees -- it scans built HTML, not the Sphinx config. The regex syntax is identical, so the entries move across unchanged. FRED is the one that mattered: it throttles datacenter IP ranges, so it times out from a runner while returning 200 in under half a second from a normal network. Seventeen duplicate issues were filed for those links between April and July. Checked the patterns against the actual published HTML from the current release, using the shipped v1.1.0 code rather than a reimplementation: across 33 files and 955 external anchors, 7 anchors match, 6 distinct URLs, and no FRED link remains checked. The other 948 are unaffected. Pinning ------- Moves from `@main` to `@v1`. #566 recorded the decision to stay on `@main` on the grounds that it is a first-party action, and that holds, but v1.1.0 shipped a default-on behaviour change (a recurring finding now refreshes one issue instead of opening a new one weekly) which reached this repo the moment it merged. `@v1` still takes patches and features automatically; it just means a future change of that kind is something we adopt rather than receive. The other three consumers -- lecture-python-intro, lecture-python-advanced.myst and continuous_time_mcs -- should move to `@v1` too, so the series does not end up split across pinning schemes. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
Pull request overview
This PR updates the repository’s scheduled link-check workflow to be more robust when downloading the latest release HTML archive, to suppress recurring known-false-positive links during checks, and to pin the QuantEcon link checker action to the @v1 major version tag.
Changes:
- Harden release asset discovery and download/extract steps (permissions, stricter shell flags, safer
curl). - Pin
QuantEcon/action-link-checkerfrom@mainto@v1. - Wire up
ignore-patternsto carry over the existing Sphinxlinkcheck_ignoreexemptions into the HTML-scanning checker.
Copilot review of #592. The comment said the entries move across "unchanged" because the regex syntax matches, which glossed over a real difference: Sphinx's linkcheck_ignore is applied with re.match, anchored at the start of the URL, while action-link-checker applies its ignore-patterns with re.search. A pattern copied from _config.yml can therefore match more here than it does there, never less. It makes no difference to the nine current entries -- each begins with https://, so matching mid-URL would need a URL with another URL embedded in it -- but a future maintainer adding a bare domain would get an entry that this checker honours and Sphinx silently ignores. Comment text only; the patterns and the behaviour are unchanged. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #566, and wires up the
ignore-patternsinput that shipped in action-link-checker v1.1.0.The hardening from #566
Applied as filed, matching what already landed in QuantEcon/lecture-python-intro#783:
contents: readadded topermissionsissues: write, andpermissions:resets every unlisted scope tonone— sogh api .../releases/latestwas running withcontents: noneset -euo pipefailand single-asset selectioncurlcurl -fsSL-f, an HTTP error page is piped intotarand surfaces as a cryptic extraction error rather than a download failureVerified the new selector against this repo's current latest release — it returns the
publish-2026aug03tarball, the same URL the old form produced.Known false positives
This repo has kept a
linkcheck_ignorelist inlectures/_config.ymlfor years. Sphinx's built-in checker honours it; this checker scans built HTML and never sees the Sphinx config, so the same links were re-reported every week. The regex syntax is identical, so the nine entries move across unchanged.FRED is the one that mattered. It throttles datacenter IP ranges, so it times out from a runner while returning
200in under half a second from a normal network — andhttps://fred.stlouisfed.org/.*has been in_config.ymlsince April. Seventeen duplicate issues were filed for those two links between April and July before the batch was closed this morning.Verification against the real site
Rather than assume the patterns match, I downloaded the current release tarball and ran the shipped v1.1.0
compile_ignore_patterns/is_ignoredagainst the actual built HTML:fred.stlouisfed.org/twice,/series/UNRATEonce), plusopenai.com/,reddit.com/,keras.io/and the pinned matplotlib source linepandas.htmlorpolars.htmlNote that
openai.com/matched the barehttps://openai.comentry —re.searchsemantics, so the trailing slash is not a problem. Four entries in the list match nothing in the current build; they are harmless and kept so the two lists stay identical.Correcting an estimate I gave earlier: the next scheduled run would have reported three FRED URLs, not six. The higher number came from counting mentions in the Markdown source, which includes prose references that never become anchors.
Pinning to
@v1#566 recorded a deliberate decision to stay on
@mainbecause this is a first-party action, and that reasoning still holds. What changed is that v1.1.0 shipped a default-on behaviour change — withcreate-issue: true, a recurring finding now refreshes one open issue instead of opening a new one each week — and it reached this repo the instant it merged, with no gate.@v1is a moving tag, so patches and features still arrive automatically. It only means a change of that kind becomes something adopted rather than received.This should not stay a one-repo decision. Three other repos use this action —
lecture-python-intro,lecture-python-advanced.mystandcontinuous_time_mcs— and a series split across two pinning schemes is worse than either scheme. They also each need their ownignore-patternsbefore their FRED reports stop. Happy to open the matching PRs.After this merges
The next scheduled run is 23:00 UTC. It should be the first clean one in months. If anything is still reported,
ignored-countin the job log shows exactly what was skipped.