Seven fixes from the backlog sweep
- Markdown links no longer mint stub notes (#93, reported by @ThiagoMafra-Integrare).
[[Label]](https://…)— everywhere on GitHub READMEs, awesome-lists and Wikipedia-shaped pages — parsed as a wikilink, andrepair --stub(default on, and mandated after every session) turned each one into a real note that then topped_most-linkedand inflated PageRank.WIKI_LINK_REnow refuses a]]immediately followed by(; the hygiene filter also rejects unsubstituted{…}placeholders and unbalanced brackets. Resolver-minted stubs are excluded from both rankers and have their stale centrality zeroed, so vaults polluted before this fix recover on the nextgraph rank. The stub marker is centralized so the minting sites and the filter cannot drift. claims ingestfinds the claims (#69, reported by @fduple). The default scan read only the legacyresearch/temp/, while every producer and consumer had moved toresearch/runs/<tag>/temp/— so the claims table was empty in every real run andclaims matrix/targetswere dead on arrival. The default now scans both;--tagnarrows to the run. A zero-file default scan prints a hint instead of a bare zero.- A Semantic Scholar rate limit is reported as a rate limit (#70, reported by @fduple). A 429 was folded into "No metadata found" — an affirmatively wrong diagnosis. 429s now retry three times with backoff (honouring
Retry-After), and an exhausted budget surfaces asrate_limitedinsources scoreoutput rather thanmissing. Nothing is cached for a throttle, so the next run retries. OptionalS2_API_KEYsupport, scoped tosemanticscholar.orghosts only — the fetch helper is shared with Unpaywall and Europe PMC, and an unscoped header would have shipped the key to them. run resumenames a skill that exists, chapters reach the manifest, step 8 keeps its preflight gaps (#100, from @darlingm's contract audit in #88).resumeprintedhyperresearch-2for a skill namedhyperresearch-2-width-sweep, and CLAUDE.md advertisesresumeas the recovery path — the slug now comes from the installer's own step table.set_chapter()had zero callers; it is gone, and thechapter-planevent the partition skill already emits now registers the chapter, so a resumed dissertation reports what is actually pending. Step 8's subagent wrote to the same file the preflight had just written; it gets its own output and a merge step, and gap records carry theidthe dispatch already referenced.- Every pipeline constant is authored once (#101, from @darlingm's contract audit in #88).
citation_density_minsat unread on the profile while two gates hardcoded their own number; the draft orchestrator hardcoded word targets the synthesizer templated, so onpremierthe two halves of a run were 60% apart; step 4 templatedloci_analystsand then said "both" and wroteloci-a.json/loci-b.json. A new test asserts everyProfilefield is read somewhere outsideprofiles.py— and it found eight more dead fields (comparisons_tensions,source_tensions,tension_survey,tension_full_reads,corpus_critic_fetchers,citation_totals,utility_scoring,vault_check_interval_s), all now wired into the prose that had been hardcoding them.char_targets_no_word_boundaryis authored per gear at the 3:1 ratio the profile already used, which moved full's argumentative range from 20000–25000 to 15000–30000 — the old span was 1.25× against a 2× word span. - Citation density is per 1000 effective words, not per 1000 characters (#76). The floor meant something roughly 3× stricter for CJK than for English. The denominator reuses #64's boundary detection: whitespace tokens where they exist, characters ÷
chars_per_word_no_word_boundary(3.0) where they don't. The floor is re-derived ascitation_density_min = 9.0per 1000 words, which is what 1.5 per 1000 characters worked out to for English prose — English verdicts are unchanged at the boundary; CJK reports are now held to the same standard per unit of content rather than a looser one. hpr servebinds its port exclusively on Windows and reports the port it bound (#104).SO_REUSEADDRmeans "reclaim TIME_WAIT" on POSIX and "share a live port" on Winsock; a second bind on a running server's port succeeded on Windows. The server class now leaves it off there.run_serverreturns and prints the actually-bound port instead ofhttp://127.0.0.1:0, so the tests stopped monkeypatchingserver_activate. CI gains a singlewindows-latestjob on 3.11 — the platform this project is developed on — with the required-check names held stable.
Security review of the sweep
A review pass over the seven fixes above found six medium-severity problems — two of them pre-existing on main — and fixed each with a regression test.
- A hostile page could hang
hpr sync.WIKI_LINK_REwas quadratic on bracket floods: 20 KB of[[took 4.8 s, 40 KB took 19 s, and 100 KB did not finish in two minutes — and the pattern runs on the whole body of every fetched page. Excluding[from both character classes makes it linear (a 300 KB flood scans in under a millisecond). Pre-existing; surfaced because #93 touched the pattern. - The Semantic Scholar key followed cross-host redirects. httpx strips
Authorizationon a cross-origin hop but notx-api-key, so a 302 from semanticscholar.org would have carried the key to any host. Redirects are followed manually now, at most five, http(s) only, with headers recomputed from each hop's URL. Separately, the host check was a bareendswithon the raw netloc, whichevilsemanticscholar.organd a userinfo trick both passed; it is now an exact-or-subdomain match on the parsed hostname. claims ingest --tag ../../..scanned outside the vault, because pathlib replaces the base on an absolute segment. A tag narrows the scan only when the resolved directory sits underresearch/runs/; files that resolve outside the vault root — symlinks — are skipped.- One malformed claims file aborted the whole ingest. Claims JSON is agent-written from fetched content: a dict where a string was expected raised out of
.strip(), a list confidence raised out of sqlite, and 200 000 nested brackets hitRecursionError. Files are size-capped at 8 MB before reading, text fields are typed and bounded, and one bad claim becomes one error entry instead of a dead run. - A crafted
chapter-planevent could corrupt the manifest. The fold introduced by #100 stringified anychaptervalue into a key and stored anytitleverbatim; a non-dictchapterscrashedresume_position. Only a string or int id and a string title fold, both bounded; everything else stays inevents.jsonl. Retry-Afterwas already clamped to sixty seconds — confirmed with a parametrized test overnan,inf, negatives and forty-digit values, all of which fall back to the 2 s / 4 s ladder.
Deferred to its own issue: hpr run init applies no validation to the vault tag at all, so ../../x scaffolds outside the vault. Same bug class as the claims fix, but it touches every run command.