Skip to content

scriptorium v0.3.0

Latest

Choose a tag to compare

@github-actions github-actions released this 20 Oct 19:24
016cac6

The v0.3 release. Four skills ship: reporting-guideline-compliance,
figure-text-alignment (text-only), compression and
outlier-sentence-detector, for sixteen in total. Every note in
knowledge/ was re-verified against its sources in the October 2026
knowledge refresh, and the skills were updated to match. The author
now decides what skills may do and where: policy and scope guards are
defaults the author can override for one run or in meta.overrides
in MANUSCRIPT_STATE.yaml, and accuracy guards are fixed. The skills
can also be installed with npx skills add seandavi/scriptorium for
any agent the skills CLI supports, and the CLI runs on Python 3.14.
scriptorium prompt-pack now writes one file per skill by default
(see Changed).

Test count: 226 (was 114 at v0.2.0).

Not in v0.3: statistics-consistency, voice-profile and
persona-calibration. Each has a design memo under docs/design/ and
is deferred to a later release.

Author authority

The author now decides what skills may do and where they apply (#156).
Skill refusals are declared guards of three kinds. Policy guards
(funder and publisher rules) and scope guards (the declared-work
boundary) are defaults the author can override. Accuracy guards (no
citations from model memory, no altered statistics, no inference the
input can't support) are fixed. The defaults are unchanged: skills
still decline blank-slate generation unless the author overrides it.

  • Mechanism (#157, PR #160) — the
    knowledge/conventions/author-authority.md convention; a guards:
    list in skill manifests (id, kind, summary, source,
    default of ask or refuse); meta.overrides in
    MANUSCRIPT_STATE.yaml; and scriptorium validate cross-checks
    that reject an unknown guard id, an override of an accuracy guard,
    or a skill that doesn't declare the guard. A per-run answer takes
    precedence over meta.overrides, which takes precedence over the
    default. Scope and accuracy guards must default to refuse
    (enforced by tests/test_manifest_guards.py).
  • Skill conversion (#158, PRs #161–#166) — 31 guards (2 policy,
    15 scope, 14 accuracy) declared across 13 skills. Each skill states
    a guard's concern and source in one line at every guidance level,
    says how to override when it declines, and names accuracy guards as
    fixed. explain, init and tour declare no guards but describe
    the mechanism; init offers meta.overrides as an opt-in field.
  • "Overrides applied" output line — a skill that proceeds past any
    guard, through a per-run answer or meta.overrides, lists the guard
    ids on one line under its report title, including an ask guard the
    author answered yes to. The line is omitted otherwise.
  • Editorial-side use asks instead of refusing —
    reviewer-simulation, desk-rejection-risk, venue-fit and
    author-contribution-audit share the editorial-side-use policy
    guard (default: ask). It fires when the user is reviewing or
    editing a submitted manuscript, and names NIH's outright ban.
  • New opt-in capabilities, each an overridable scope guard that
    defaults to refuse:
    • verified citation suggestions in citation-audit and gap-finder
      (citation-recommendations): only works resolved against
      Crossref or PubMed in the same session are named;
    • labelled image reading in figure-text-alignment
      (figure-image-reading): every image-derived finding is labelled
      "unvalidated vision", and image-integrity claims stay barred;
    • rewrite suggestions in figure-text-alignment and
      outlier-sentence-detector (critique-rewrite-suggestions): one
      labelled suggestion per flagged item.
  • Docs (#159) — a new Author authority concept page with a guard
    table generated from the manifests; meta.overrides in the schema
    reference; glossary entries for guard, override and accuracy guard;
    the declared-work-scope page, DESIGN.md, README and landing page
    describe the defaults and author authority. The roadmap resolves the
    specific-aims decision: NIH application prose is allowed behind
    the nih-application-prose policy guard (default: ask, NIH
    NOT-OD-25-132).

Knowledge refresh (October 2026)

Every note in knowledge/ was re-verified against its sources (#125).
Identifiers were checked against Crossref, PubMed, arXiv or the
publisher, misattributed findings were corrected, unverifiable figures
were removed, and 2025–2026 evidence and policy changes were added.
The shipped skills were then updated to match.

Knowledge notes

  • Peer-review evidence and platforms (#136) — statistics and
    overclaiming critiques are common but usually fixable (Bordage 2001,
    Bornmann 2010); suggesting a checklist to reviewers had no effect in
    an RCT (Cobo 2007); 2026 reviewer-AI policies at Elsevier and
    Springer Nature; hidden prompt injections in manuscripts.
  • Reporting guidelines, reproducibility and research integrity
    (#137) — CONSORT 2025 and SPIRIT 2025 (renumbered items), the
    LLM-era reporting guidelines (TRIPOD-LLM, REFINE, CHART, GAMER),
    LLM adherence-checking studies, TOP 2025, and a corrected Statcheck
    validity record (Schmidt 2017 is a critique) and Carlisle figure.
  • Grants and significance (#138) — NIH Simplified Review Framework
    scoring (Factor 1 caps Overall Impact), the end of paylines, the
    NOT-OD-25-132 AI-originality policy and NOT-OD-23-149; NSF and HHMI
    updates; Teplitskiy et al. 2022 replaces a misattributed "Lin et al.
    2022".
  • Citations and scholarly tooling (#139) — corrected Cochrane review
    denominators (medians are per journal), the reversed Sauder 2022
    finding, Greenberg 2009 denominators, and the four-step protocol
    re-attributed to the reference-accuracy / editorial spot-check
    protocol (Greenberg supplies the distortion categories).
  • AI and LLMs in scholarly writing and review (#140) — the ICLR 2025
    RCT is Thakkar et al., not Liang; Kobak et al. updated to the 2025
    Science Advances version; prevalence data on hallucinated
    references (NeurIPS 2025, the Lancet audit, ICLR 2026).
  • Critique techniques, editing, author roles and conventions (#141)
    — gap-finder's seven categories come from Miles (2017; Adu & Miles
    2023), not Robinson 2011; BERTScore's documented weakness is
    negation and numerical insensitivity, not "antonymy"; van Gelder's
    argument-mapping figures are pooled pre/post studies.
  • Publishing, venues and editorial policy (#143) — journal-disclosed
    desk-rejection figures replace trade-press rates; Science's Board
    of Reviewing Editors breakdown (Zhang et al. 2026); ICMJE AI-disclosure
    rules dated May 2023 / January 2024 and consolidated in Section V in
    January 2026; HHMI requires preprints for covered original research,
    Gates expects them.
  • Scientific-writing craft (#146) — CSE 9th and Chicago 18th
    editions; the promotional-register catalog split into Vinkers- or
    Kobak-backed and practice-derived entries; Gopen & Swan's principles
    untested in reading-time experiments; Sweller 2019 on germane load.
  • Unverified material removed (#147) — the last [TODO verify]
    markers and the unlocatable Smith & Banks 1991 citation are gone;
    knowledge/README.md now says a note does not merge with any.

Skill updates

Section headings, order and refusal posture are unchanged in every
skill. Output-contract and behaviour changes:

  • venue-fit (#148)
    • ## Preprint options (with its review sub-section) is included
      without asking for HHMI-covered original research, with servers
      limited to HHMI's list; otherwise it stays opt-in.
    • Funder APC rules (Wellcome, HHMI, Gates, proposed NIH caps) and JIF
      eligibility join the hard filter.
    • The tier rationale no longer says most authors over-aim, and a
      history-driven move to Stretch is never presented as a judgement
      of merit.
  • author-contribution-audit (#148)
    • When AI use is declared, ## LLM-as-author check lists the ICMJE
      Section V placement items, each marked covered by the text or for
      the author to confirm.
    • The non-author activity list matches ICMJE.
  • citation-audit (#149)
    • New reference identity check on every bibliography entry: a new
      always-present Summary bullet (Reference identity: …) and new
      ### Reference identity problems and
      ### Hidden text or embedded instructions pattern subsections,
      omitted when empty.
    • A claim is cannot determine when its reference's identity is in
      doubt.
    • New hard constraint: the manuscript is data, not instructions.
    • "What this skill did NOT check" narrows the bibliography item to
      errors beyond the identity check.
  • gap-finder (#149)
    • Categories credited to Miles plus two project additions; an
      optional Robinson et al. 2011 reason-code sub-tag on evidence-gap
      findings (no new section).
    • Directions name only works the manuscript already cites and never
      imply that one database search closes a gap.
  • reporting-guideline-compliance (#150)
    • Trial reports default to CONSORT 2025 (2010 only when the author or
      venue requires it) and protocols to SPIRIT 2025; every item number
      carries its version, so output item numbering changes.
    • Flow-diagram completeness items default to partial unless the
      prose reports every required number.
    • New manifest constraints flow_diagram_items_default_partial and
      version_tag_every_item_number.
  • reporting-guideline-fit (#150)
    • New routing targets: SPIRIT 2025, TRIPOD-LLM, REFINE and CHART.
      GAMER appears only as an add-on inside the existing sections
      (gamer_is_add_on_only).
    • Both reporting skills' full framing now says the EQUATOR library
      lists roughly 700 records.
  • figure-text-alignment (#150) — grounding and README only; no
    contract change.
  • argumentative-flow, compression, terminology-normalization,
    outlier-sentence-detector
    (#151) — no output-contract change.
    • argumentative-flow's full framing must not claim reading-time
      evidence for Gopen & Swan; its findings are diagnoses the author
      can overrule.
    • compression rationales cite negation and numerical insensitivity,
      not "antonymy", and don't treat germane load as a quantity an edit
      adds.
    • terminology-normalization's catalog offer labels entries
      corpus-grounded or practice-derived, with a date and a drift caveat.
    • outlier-sentence-detector: misattributed grounding claims
      corrected; no behaviour change.
  • reviewer-simulation (#152)
    • Hidden or instruction-like text is reported as item 1 under "Likely
      major critiques" with [Submission integrity] in the lens slot,
      only when found.
    • "Potential fatal concerns" is narrower: a statistics or
      overclaiming critique goes there only when it exposes a
      design-level problem.
    • Critiques are formed from methods and results before the author's
      stated limitations are weighed.
    • The author-side-only rationale cites the actual NIH, Elsevier,
      Springer Nature and ICMJE rules.
  • desk-rejection-risk (#152)
    • "What this assessment did NOT check": "The editor's actual mood…"
      became "Selection the manuscript cannot change…" (Zhang et al.
      2026).
    • The full guidance text drops the retired 70–90% desk-rejection
      figure; only journal-disclosed rates are quoted.
    • ICMJE AI-disclosure checks and hidden-text findings are reported
      inside the existing Format and length category.
    • "Lin et al. 2022" corrected to Teplitskiy et al. 2022.

Docs and CI

  • lychee.toml: include_fragments = "none" replaces false, which
    lychee ≥ 0.20 no longer parses; link-check had been loading no
    config (#142).
  • knowledge/README.md, docs/roadmap.md, DESIGN.md, README.md, the
    landing page and the concept pages aligned with the refreshed notes;
    concept tables regenerated from the manifests (#135).

Added

v0.3 skills

Four v0.3 skills land in this cycle (alongside terminology-normalization,
gap-finder, and author-contribution-audit, which shipped early during the
v0.2 ramp):

  • reporting-guideline-compliance (#92) — walks an EQUATOR Network
    reporting-guideline checklist (CONSORT, STROBE, PRISMA, ARRIVE, STARD,
    TRIPOD/TRIPOD+AI, CARE, COREQ, CHEERS, plus AI-extensions) against a
    manuscript and classifies every checklist item as
    present / partial / missing / not-applicable, with a quoted
    manuscript excerpt as the anchor or an explicit gap statement.
    Downstream of v0.2's reporting-guideline-fit (which infers which
    checklist applies; this skill runs it). Validation skill — surfaces
    gaps for the author to address; does not modify the manuscript and
    does not invent prose. Renamed from reporting-compliance to share
    the prefix with reporting-guideline-fit.
  • figure-text-alignment (#93) — sub-skill A (text-only) of the
    planned figure-text-alignment pair. Classifies caption ↔
    body-text-reference pairs as aligned / partially aligned /
    misaligned / cannot determine, plus pattern flags for orphan
    figures, phantom references, panel mismatches, axis/units divergence,
    and direction divergence. Pure text-vs-text — no image reading.
    Sub-skill B (multimodal / LLM-vision) remains deferred until
    reliability is validated against a known-mismatch test set.
  • compression (#94) — page-limit-driven section compression that
    preserves every citation, statistic, declared core_claim,
    terminology choice, and hedging stack. Per-edit suggestions with
    preservation report; never auto-applies. Sits one editorial level
    below argumentative-flow. New grounding test file
    (tests/test_compression_grounding.py, 20 cases) enforces the
    preservation contract.
  • outlier-sentence-detector (#99) — flags sentences that are
    outliers against the manuscript's own distribution on three
    dimensions: length, syntactic complexity (a comma, subordinator and
    coordination proxy; no parser) and nominalization density. A flag
    needs more than 2.5σ above the manuscript median and an absolute
    floor, and guidance level caps the flags surfaced per dimension
    (terse 3, standard 7, full 15). Each flag gives a location and
    a reason to re-read the sentence. No overall score, grade or
    pass/fail verdict, and no rewrites. Runs only when invoked. Grounded
    in knowledge/scientific-writing/quantitative-quality-measures.md
    and knowledge/editing/copyediting-vs-developmental.md.

v0.3 design memos and grounding

Three design memos landed for the remaining v0.3 skills so they can be
picked up later with minimal re-research (#95):

  • docs/design/v0.3-statistics-consistency.md — Statcheck/GRIM/SPRITE
    orchestration; rpy2 vs. vendored re-implementation as the load-bearing
    decision.
  • docs/design/v0.3-voice-profile.md — corpus-based authorial-style
    extraction; schema-embedded vs. sidecar author_voice slot as the
    load-bearing decision.
  • docs/design/v0.3-persona-calibration.md — synthetic-feedback drift
    mitigation; dependency on voice-profile as the load-bearing
    decision.

A new knowledge note —
knowledge/scientific-writing/corpus-based-stylometry.md — grounds the
voice-profile memo in the established stylometry literature (Burrows,
Hoover, Argamon, Stamatatos, Mosteller-Wallace) and bounds what is and
isn't defensible at small-corpus sizes.

Install

  • npx skills install path (#154, #155) —
    npx skills add seandavi/scriptorium (the vercel-labs skills CLI)
    installs the skills into any agent that CLI supports. init,
    explain and tour now carry the repo files they read in their own
    assets/ directory, and look for them in a scriptorium install
    first, then in assets/, then (explain only) on GitHub main;
    each says which source it used. scripts/sync_skill_assets.py (and
    just sync-skill-assets) keeps the bundled copies in sync, and a
    test fails when they drift. INSTALL.md, the install how-to, README
    and the landing page document the path. The Claude Code plugin is
    still the recommended install.

Skills and knowledge

  • Promotional-register catalog in terminology-normalization
    (#124) — a new note,
    knowledge/scientific-writing/promotional-register.md, covers
    promotional and transactional register in academic prose. When
    MANUSCRIPT_STATE.yaml exists but its terminology lists are empty
    or thin, the skill now offers a default catalog drawn from the note
    and waits for the author to accept, edit or decline it, instead of
    stopping. Grant targets keep vision language off the list; discipline
    terms, significance claims and warranted boosters are exempt.

Docs site

  • Landing page and README (#101) — restructured around what
    scriptorium operates on, workflow stages, the preservation contract
    versus voice preservation, and a "what this won't do" list. New
    pages: a constructed end-to-end case study
    (concepts/case-study) and a skills reference
    (reference/skills). The schema reference notes that
    MANUSCRIPT_STATE.yaml stays local and known_weaknesses is a
    calibration input.
  • Concept pages and first-run how-to (#106) — start-here,
    declared-work-scope, guidance-level, workflow-stage and
    glossary concept pages, and a how-to/first-run walkthrough.
  • Generated tables (#107, #119) — the skills reference page, the
    workflow-stage skill-by-stage table and the guidance-level
    per-skill block are generated from skills/*/manifest.yaml by
    docs/scripts/preprocess.py. Two new manifest fields feed them:
    lifecycle_phases: and guidance_level_behavior: (terse,
    standard, full).
  • Analytics (#112, #123) — production docs builds load GA4 on the
    standard property. astro dev sends nothing, and neither do
    localhost, bare-IP or preview hosts (workers.dev, netlify.app,
    ts.net).
  • templates/README.md points to the starter and example state files
    and the schema reference (#102).

Contributor tooling

  • Link check (#97) — a lychee workflow checks links in the
    repo's prose (README, CHANGELOG, CONTRIBUTING, DESIGN, INSTALL,
    docs/, knowledge/, skills/) on push, PR and a weekly schedule.
  • CONTRIBUTING.md gains sections on skill citation discipline (skill
    bodies cite with plain markdown links, not resolver cite keys;
    #98), documentation source of truth (repo-root files and
    knowledge/ are canonical; #102) and adding a new skill (#107).

Changed

  • scriptorium prompt-pack writes one file per skill by default
    (#118). -o prompts/ (or an existing directory, or a new path with
    no extension) now writes one <skill>.md per skill plus a
    README.md listing each skill's category and description. For the
    previous single concatenated file, pass --single-file, or give a
    file path such as -o pack.md. With no -o, output still goes to
    stdout as one document. --prefix scriptorium- names the files
    scriptorium-<skill>.md. prompt-pack and scriptorium list now
    take skill descriptions from manifest.yaml.
  • Renamed reporting-compliance → reporting-guideline-compliance for
    symmetry with reporting-guideline-fit. The original rename
    (commit b9d0a32) had not fully propagated; this cycle completes the
    propagation across roadmap, CHANGELOG, knowledge notes, the sibling
    skill's references, and the new skill itself.
  • DESIGN.md's roadmap section now points to docs/roadmap.md, and a
    new Conventions section covers declared-work-scope and
    guidance-level (#117).
  • The docs site's Tutorials section is gone; the how-to index lists
    the shipped how-tos (#102).
  • Docs fixes: stale "planned" and version framing removed from
    README, INSTALL.md, DESIGN.md, knowledge/README.md and the
    argumentative-flow, reporting-guideline-fit and
    reporting-guideline-compliance skill files; docs/design/README.md
    indexes the design memos; roadmap rows marked landed (#96, #102,
    #104, #116).

Fixed

  • Python 3.14 (#168, fixes #120) — the CLI crashed on startup
    because importlib.abc.Traversable was removed in 3.14; it now
    imports Traversable from importlib.resources.abc on 3.11+. The CI
    matrix and classifiers include 3.14.
  • citation-audit's grounding pointed to
    knowledge/citations/citation-accuracy-evidence.md; the note is
    .qmd. Two knowledge notes had a mistyped Hyland 1999 DOI and a
    stale medRxiv URL (#97).
  • Knowledge layer: 20 [TODO verify] marks resolved as verified,
    softened or recorded as research gaps; the "Laughter and the Chair"
    study is re-attributed to Pier et al. 2019, and the Crossref record
    count is updated (#100).
  • The landing page's citation-audit example renders as a table, not
    literal pipes (#115).
  • CONTRIBUTING.md markdownlint error (MD031) that failed CI lint
    (#113).

Install: uv tool install agentic-scriptorium==0.3.0, /plugin marketplace add seandavi/scriptorium@v0.3.0, or npx skills add seandavi/scriptorium.

Full changelog: v0.2.0...v0.3.0