Skip to content
m4bwav edited this page Oct 11, 2026 · 2 revisions

science-study-evaluator

A skill for AI agents that answers "is this study legit?" You share a news article, a press release, a DOI or a paper. The agent finds the primary source, reads the paper rather than the coverage, checks conflicts of interest, works out where the headline outruns the findings, and gives one of five verdicts with the reasoning explained: Solid, Real but overhyped, Weak, Junk / predatory, or Fabricated / misattributed.

It ships with paper_meta.py, a standard-library Python script that pulls a paper's Crossref, Unpaywall and PubMed records in one call. It also flags retractions and saves an open article's full text as greppable text.

What it is not: a literature search, a reviewer for your own manuscript, or medical advice. It judges how well a study supports its claims.

The smallest example

Ask your agent "is this study legit?" with a link, and the skill runs the script first. Run it yourself from skills/science-study-evaluator/scripts, here on a closed paper with no contact email set:

$ python3 paper_meta.py 10.1111/jssr.70101 --similar 0 | grep -v '^Abstract'
== CROSSREF
Title: Cognitive Ability and Christian Nationalism
Journal: Journal for the Scientific Study of Religion | Publisher: Wiley | Type: journal-article
DOI: 10.1111/jssr.70101 | Published: 2026-09-11 | Online: 2026-09-11
  received: 2026-06-07
  accepted: 2026-08-21
  published: 2026-09-11
Authors:
  - Samuel L. Perry  [Department of Sociology University of Oklahoma  Norman Oklahoma USA]
License: http://onlinelibrary.wiley.com/termsAndConditions#vor
References deposited: 77
== UNPAYWALL
skipped: no contact email (pass --email or set SSE_EMAIL)
== PUBMED
no PMID (too new, or the journal is not indexed in PubMed)
== NEXT
Open-copy check skipped (no email): rerun with --email or SSE_EMAIL, or try the publisher page; if it stays closed, grade at abstract level and say so.
$ echo $?
0

The abstract lines are left out of that run with grep -v. Every output on these pages was produced on 2026-10-11 by version 0.1.0 against the live public databases on Linux, so a record that changed since then prints differently.

Pages

  • Getting started: install the skill in Claude Code, Claude Desktop or another agent, and set the contact email.
  • How an appraisal works: the four steps, the six checks every review runs, and the verdict scale.
  • Commands: every option of paper_meta.py, its output sections, streams and exit codes.
  • Configuration: SSE_HOME, reader.md, SSE_EMAIL and NCBI_API_KEY.
  • Errors and edge cases: closed papers, journals outside PubMed, retractions, blocked sites, bad input.
  • Recipes: JSON from Python, several DOIs at once, searching a full text.
  • FAQ: what people ask.
  • Versions and upgrading: what each release changed.
  • Development: tests, evals, CI and how the checklist refreshes itself.

Clone this wiki locally