Skip to content

Getting Started

m4bwav edited this page Oct 11, 2026 · 1 revision

The skill is a folder of instructions (skills/science-study-evaluator/) plus one Python script. Any agent that reads SKILL.md skills can use it.

What runs where

Where The skill paper_meta.py
Claude Code, as a plugin yes yes, needs Python 3.10 or newer and internet access
Claude Desktop, claude.ai yes, as an uploaded skill (not tested) only if the app's sandbox reaches the scholarly APIs; otherwise the skill uses the agent's own web tools
GitHub Copilot, Codex, Cursor yes, as a personal skill (not tested) yes where the agent can run Python

The evals and the script ran on Windows 11 and on Ubuntu under WSL2. The unit tests run in CI on Windows, macOS and Linux with Python 3.10 and 3.14.

Install

Claude Code:

/plugin marketplace add m4bwav/science-study-evaluator
/plugin install science-study-evaluator@science-study-evaluator

Any other agent that reads personal skills: clone the repository and copy or link skills/science-study-evaluator into the agent's skills folder (~/.claude/skills/, ~/.agents/skills/ or ~/.copilot/skills/).

Claude Desktop and claude.ai: zip the skills/science-study-evaluator folder and upload it under Settings, Capabilities, Skills. Not tested.

Set a contact email

Unpaywall, the open-access database, requires a contact address, and without one the script skips it. Set SSE_EMAIL once (Windows: setx SSE_EMAIL you@example.org, then restart the agent; macOS and Linux: export SSE_EMAIL=you@example.org in your shell profile), or pass --email per run. See Configuration.

Check the script

From skills/science-study-evaluator/scripts:

$ python3 paper_meta.py --version
paper_meta.py 0.1.0
$ echo $?
0

First appraisal

Ask the agent in plain words: "is this study legit?" with a link or DOI, "does this paper actually show that?", or "fact-check this health article". It runs the script, reads the paper, writes the review and saves it in your folder (Configuration). In the eval run on 2026-10-11, the prompt "is this one solid? 10.1073/pnas.2528246123" ran paper_meta.py with --fulltext, fetched the whole article from Europe PMC, saved the review under ~/.science-study-evaluator/appraisals/ and opened with "Verdict: Weak".

Uninstall

Claude Code: /plugin uninstall science-study-evaluator@science-study-evaluator (not tested). Elsewhere, delete the skill folder. Your appraisals stay in your own folder until you delete it.

Clone this wiki locally