Repository navigation
Development
| Path | What it is |
|---|---|
skills/science-study-evaluator/SKILL.md |
the procedure the agent follows |
skills/science-study-evaluator/references/ |
methodology.md (the checklist), fetch-routes.md, explaining.md
|
skills/science-study-evaluator/scripts/paper_meta.py |
the metadata script |
skills/science-study-evaluator/*.md, evergreen.json
|
research log, changelog, lessons, test runs, setup and state |
tests/test_paper_meta.py |
offline unit tests |
evals/ |
eval cases in claude plugin eval format |
ai-docs/ |
handoff, decisions and notes from work sessions |
They run offline: each database answer is an invented fixture served by a fake fetcher. From the repository root:
$ python3 tests/test_paper_meta.py 2>&1 | tail -n 1
OK (skipped=1)
$ echo $?
0
That is 15 tests passing and one live smoke test skipped; SSE_LIVE=1 runs it against the real APIs. CI runs the suite on Windows, macOS and Linux with Python 3.10 and 3.14 on every push, and the live test every Monday and on demand.
The cases check that the skill fires on the right prompts, stays quiet on decoys (editing your own manuscript, a t-test on your data), and, given a DOI, runs paper_meta.py and opens with a verdict. Results are in TESTS.md; 6 of 6 passed on 2026-10-11.
claude plugin eval . --ablation none --no-publish -j 5 --tag trigger --trust-plugin
claude plugin eval . --ablation none --no-publish -j 1 --tag bash --allow-tools Bash Write Edit WebFetch WebSearch --trust-pluginThe second needs a Bash grant and network, so on Windows it runs under WSL2.
The skill is evergreen. evergreen.json holds the schedule (tier moderate, about 30 days). A refresh runs the searches in RESEARCH.md, logs each finding as an R- entry with its sources, and edits methodology.md only for a change that adds or retires a check. It bumps the methodology's version line and logs a C- entry with the reason. MAINTENANCE.md has the procedure for machines without the evergreen plugin.
A release bumps the version in .claude-plugin/plugin.json, plugin.json, VERSION in paper_meta.py, metadata.version in SKILL.md and version in evergreen.json together, tags vX.Y.Z, and publishes a GitHub release with the changelog entry as notes.
Every output on these pages comes from a verification program run against the released tag. The program and its saved output are in the repository's ai-docs/notes/, so the next release can rerun it and compare.
This wiki describes science-study-evaluator 0.1.0 and was last updated on 2026-10-11. The skill and script are MIT licensed. Report problems in the issues.
Using it
The releases
Contributing
Elsewhere