____ ______ _ __ _
/ __ \____ ___ ____ / ____/____(_) /_(_)___ ___ _____
/ / / / __ \/ _ \/ __ \/ / / ___/ / __/ / __ `/ / / / _ \
/ /_/ / /_/ / __/ / / / /___/ / / / /_/ / /_/ / /_/ / __/
\____/ .___/\___/_/ /_/\____/_/ /_/\__/_/\__, /\__,_/\___/
/_/ /_/
______
/ ____/___ ____ ___ ____ ___ ____ ____ _____
/ / / __ \/ __ `__ \/ __ `__ \/ __ \/ __ \/ ___/
/ /___/ /_/ / / / / / / / / / / / /_/ / / / (__ )
\____/\____/_/ /_/ /_/_/ /_/ /_/\____/_/ /_/____/
Infrastructure for scientific criticism you can inspect, challenge, and reproduce.
Automated tools can draft reviews in seconds. Science still needs criticism you can trace — from a manuscript passage, through a claim and its evidence, to adjudication and resolution.
OpenCritique Commons is open, system-neutral infrastructure for that chain. It helps researchers, tool builders, and reviewers share a common form for scientific concerns, keep human judgment in the loop, and evaluate systems without mistaking fixtures or preferences for evidence.
This is early work. It aims to be useful, honest, and welcoming to contributors.
| Package | Role |
|---|---|
opencritique_schema |
Shared shapes for concerns, evidence, adjudication, and resolution |
opencritique_registry |
Artifact storage, rights controls, and expert adjudication workflows |
opencritique_evaluation |
Matching, scoring, sensitivity checks, and signed scorecards |
opencritique_adapters |
Bridges from external reviewer systems into the shared schemas |
opencritique_acquisition |
Records for bringing external sources in with rights in mind |
The repo also includes schemas, fixtures, synthetic benchmark cases, docs, and tests — enough to develop against today. The evaluation story is still unfinished, and we say so.
Python 3.12+ is required.
git clone https://github.com/fraware/OpenCritique-Commons.git
cd OpenCritique-Commons
python -m venv .venv
source .venv/bin/activate # Windows: .venv\Scripts\activate
python -m pip install --upgrade pip
python -m pip install -e ".[dev]"
bash scripts/check.sh # lint, types, and testsOn Windows, Git Bash or WSL works well for scripts/check.sh. After the editable install you can also run ruff, pyright, and pytest directly.
Contributions are welcome — code, docs, tests, adapters, and carefully sourced cases.
- Read CONTRIBUTING.md and the Code of Conduct.
- Skim GOVERNANCE.md and SECURITY.md if you are changing process, APIs, or trust boundaries.
- Open an issue or pull request with a clear scope and the checks you ran.
Good first areas:
- Adapters — map another reviewer system into the shared schemas
- Docs — clarify concepts for newcomers
- Tests and verifiers — harden the contracts the rest of the project relies on
- Cases — add or refine examples with care for rights and provenance
Not sure where to start? Open an issue and say what you enjoy working on. We would rather help you find a fit than lose a contributor to setup friction.
- Evidence — criticism should point at something real in the manuscript or record
- Uncertainty — confidence and disagreement belong in the model, not only in footnotes
- Fairness — adjudication should be inspectable; preferences should not hide as metrics
- Rights — manuscripts, annotations, and cases carry their own permissions
- Reproducibility — schemas, runs, and scorecards should be checkable by others
This repository is early infrastructure. Packages, schemas, and fixtures exist so people can build and test together.
We are not claiming that any AI reviewer is accurate, calibrated, or better than another. Synthetic fixtures exercise software behavior. They do not establish scientific reliability. Performance-style claims stay off until stronger evidence and independent review are in place.
- Software: Apache License 2.0
- Cite the project with CITATION.cff
- Data (manuscripts, annotations, benchmark material) retains separate rights metadata
- CONTRIBUTING.md — setup, commits, and pull requests
- GOVERNANCE.md — how decisions are made
- SECURITY.md — how to report vulnerabilities
- CODE_OF_CONDUCT.md — community expectations