Contributing

Contributing

Setup

See Getting Started. Run make dataset once, then make test.

Branching and commits

• Branch from main: feature/<short-name> or fix/<short-name>.
• Small, focused commits with a clear message. Open a pull request and get one review from a teammate who owns a different workstream.

Before opening a PR

• make test passes.
• Output is still deterministic (the determinism test covers this).
• If thresholds or rules changed: bump rubric_version, rerun make evaluate, and note whether the change was tuned on dev or held-out data.
• If the output schema changed: update docs/schema.md and the Result Schema page, and tell the dashboard owner.
• No secrets, credentials or personal data committed.
• AI assistance used for the change is noted in the PR (the project discloses AI use).

Adding a rubric rule

1. Define feature, normalization, window, threshold, direction and template in rubric.py.
2. Add a detector in compare.py and thresholds in config.py.
3. Add a dataset flaw family if it can be injected, and a test.
4. Document it on the Rubric page.

Adding data

Follow Dataset: provenance and license for the text and the recording, transcript, format, speaker notes, flaw type and severity, spans, and transformation parameters. Mark labels as human-reviewed only if a person reviewed them.

Style

Python, readable over clever. Keep functions deterministic. No hidden randomness; seed anything random.