Quality Judge calibration: qualified human reviewers wanted

[gavinjudd]

Git Mog v0.3.0 introduces a parser-backed Quality Judge Preview. It is preview-only and does not affect the canonical Git Mog score, winner, verdict, battle key, evidence, or score coverage. Quality score activation is structurally disabled while human calibration is pending.

We are recruiting qualified human reviewers for the preregistered, blinded pairwise calibration gate.

Who can review

Reviewers must be genuine people with material software-engineering experience reading, changing, testing, or maintaining production code. Model, agent, LLM-generated, synthetic, or maintainer-inferred judgments do not count.

The activation gate requires all of the following before any quality points can enter canonical scoring:

• at least 3 qualified human reviewers;
• at least 200 valid pairwise judgments;
• at least 75 distinct public repositories;
• every activated parser language represented;
• no single reviewer contributing more than 50% of judgments;
• the preregistered reliability, holdout, cohort, rank-correlation, leakage, and bias thresholds passing.

What reviewers do

For each anonymized pair, answer: Which sampled codebase is easier to understand, change, test, and trust? Reviewers may choose either side, a tie, or insufficient evidence and record confidence.

The review tool fetches exact immutable public source only at review time, keeps it process-only, and exports schema-validated judgments without source. Do not paste private source, credentials, tokens, personal information, repository popularity, or owner identity into judgments or this Discussion.

Read these contracts before volunteering:

• Calibration overview
• Reviewer rubric
• Preregistration and thresholds
• Product calibration boundary

The private holdout remains sealed and is not used for tuning. Calibration has not begun, and no human judgments have been collected yet.

If you meet the reviewer requirements and want to participate, reply with a short description of your relevant engineering experience and the languages you can review. Do not include employer-confidential details.

Even after a complete passing report, score activation requires a later release, independent review, exact version binding, and the separate explicit gate APPROVE QUALITY JUDGE SCORE ACTIVATION. That phrase is invalid while the calibration artifact is incomplete.

[gavinjudd]

Policy update: PR #25 retired the external human-reviewer calibration program. Quality Judge remains visible and informational, separate from the battle score, with scoreInfluence 0. Any future canonical use of code quality requires a separate, reviewed scoring release. This recruitment thread is now closed as historical context.