Rubric

Rubric (rubric-1.0)

Every rule defines feature, normalization, window, threshold, direction of concern and feedback template. Source of truth: arachne/rubric.py and arachne/config.py.

Rule	Dimension	Feature	Normalization	Window	Fires when
PACE-1	Pacing	Articulation time per window (pauses excluded)	log(ref/part duration) minus the utterance-wide median ratio	3 words	robust z ≥ 2.5 and delta ≥ 0.20 log (about 22%) over ≥ 3 words
PAUSE-1	Pauses	Silent gap after a word	Seconds, matched gap	1 gap	participant gap ≥ 0.30 s longer than reference and z ≥ 2.5
PAUSE-2	Pauses	Silent gap after a word	Seconds, matched gap	1 gap	reference gap ≥ 0.25 s and participant gap ≤ 40% of it
PITCH-1	Pitch contour	Local spread of word pitch	Semitones re speaker median, divided by speaker robust SD; log ratio of window SD minus global median	5 words	z ≥ 2.5, change ≥ 0.59 log (about 1.8×), reference spread ≥ 0.3, over ≥ 4 words
ENERGY-1	Energy	Mean word energy	dB relative to the speaker's median word energy; difference minus global median difference	1 word, merged	z ≥ 2.5 and delta ≥ 4 dB over ≥ 3 words

A pause shorter than 0.15 s is treated as articulation, not a pause.

Scoring

dimension = 100 − Σ over regions [ 8 × (0.5 + severity) + 40 × coverage × severity ]
overall   = weighted mean of available dimensions
weights   = pacing 0.30 · pauses 0.20 · pitch 0.25 · energy 0.25
severity  = min(1, |delta| / (4 × minimum delta of the rule))

coverage is region duration divided by total speech duration. Withheld dimensions are dropped and the weights renormalised.

Insufficient evidence

Scores are withheld (status: insufficient_evidence) when estimated SNR is below 12 dB. Pitch findings are withheld when voiced coverage is below 25%. Alignment failures and implausible speaking rates return errors.

Example explanations (format)

• Pacing: "Words 18–25 (10.4–13.1s): articulation was faster than the reference, 182 vs 118 words/min (+54% after removing the overall pace difference of +2%). Rule PACE-1 (z=9.3)."
• Pause: "After word 31 ('...'): pause of 1.87s vs 0.06s in the reference (+1.81s longer). Rule PAUSE-1 (z=18.1)."

Numbers above are illustrative of the format.

Changing a rule

Edit config.py thresholds (or add a rule in rubric.py and compare.py), bump rubric_version, rerun make evaluate, and record what changed. Don't tune on the held-out speeches without saying so.