fix(tier3): validator average_score divides by validated+rejected instead of validated count - #7964
Conversation
…tead of validated count ValidatorAgent.validate_poa_proof maintains average_score as a running mean over VALIDATED scores (rejected proofs contribute no score to the numerator). But the update divided by total_validated + total_rejected, so any rejected proof dilutes the mean. Example: reject one proof, then validate one scoring 100 -> reported average_score is 50.0 instead of 100.0. The error appears whenever any proof is rejected and surfaces through get_stats() / PipelineOrchestrator.get_stats(). Fix: divide by total_validated (the count actually summed), matching the numerator. Add a regression test that fails on main (50.0) and passes with the fix; full tier3 pipeline suite stays green (33 passed).
|
Welcome to RustChain! Thanks for your first pull request. Before we review, please make sure:
Bounty tiers: Micro (1-10 RTC) | Standard (20-50) | Major (75-100) | Critical (100-150) A maintainer will review your PR soon. Thanks for contributing! |
IcanBENCHurCAT
left a comment
There was a problem hiding this comment.
LGTM — clean fix. average_score should only divide by validated proofs, not validated+rejected. Well-documented with a clear regression test. ✅
IcanBENCHurCAT
left a comment
There was a problem hiding this comment.
LGTM — clean fix. average_score divides by validated count only, not validated+rejected. Well-documented with clear regression test. ✅
|
Verified, with a scope note. The bug is real: CI is green. |
Problem
ValidatorAgent.validate_poa_proof(tier3/agents/validator_agent.py) maintainsaverage_scoreas a running mean over validated scores — rejected proofs contribute no score to the numerator (theelsebranch only bumpstotal_rejected). But the update divided by the combined count:Because the numerator only ever sums valid scores, dividing by
validated + rejecteddilutes the mean whenever any proof was rejected.Impact
total_validated=1,total_rejected=1, soaverage_score = (0*1 + 100) / 2 = 50.0instead of100.0.80.0, correct mean is75.0.The error appears whenever any proof is rejected and surfaces through
get_stats()and up throughPipelineOrchestrator.get_stats()asvalidator_stats.Fix
Divide by
total_validated— the count of scores actually summed — so numerator and denominator agree:The sibling
RewardAgent(tier3/agents/reward_agent.py) already follows this rule foraverage_reward— it divides the summed value bytotal_distributed, the count actually accumulated.Test
Added
test_average_score_excludes_rejected_from_denominatortotier3/tests/test_pipeline.py: reject one proof, validate one (score 100), assertaverage_score == 100. Fails on main (50.0), passes with the fix. The existingtest_validation_stats_trackingonly ever validates (no rejects), so it never exercised this path. Fulltier3/tests/test_pipeline.pysuite: 33 passed.