Skip to content

Stage 1 dataset evaluator ignores redundant points; re-quantify agreement precision #18

Description

@jonfroehlich

Follow-up from #9 (this part was intentionally split out).

stage_one/dataset_evaluation/evaluate.py matches one-to-one on the TP side (claims one GT, then breaks — correct), but routes redundant generated points near an already-matched GT to an "ignored" counter instead of counting them as false positives. This makes the published Stage 1 agreement precision (94.03%) slightly optimistic; recall (92.45%) is unaffected.

Fix: count those redundant points as FPs (align with rampnet/metrics.py semantics), re-run against manual_labels/, and record the delta. Expected to be small — redundant generated points within 90 px of the same ramp are rarer for Stage 1 output than for model peaks — but it should be quantified rather than assumed.

Context: README erratum and docs/eval_protocol_verification.html.

🤖 Generated with Claude Code

https://claude.ai/code/session_01TAaPQDYhNDmYmxvrz4Lgm7

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions