Correct the denominator in the model-agreement endpoint's API documentation

[mihow]

Summary

The API documentation for the model-agreement endpoint named the wrong denominator. Six of the field descriptions said the agreement percentages and confidence intervals were calculated over verified_with_prediction_count, when the endpoint actually divides them by comparable_count. Those two numbers are not the same: a verification that carries no taxon — someone leaving a comment rather than naming a species — has a machine prediction but nothing to compare it against, so it counts toward the first and not the second. Anyone reading the schema and recomputing a percentage from the counts we return would get a different answer than the endpoint gives them.

Nothing about the endpoint's behaviour changes here. The numbers it returns were always calculated against comparable_count, and the serializer's own class docstring already said so. Only the per-field descriptions had drifted away from it, and those are what a consumer actually sees in the browsable API and the generated OpenAPI schema.

The same confusion had been shipped in the frontend tooltips, and was corrected there in #1308.

List of Changes

#	What changes for a consumer	How
1	The description of the any-rank agreement percentage now names the comparable set, so recomputing it from the returned counts matches what the endpoint reports	agreed_any_rank_pct help text
2	The same correction for the coarser-rank percentage	agreed_coarser_rank_pct help text
3	The four confidence bounds now describe the right condition for coming back empty: they are null when nothing is comparable, not when nothing has a prediction	agreed_exact_ci_low / _high and agreed_any_rank_ci_low / _high help text
4	Two claims that nothing was pinning are now covered by tests	TestModelAgreementForProject gains an assertion that the any-rank percentage divides by comparable_count, plus a case where every verification is comment-only, so verified_with_prediction_count is non-zero while comparable_count is zero and all four bounds come back null

Test plan

• TestModelAgreementForProject — 5 tests green (4 existing plus the new confidence-interval case).
• -k Agreement -k Stats — 25 tests green.
• Pre-commit hooks pass, flake8 included.
• Checked against the implementation before editing: wilson_interval (ami/utils/stats.py:33) returns None when its total argument is zero, and model_agreement_for_project passes comparable as that argument. The percentages are built by _pct(..., comparable) (ami/main/models_future/occurrence.py:294-295, 325).
• The repository has no committed OpenAPI schema file, so nothing needs regenerating alongside this.

Summary by CodeRabbit

• Bug Fixes

  • Corrected model agreement help text to describe agreement percentages and confidence intervals using the comparable occurrence count.
  • Improved agreement reporting when verified occurrences lack a human-assigned taxon, including accurate denominator handling and unavailable confidence intervals when no occurrences are comparable.

• Tests

  • Added regression coverage for confidence interval and agreement-rate calculations in these scenarios.

[netlify]

✅ Deploy Preview for antenna-preview canceled.

Name	Link
🔨 Latest commit	b4e67e4
🔍 Latest deploy log	https://app.netlify.com/projects/antenna-preview/deploys/6a98aff2d675d800081256aa

[netlify]

✅ Deploy Preview for antenna-ssec canceled.

Name	Link
🔨 Latest commit	b4e67e4
🔍 Latest deploy log	https://app.netlify.com/projects/antenna-ssec/deploys/6a98aff2e1aaa6000848f889

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Team

Run ID: 78c869a3-6ab6-4098-8e39-309fc67846cb

📥 Commits

Reviewing files that changed from the base of the PR and between 8698cc8 and b4e67e4.

📒 Files selected for processing (2)

• ami/main/api/serializers.py
• ami/main/tests.py

Included review availability: Your plan provides up to 1 included review per hour; 0 remain after this review.

---

📝 Walkthrough

Walkthrough

The change updates model-agreement help text to use comparable_count and adds regression coverage for agreement percentages and Wilson confidence intervals when predictions lack human taxa.

Changes

Model agreement metrics

Layer / File(s)	Summary
Agreement denominator help text ami/main/api/serializers.py	Agreement percentage and Wilson confidence interval help text now references comparable_count.
Agreement metric regression tests ami/main/tests.py	Tests verify 100% any-rank agreement for a comparable occurrence and null confidence interval bounds when comparable_count is zero.

Estimated code review effort: 2 (Simple) | ~10 minutes

Merge Risk: ⚪ Minimal · up to b4e67

This change corrects model-agreement documentation and adds tests without changing endpoint behavior, so no actionable merge-blocking risk remains after normal checks and review.

Suggested reviewers: mohamedelabbas1996

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly identifies the main change: correcting the denominator in the model-agreement API documentation.
Description check	✅ Passed	The description explains the problem, affected fields, consumer impact, test coverage, validation steps, and deployment impact. It omits some template headings, but it is sufficiently complete and dir…
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 2 functions across 2 files.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

Full details: Description check

Explanation

The description explains the problem, affected fields, consumer impact, test coverage, validation steps, and deployment impact. It omits some template headings, but it is sufficiently complete and directly relevant.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch fix/model-agreement-help-text

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[Copilot]

🟢 Approval recommended

The changes are narrowly scoped to documentation help text plus targeted tests that validate the documented denominator/CI behavior without altering endpoint logic.

Pull request overview

This PR fixes drift in the ModelAgreement API schema documentation so that agreement percentages and Wilson confidence interval fields correctly describe comparable_count (not verified_with_prediction_count) as the denominator/guard condition, aligning the OpenAPI/browsable-API help text with the endpoint’s actual behavior.

Changes:

• Update ModelAgreementSerializer field help_text to reference comparable_count for agreed_*_pct and CI nullability.
• Add tests that (1) assert agreed_any_rank_pct uses comparable_count and (2) verify CI bounds return None when comparable_count == 0 even if predictions exist.

File summaries

File	Description
ami/main/api/serializers.py	Corrects per-field help text to name comparable_count as the denominator / CI-null condition.
ami/main/tests.py	Adds coverage ensuring the agreement pct denominator and CI nullability match comparable_count semantics.

Review details

• Files reviewed: 2/2 changed files
• Comments generated: 0
• Review effort level: Lite

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.