Serve a provider's inference region as a facet - #447
Conversation
Fish Audio runs inference in us-west-1 while our workers are in us-east-1, so their latency carries a cross-country round-trip. Records the region on the registry entry and serves it as an Inference region facet; models stay unlabelled unless the provider confirmed where they serve from.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
Warning Review limit reached
Next review available in: 47 minutes You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (3)
WalkthroughRegistered models now support optional serving regions. Fish Audio models declare Suggested reviewers: 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches 💡 1📝 Generate docstrings 💡
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Fish Audio serves inference from us-west-1 while our workers run in us-east-1, so their latency carries a round-trip we can't subtract. Models served outside our region now show a globe beside the model on the box plot, the WER bars and the comparison table, a row in each chart tooltip, and an Inference region filter chip. The methodology note says why the gap exists.
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@web/components/overview/AboutMethodology.tsx`:
- Line 57: Update the methodology text in AboutMethodology so it distinguishes
excluded connection setup from included request-path network latency, avoiding
the claim that measurements reflect only the model. Replace “wherever they've
told us” with wording that indicates a provider region is labeled only when
confirmed.
In `@web/lib/utils/facets.ts`:
- Around line 232-237: Update the facet-group filtering condition in the
region/category construction logic so REGION_CATEGORY also requires at least one
shared region value before the group is pushed. Preserve the existing
SOURCE_CATEGORY and other-category thresholds, ensuring globally defined regions
with no visible model values do not render an empty facet.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: 5d95b06f-7cf1-4961-bbd6-3fba721db5f3
⛔ Files ignored due to path filters (1)
web/lib/api/generated/schema.tsis excluded by!**/generated/**
📒 Files selected for processing (16)
runner/src/coval_bench/registries/models.pyweb/components/charts/d3/BoxPlot.tsxweb/components/charts/tooltips/BarTooltip.tsxweb/components/charts/tooltips/ScatterTooltip.tsxweb/components/dashboard/BoxPlotSection.tsxweb/components/dashboard/LatencyAccuracySection.tsxweb/components/dashboard/ModelComparisonSection.tsxweb/components/dashboard/ModelComparisonTable.tsxweb/components/dashboard/QualityBarSection.tsxweb/components/overview/AboutMethodology.tsxweb/components/shared/DedicatedInferenceInfo.tsxweb/components/shared/InferenceRegionInfo.tsxweb/components/shared/TooltipBadge.tsxweb/hooks/useDashboardState.tsxweb/lib/utils/facets.tsweb/types/chart.types.ts
🚧 Files skipped from review as they are similar to previous changes (1)
- runner/src/coval_bench/registries/models.py
An exempt category skipped the value-count check entirely, so benchmarks with no region-labelled model rendered an empty group header. Region now needs one value rather than two. Also drops the claim that latency reflects only the model, since excluding connection setup doesn't exclude the request round-trip.
2 participants
Fish Audio flagged that we measure them from us-east-1 while their inference runs in us-west-1, so their TTFA carries a cross-country round-trip that isn't the model's own speed.
This records an optional
regionon the registry entry and serves it as an "Inference region" facet on/v1/providers. API-only. The web change that renders the badge follows once this is deployed, it needs prod/v1/providersto serveregionbefore the badge has anything to show.Greptile Summary
Adds an optional confirmed inference region to registered models and exposes it as a raw-valued facet through
/v1/providers.us-west-1.regiontag category and “Inference region” display label.Confidence Score: 5/5
The PR appears safe to merge; no concrete blocking or independently actionable non-blocking defect remains.
The optional field defaults safely, all current Fish Audio models are labeled consistently, the API schema accepts the new category, and provider runtime configuration remains isolated from the metadata field.
Reviews (1): Last reviewed commit: "Serve a provider's inference region as a..." | Re-trigger Greptile
Context used: