Skip to content

fix: stop the flatline check crying wolf, and clear it on recovery - #158

Merged
GeiserX merged 1 commit into
mainfrom
fix/flatline-false-positives
Aug 2, 2026
Merged

fix: stop the flatline check crying wolf, and clear it on recovery#158
GeiserX merged 1 commit into
mainfrom
fix/flatline-false-positives

Conversation

@GeiserX

@GeiserX GeiserX commented Aug 2, 2026

Copy link
Copy Markdown
Owner

Two CodeRabbit findings on #149 that landed after it merged. I'd missed them; both defeat the restraint the feature was built around.

1. A broken collector read as a flatline

get_flatlined_services queried all historical rows. A collector that fails stops writing new ones — its earlier readings are still there and, naturally, unchanged. A removed service that kept its history behaved identically.

Those are different faults with different fixes, and reporting them as "running but not earning" is exactly the crying wolf that makes a report worth ignoring.

The query now requires a reading from today, which a failing collector does not produce.

2. A recovered service stayed in cooldown forever

record_alert suppresses a repeat within its quiet window, and nothing cleared the stored flatline alert when a balance started moving again. So a service that recovered and then went flat again inside that window was silently swallowed — the alert nobody gets.

_flatline_check now clears the cooldown for every subject that is no longer flat.

Regression tests

  • a collector with seven unchanged days but nothing today → not reported
  • a removed service with only old history → not reported
  • the collection hook clears exactly the recovered subject and leaves the still-flat one alone

Verification

  • ruff check . && ruff format --check . — clean
  • pytest --cov=app --cov-fail-under=901459 passed, coverage 94.25%

Two CodeRabbit findings on #149 that landed after it merged. Both defeat
the restraint the feature was built around.

A broken collector read as a flatline. get_flatlined_services queried all
historical rows, and a collector that fails simply stops writing new ones -
its earlier readings are still there and, naturally, unchanged. A removed
service that kept its history behaved the same way. Both are different
faults with different fixes, and reporting them as "running but not
earning" is exactly the crying wolf that makes a report worth ignoring.
The query now requires a reading from today, which a failing collector does
not produce.

A recovered service stayed in cooldown forever. record_alert suppresses a
repeat within its quiet window and nothing cleared the stored flatline
alert when a balance started moving again, so a service that recovered and
then went flat again inside that window was silently swallowed - the alert
nobody gets. _flatline_check now clears the cooldown for every subject that
is no longer flat.

Regression tests cover both: a collector with seven unchanged days but
nothing today, a removed service with only old history, and that the
collection hook clears exactly the recovered subject and leaves the
still-flat one alone.
@coderabbitai

coderabbitai Bot commented Aug 2, 2026

Copy link
Copy Markdown

Warning

Review limit reached

@GeiserX, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 31 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: da3909dd-0989-4d8f-aba3-25bcdd8ccaae

📥 Commits

Reviewing files that changed from the base of the PR and between fad13ec and 6d2ff36.

📒 Files selected for processing (3)
  • app/database.py
  • app/main.py
  • tests/test_earnings_flatline.py

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@codecov

codecov Bot commented Aug 2, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 94.24%. Comparing base (fad13ec) to head (6d2ff36).

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #158      +/-   ##
==========================================
+ Coverage   94.23%   94.24%   +0.01%     
==========================================
  Files          36       36              
  Lines        4475     4486      +11     
==========================================
+ Hits         4217     4228      +11     
  Misses        258      258              
Files with missing lines Coverage Δ
app/database.py 93.45% <100.00%> (+0.06%) ⬆️
app/main.py 96.87% <100.00%> (+0.01%) ⬆️
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@GeiserX
GeiserX merged commit 05206ee into main Aug 2, 2026
8 checks passed
@GeiserX
GeiserX deleted the fix/flatline-false-positives branch August 2, 2026 12:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant