Skip to content

feat: filter alerts/sequences/slack by daily fire risk score#583

Merged
MateoLostanlen merged 36 commits intomainfrom
feat/risk-api-filter
May 5, 2026
Merged

feat: filter alerts/sequences/slack by daily fire risk score#583
MateoLostanlen merged 36 commits intomainfrom
feat/risk-api-filter

Conversation

@MateoLostanlen
Copy link
Copy Markdown
Member

Why

Operators report way too many alerts on rainy / low-risk-weather days, when there is essentially no chance of an actual wildfire. The detection model still picks up smoke-like patterns (clouds, mist, exhaust) and they flood Slack and the alerts list, drowning real signal. This PR cross-references the per-camera daily Fire Weather Index from pyro-risk-api and drops low-confidence sequences when the local fire risk is low or very_low — the regime where false positives dominate.

Summary

  • Pulls a daily FWI class per camera from pyro-risk-api (cached in RAM, refreshed at 04:00 UTC) and uses it to drop low-confidence sequences during low-risk weather. On low days, drops sequences with max_conf < 0.45; on very_low, drops below 0.6; on moderate+ (or unknown / risk-api unreachable), behaviour is unchanged.
  • Applied to /alerts/unlabeled/latest, /alerts/all/fromdate, /sequences/unlabeled/latest, /sequences/all/fromdate, and to Slack notifications at sequence creation. Alerts that lose all their sequences are dropped from the response.
  • Filter is pushed into the SQL WHERE clause (per-camera CASE expression on Sequence.max_conf), so LIMIT/OFFSET pagination stays exact instead of returning sparse pages.
  • The /all/fromdate endpoints query the risk-api /scores/{date}?organization_id=... per request so the threshold matches the requested day; the daily cache only powers the "today" code paths.
  • All four list endpoints accept an optional ?risk_score=<fwi_class> query param that overrides the per-camera lookup and applies a single class to every sequence — useful as a kill-switch (risk_score=moderate disables filtering) or to preview a different risk regime without touching the cache.

Schema change

Adds a nullable max_conf column to sequences, maintained at ingest:

  • New sequence: computed from the primary bboxes of the bootstrap detections only (sibling boxes in others_bboxes represent unrelated detections in the same image and would otherwise inflate the score).
  • Existing sequence extension: atomically bumped via portable CASE WHEN (max_conf := GREATEST(max_conf, candidate) semantics, runs on Postgres + SQLite).
  • An Alembic migration adds the column and backfills existing rows from Detection.bbox only.
  • Note: max_conf is monotonic non-decreasing — not recomputed when a detection is deleted/reassigned.

Config

Adds RISK_API_URL, RISK_API_LOGIN, RISK_API_PWD, RISK_REFRESH_HOUR_UTC (default 4) to .env.example and docker-compose.yml. FWI thresholds live in code as FWI_MIN_CONF (dict in services/risk.py) covering all 6 EFFIS classes. When RISK_API_URL is unset, the feature is fully disabled (fail-open).

@codecov
Copy link
Copy Markdown

codecov Bot commented May 5, 2026

Codecov Report

❌ Patch coverage is 94.14414% with 13 lines in your changes missing coverage. Please review.
✅ Project coverage is 89.80%. Comparing base (8917c95) to head (ce064b8).

Files with missing lines Patch % Lines
src/app/services/sequence_confidence.py 77.77% 10 Missing ⚠️
src/app/api/api_v1/endpoints/alerts.py 97.05% 1 Missing ⚠️
src/app/api/api_v1/endpoints/detections.py 92.85% 1 Missing ⚠️
src/app/services/risk.py 98.36% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #583      +/-   ##
==========================================
+ Coverage   89.00%   89.80%   +0.79%     
==========================================
  Files          53       55       +2     
  Lines        2219     2422     +203     
==========================================
+ Hits         1975     2175     +200     
- Misses        244      247       +3     
Flag Coverage Δ
backend 89.76% <94.14%> (+0.84%) ⬆️
client 90.40% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copy link
Copy Markdown
Member

@fe51 fe51 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot for this exploratory PR !

A preliminary review following our discussion and before we’ve looked at the test updates.

These aren’t critical comments at this stage.

Perhaps the endpoint should make it clear that the alerts retrieved are, by default, filtered based on a risk score?

Comment thread src/app/api/api_v1/endpoints/alerts.py Outdated
return mapping


async def _resolve_class_per_camera(
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

name shloud be more explicit, something like, fire_risk or fwi

Suggested change
async def _resolve_class_per_camera(
async def _resolve_FWI_class_per_camera(

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done :)

Comment thread src/app/api/api_v1/endpoints/alerts.py Outdated
Comment on lines +155 to +156
seq_match = seq_match.where(Sequence.last_seen_at > utcnow() - timedelta(hours=24))
seq_match = seq_match.where(Sequence.is_wildfire.is_(None)) # type: ignore[union-attr]
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not ? (same approach as before)
Might even be relavant to chain with previous lines from l152

Suggested change
seq_match = seq_match.where(Sequence.last_seen_at > utcnow() - timedelta(hours=24))
seq_match = seq_match.where(Sequence.is_wildfire.is_(None)) # type: ignore[union-attr]
seq_match = seq_match.where(Sequence.last_seen_at > utcnow() - timedelta(hours=24)).where(Sequence.is_wildfire.is_(None)) # type: ignore[union-attr]

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you are right better to chain , juste updated for readibily

Comment thread src/app/models.py Outdated
last_seen_at: datetime = Field(..., nullable=False)
# Highest detection confidence ever attached to this sequence.
# Monotonic: never recomputed downward when detections are deleted/reassigned.
max_conf: Union[float, None] = Field(None, nullable=True)
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am ok to add comment for field desc. However, would not it be the opportunity to put this comment as a postgres field desc

Suggested change
max_conf: Union[float, None] = Field(None, nullable=True)
max_conf: Union[float, None] = Field(None, nullable=True, description="Highest detection confidence ever attached to this sequence. Monotonic: never decremented when detections are deleted or reassigned.",
)

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

all good

Copy link
Copy Markdown
Member

@fe51 fe51 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the updates, I might I have missed something, but, some tests might be missing

/alerts/fromdate has zero tests —> it got risk_score added same as the other endpoints, but nothing covers it. It also uses the distinct target_date code path in _resolve_class_per_camera, which is the only branch that hits risk_service.get_scores_for_date() —> that async path is not exercised by any test in the file.

Alert with mixed sequences —> no test for an alert that has one sequence above threshold and one below. The alert would appear (it matched the seq_match subquery via the passing sequence) but with only the surviving sequence in its payload. Worth having a test to pin that behavior as intentional. What do you think ?

Comment on lines +94 to +103
@pytest.mark.asyncio
async def test_unlabeled_latest_keeps_all_when_class_is_moderate(
async_client: AsyncClient, detection_session: AsyncSession, reset_risk_cache
):
camera_id = pytest.camera_table[0]["id"]
pose_id = pytest.pose_table[0]["id"]
low_seq = await _seed_unlabeled_sequence(detection_session, camera_id, pose_id, max_conf=0.10, minutes_ago=30)

risk_service._scores = {camera_id: "moderate"}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I might be a bit paranoid about this filter, but I reckon it might be worth introducing tests for ‘moderate’ and above right now, so that if we update the rules for those risk levels later, they’ll already have been tested.

Below this suggestion (but not tested)

Suggested change
@pytest.mark.asyncio
async def test_unlabeled_latest_keeps_all_when_class_is_moderate(
async_client: AsyncClient, detection_session: AsyncSession, reset_risk_cache
):
camera_id = pytest.camera_table[0]["id"]
pose_id = pytest.pose_table[0]["id"]
low_seq = await _seed_unlabeled_sequence(detection_session, camera_id, pose_id, max_conf=0.10, minutes_ago=30)
risk_service._scores = {camera_id: "moderate"}
@pytest.mark.asyncio
@pytest.mark.parametrize("fwi_class", ["moderate", "high", "very_high", "extreme"])
async def test_unlabeled_latest_keeps_all_when_class_is_moderate_or_above(
fwi_class: str, async_client: AsyncClient, detection_session: AsyncSession, reset_risk_cache
):
camera_id = pytest.camera_table[0]["id"]
pose_id = pytest.pose_table[0]["id"]
low_seq = await _seed_unlabeled_sequence(detection_session, camera_id, pose_id, max_conf=0.10, minutes_ago=30)
risk_service._scores = {camera_id: fwi_class}

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

@MateoLostanlen MateoLostanlen requested a review from fe51 May 5, 2026 13:59
Copy link
Copy Markdown
Member

@fe51 fe51 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for adding new tests

@MateoLostanlen MateoLostanlen merged commit b4fa415 into main May 5, 2026
23 of 24 checks passed
@MateoLostanlen MateoLostanlen deleted the feat/risk-api-filter branch May 5, 2026 14:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants