Skip to content

Implement rain rate processing and website view#53

Open
cchwala wants to merge 1 commit into
mainfrom
feature/rain-rates-subsite
Open

Implement rain rate processing and website view#53
cchwala wants to merge 1 commit into
mainfrom
feature/rain-rates-subsite

Conversation

@cchwala

@cchwala cchwala commented Jul 13, 2026

Copy link
Copy Markdown
Member

Feature overview:

  • New /rain-rates page showing CML map with rain-rate-based coloring
  • Interactive time selector: Current, Last 1h, -2h to -1h, -3h to -2h, -4h to -3h, Last 1d (sum), -2d to -1d (sum)
  • Embedded Grafana dashboard for rain rate time series per CML
  • Multi-user support with Row-Level Security isolation

Database changes:

  • Migration 010: Add cml_rain_data table for processed rain rates
  • Migration 011: Create cml_rain_stats aggregated view
  • Migration 012: Add cml_rain_data_1h continuous aggregate with compression
  • RLS policies applied to all rain data tables/views

Processor service (new):

  • Modular workflow architecture with registry pattern
  • pycomlink-based processing: TL->wet/dry->baseline->WAA->rain rate
  • Configurable per-user settings (poll interval, data window)
  • State management for tracking last processed time
  • Backfill script for historical data processing in batches

Webserver enhancements:

  • /api/rain-stats endpoint with offset parameter for time windows
  • /api/cml-map endpoint for CML metadata
  • /rain-rates route with Leaflet.js map visualization
  • Resizable Grafana iframe panel

Grafana integration:

  • Provisioned dashboard (UID: cml-rain-rates)
  • Time series panel querying cml_rain_data_secure view
  • Auto-refresh every 10s, default range 6h

Docker Compose updates:

  • Added processor service with persistent state volume
  • Archive generator/loader services for historical data
  • MNO simulators for demo_openmrg and demo_orange_cameroun

Documentation:

  • RAIN_PROCESSING_IMPLEMENTATION.md with architecture details
  • Jupyter notebook for validation and debugging

Feature overview:
- New /rain-rates page showing CML map with rain-rate-based coloring
- Interactive time selector: Current, Last 1h, -2h to -1h, -3h to -2h, -4h to -3h, Last 1d (sum), -2d to -1d (sum)
- Embedded Grafana dashboard for rain rate time series per CML
- Multi-user support with Row-Level Security isolation

Database changes:
- Migration 010: Add cml_rain_data table for processed rain rates
- Migration 011: Create cml_rain_stats aggregated view
- Migration 012: Add cml_rain_data_1h continuous aggregate with compression
- RLS policies applied to all rain data tables/views

Processor service (new):
- Modular workflow architecture with registry pattern
- pycomlink-based processing: TL->wet/dry->baseline->WAA->rain rate
- Configurable per-user settings (poll interval, data window)
- State management for tracking last processed time
- Backfill script for historical data processing in batches

Webserver enhancements:
- /api/rain-stats endpoint with offset parameter for time windows
- /api/cml-map endpoint for CML metadata
- /rain-rates route with Leaflet.js map visualization
- Resizable Grafana iframe panel

Grafana integration:
- Provisioned dashboard (UID: cml-rain-rates)
- Time series panel querying cml_rain_data_secure view
- Auto-refresh every 10s, default range 6h

Docker Compose updates:
- Added processor service with persistent state volume
- Archive generator/loader services for historical data
- MNO simulators for demo_openmrg and demo_orange_cameroun

Documentation:
- RAIN_PROCESSING_IMPLEMENTATION.md with architecture details
- Jupyter notebook for validation and debugging
@cchwala cchwala changed the title Implement Rain Rates sub-site with historical time selector Implement rain rate processing and website view Jul 13, 2026
@codecov

codecov Bot commented Jul 13, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 13.04348% with 40 lines in your changes missing coverage. Please review.
✅ Project coverage is 85.66%. Comparing base (6835796) to head (c631458).

Files with missing lines Patch % Lines
webserver/main.py 13.04% 40 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main      #53      +/-   ##
==========================================
+ Coverage   85.42%   85.66%   +0.24%     
==========================================
  Files          28       24       -4     
  Lines        3012     2693     -319     
==========================================
- Hits         2573     2307     -266     
+ Misses        439      386      -53     
Flag Coverage Δ
mno_simulator 86.12% <ø> (ø)
parser 91.99% <ø> (ø)
scripts ?
webserver 68.96% <13.04%> (-4.82%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@cchwala cchwala mentioned this pull request Jul 13, 2026
15 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant