Repository navigation
Releases
Web Interface v1.1.0 — deployed state
The web interface as it runs on the server today (https://ilha3d.com/sapi/). This tag is new: it
was created when the project moved to this public repository, to record the deployed state, which
had moved on from webserver-v1.0.0 without a release.
Changes since v1.0.0
- Predictive-alert e-mail chart embedded inline (Content-ID) instead of a static URL that each
later alert overwrote, so an e-mail opened later showed a newer chart than the one it was sent
with (#190) - Kalman smoothing retired from the level charts and the filter cron;
level_delta_cmis the
only filtered series - Chart zoom by mouse wheel or drag now needs Ctrl/Shift, which stops accidental zooming
- Series colours: filtered Δ is purple and raw data is teal, so neither is confused with the
±σ band - MERGE precipitation stats no longer fail on an undefined array index when CPTEC data arrives
late (#205) - UTC tooltips in
predictions.php, matching the axis labels (#176) - AGPL-3.0-or-later headers on every source file (
#80) - The Hostinger account id is replaced by
<HOSTINGER_USER>everywhere
Deploying
PHP 7.4+ / MariaDB. Copy the *-sample.* files under private_configs/ and
cron/credenciais-sample.php to their real names, fill in your own database, mail, API and
Telegram values, and substitute <HOSTINGER_USER> with your own account path. For the database,
apply database/schema_20260313.sql and then the migration_*.sql files in issue order.
Asset
webserver-v1.1.0.tar.gz — a snapshot of webserver/ at this tag.
Licence: AGPL-3.0-or-later — see LICENSE and NOTICE.
Issue numbers such as #190 refer to the private development repository where this code was
developed. The source of this tag is in this repository at webserver/.
Web Interface v1.0.0
PHP/Bootstrap/Leaflet monitoring and management web application for SAPI, operational since 2026-03-15 (Issue #52) at https://ilha3d.com/sapi/. This tag/release closes the gap identified for this component in the project's Production Release Procedure (same gap previously found and fixed for MERGE2MySQL, Issue #162).
What's included in the attached webserver-v1.0.0.tar.gz
- Public dashboard, admin panel, alert rules/contacts management (
public_html/sapi/) - Alert cron engine (email + Telegram + WhatsApp via CallMeBot) and Kalman/delta level-filtering cron (
cron/) - M4-LGBM and M6-MLR prediction visualization (
predictions.php,api/predictions*.php) - MERGE/CPTEC precipitation overlay and accumulation stats (
view.php) - DB migration scripts and a schema snapshot (
migration_issue*.sql,migration_m6_mlr.sql,database/schema_20260313.sql) - Credential templates only (
*-sample.php,*-sample.json) — no real secrets. Real config filenames (sapi.php,db_config.php,mail_config.php,api_config.php,credenciais.php) are.gitignored and were confirmed absent from the working tree before packaging.
Stack
PHP 7.4+ / MariaDB, deployed on Hostinger shared hosting.
DB schema fidelity note
database/schema_20260313.sql is a schema-only snapshot from 2026-03-13. To reconstruct the current schema, apply it followed by the migration files in issue order (migration_issue52.sql → migration_issue57.sql → migration_issue108.sql → migration_issue110.sql → migration_m6_mlr.sql). This is not a single live re-dump.
Issues resolved
#52, #56, #57, #61, #66, #67, #69, #108, #110, #114, #120, #135
Known open limitations (not blockers)
#53— alert message format not yet aligned to IDAP standard#59— level-filter parameters (levelZero, maxDelta, Kalman Q/R, etc.) still hardcoded, not exposed in admin UI#63— alert triggers still readlevel_kalman_cm(confirmed ~25 min lag at flood peak); should switch tolevel_delta_cm
Tracked in Issue #167.
Public-repository changes: the real Hostinger account id is replaced by <HOSTINGER_USER> in every file (substitute your own when deploying), and the schema snapshot is renamed to database/schema_20260313.sql. The attached tarball is regenerated from this tag. The version that runs on the server today is webserver-v1.1.0.
Licence: AGPL-3.0-or-later — see LICENSE and NOTICE. This release predates the project's formal licence; as sole copyright holder, the author extends AGPL-3.0-or-later to it retroactively (2026-07-28).
Imported from the private development repository, where this release was first published. Issue numbers such as #31 refer to that repository. The source of this tag is in this repository at webserver/.
Station-03 v1.0.0 - First stable production deployment
Station-03 v1.0.0 - Production Firmware
Deployed: 2026-03-01 | Second bridge, Florianópolis Norte (~30 m from gateway)
Hardware: STM32 Nucleo L476RG + US-100 (serial mode) + LilyGo LoRa32
Calibration: levelZero = 236 cm (reads 199 cm at river level 37 cm)
What's included
- US-100 ultrasonic sensor in serial mode with surface temperature reading
- ADC monitoring for Vbat and Vpanel
- LoRa transmission using SensorData struct (MessagePack compatible)
- Deep sleep via STM32 STOP2 mode with RTC wake-up (~5 min interval)
Firmware binary
station-03-v1.0.0-nucleo_l476rg.bin — flash with STM32CubeProgrammer via SWD
(Note: ST-Link was removed from the Nucleo board to reduce power consumption — external SWD wiring required)
Issues resolved
- Closes
#31
Note
Source: firmware/station-03/
Source note: the source in this tag is the production copy promoted on 2026-03-01, the one that matches the attached binary (all 193 embedded strings identical; a rebuild gives 60,604 bytes against the released 60,572 because library versions resolve differently today).
Licence: AGPL-3.0-or-later — see LICENSE and NOTICE. This release predates the project's formal licence; as sole copyright holder, the author extends AGPL-3.0-or-later to it retroactively (2026-07-28).
Imported from the private development repository, where this release was first published. Issue numbers such as #31 refer to that repository. The source of this tag is in this repository at firmware/station-03/.
Station-02 v1.0.0 - Production Firmware
Station-02 v1.0.0 - Production Firmware
Deployed: January 2, 2026 | Lawn in front of the residence (same building as the gateway), Florianópolis Norte — precipitation-only station, not on a bridge
Hardware: STM32 Nucleo F103RB + SAPI Shield Morpho V1.2 + SAPI LoRa Arduino Shield V1.0
Sensors: BMP280 (temperature/pressure), tipping bucket rain gauge (0.25 mm/pulse), ADC (Vpanel, Vbat)
What's included
- Rain gauge pulse counting with deep sleep between measurements
- BMP280 temperature and pressure readings
- ADC monitoring for Vbat and Vpanel
- LoRa transmission using SensorData struct
- STM32 deep sleep with RTC wake-up
Firmware binary
station-02-v1.0.0-nucleo_f103rb.bin — flash via ST-Link with STM32CubeProgrammer or:
pio run -t upload
Note
Source: firmware/station-02/
RAM: 8.8% (1796 / 20480 bytes) | Flash: 35.2% (46176 / 131072 bytes)
Licence: AGPL-3.0-or-later — see LICENSE and NOTICE. This release predates the project's formal licence; as sole copyright holder, the author extends AGPL-3.0-or-later to it retroactively (2026-07-28).
Imported from the private development repository, where this release was first published. Issue numbers such as #31 refer to that repository. The source of this tag is in this repository at firmware/station-02/.
Station-01 v1.0.0 - Production Firmware
Station-01 v1.0.0 - Production Firmware
Deployed: 2026-04-07 | Bridge 01, Florianópolis Norte, Cachoeira do Bom Jesus
Hardware: STM32 BluePill F103C8T6 + HC-SR04 ultrasonic + RFM95W LoRa 915 MHz
Calibration: levelZero = 291.0 cm (calibrated 2025-12-30)
Firmware version string: SAPM_Sensor_BluePill_2026040701
What's included
- HC-SR04 ultrasonic sensor with median filter (11 readings) for outlier rejection
- LoRa transmission at 915 MHz, 20 dBm TX power
- STM32 internal RTC with deep sleep (~1 min sampling interval)
- HMAC-SHA256 authenticated MessagePack payload
- Watchdog timer support
Firmware binary
station-01-v1.0.0-bluepill_f103c8.bin — flash via FTDI serial adapter:
- Connections: RX→A9, TX→A10, GND→GND, 5V→5V
- Command:
pio run -t upload --upload-port /dev/ttyUSB0(Linux) or setupload_port = COM10inplatformio.ini(Windows)
Issues resolved
- Closes
#82— Station-01 firmware migrated to main SAPI repository - Fixes alexandreberg/SAPM_Sensor_BluePill#3 — median filter result was not assigned back to
readingDistancebefore LoRa TX; all previous versions transmitted the last raw reading instead of the filtered median
Source
firmware/station-01/
Licence: AGPL-3.0-or-later — see LICENSE and NOTICE. This release predates the project's formal licence; as sole copyright holder, the author extends AGPL-3.0-or-later to it retroactively (2026-07-28).
Imported from the private development repository, where this release was first published. Issue numbers such as #31 refer to that repository. The source of this tag is in this repository at firmware/station-01/.
SAPI Shield Morpho V1.2 — Rain Gauge / Environmental Sensor Interface Shield
KiCad 9 hardware design for the SAPI Shield Morpho V1.2, the ST Morpho-format
interface board used by SAPI monitoring stations. Adds tipping-bucket rain
gauge pulse counting, serial/trigger-echo ultrasonic sensor interfaces
(US-100, HC-SR04, AJ-SR04), I2C environmental sensor bus (SHT-20, BMP280),
ADC solar/battery voltage monitoring, ST-Link connector, extra 3.3V/5V
switched rails, and 5 spare GPIOs.
Issue resolved: #151 (KiCad 9 PCB designs added to repo)
Design status: fabrication-ready layout (gerbers included under
gerber_to_order/). Issue #159 (adding onboard MB85RC256V FRAM) is an open,
separate future revision — not included in this release.
Attached: .tar.gz snapshot of hardware/morpho-shield-v1.2/
(schematic, PCB layout, project libraries, gerbers, datasheets).
Referenced in the dissertation's Produto Técnico Tecnológico (PTT) chapter.
Asset: the tarball is regenerated from this tag with the 3D models as real files, so it can be used without Git LFS.
Licence: CERN-OHL-S-2.0 — see LICENSES/CERN-OHL-S-2.0.txt and NOTICE. Third-party 3D models, symbols, footprints and datasheets keep their original terms. This release predates the project's formal licence; as sole copyright holder, the author extends CERN-OHL-S-2.0 to the SAPI design files retroactively (2026-07-28).
Imported from the private development repository, where this release was first published. Issue numbers such as #31 refer to that repository. The source of this tag is in this repository at hardware/morpho-shield-v1.2/.
SAPI LoRa Arduino Shield V1.0 — LoRa Radio Communication Shield
KiCad 9 hardware design for the SAPI LoRa Arduino Shield V1.0, an
Arduino-footprint radio shield supporting RFM69HC and LILYGO LoRa32
modules. The project's SAPI implementation runs at 915 MHz, but the shield
layout accepts radio modules of any frequency without layout changes.
Includes a debug LED (disable-able for low power), a reset button, and
reconfigurable GPIOs for radio and LED control.
Issue resolved: #151 (KiCad 9 PCB designs added to repo)
Design status: fabrication-ready layout (gerbers included under
gerber_to_order/). Issue #159 (adding onboard MB85RC256V FRAM) is an open,
separate future revision — not included in this release.
Attached: .tar.gz snapshot of hardware/lora-arduino-shield-v1.0/
(schematic, PCB layout, project libraries, gerbers, BOM).
Referenced in the dissertation's Produto Técnico Tecnológico (PTT) chapter.
Asset: the tarball is regenerated from this tag with the 3D models as real files, so it can be used without Git LFS.
Licence: CERN-OHL-S-2.0 — see LICENSES/CERN-OHL-S-2.0.txt and NOTICE. Third-party 3D models, symbols, footprints and datasheets keep their original terms. This release predates the project's formal licence; as sole copyright holder, the author extends CERN-OHL-S-2.0 to the SAPI design files retroactively (2026-07-28).
Imported from the private development repository, where this release was first published. Issue numbers such as #31 refer to that repository. The source of this tag is in this repository at hardware/lora-arduino-shield-v1.0/.
Model6-MLR v2.0.0 — retrained on the full record, thresholds 60/75/90
Production retrain of the M6 multiple linear regression model. Deployed 2026-09-12
13:08 UTC to /home/ilha3d/SAPI/MLR_Production/models/model6_mlr.pkl, replacing the model
trained on 2026-05-16.
Training
| Trained | 2026-09-12 12:52 UTC |
| Trained by | train_production_from_backtest.py --model m6 --train-end 2026-09-10 |
| Training set | 177,154 steps of 5 min, 10 contiguous segments, 2024-12-30 → 2026-09-09 |
| Mode | TRAIN_MODE = "continuous" — the whole record, floods and dry weather |
| Hyper-parameters | lags=[-24,-18,-12,-9,-6,-3,-1], lags_past_covariates=[-18,-12,-6,-1], output_chunk_length=24, fit_intercept=True, random_state=42 |
| Libraries | darts 0.43.0, scikit-learn 1.8.0, pandas 2.3.3, numpy 2.4.3, Python 3.13.5 |
| Thresholds | Atenção 60 / Alerta 75 / Inundação 90 cm, both stations |
Trained by the same code that produced the published backtest results —
train_production_from_backtest.py imports config and real_utils from
results_final_m6_mlr/ rather than re-implementing the pipeline. The previous production
trainer was a separate implementation that had drifted and is now deprecated.
TRAIN_END is pushed past the end of the record at runtime, so the deployed model is fitted
on every available sample. The chronological hold-out at 2026-05-01 exists to evaluate;
the published out-of-sample metrics therefore describe a model trained on less data than
this one, and are a conservative floor for it rather than a description of these weights.
Issues resolved
#223— continuous training instead of flood-window-only slicing#225— thresholds recalibrated 50/60/75 → 60/75/90 after the ~18 cm baseline drift#227— the 9 sensor-artifact windows masked out of the level series#124— Station-02 rain spread over 3 five-minute steps, in training and inference- Also fixed: the M6 MERGE fallback pointed at
merge_20260404.csv, a frozen dump ending
2026-04-04 in a format that reader cannot parse. It now reads the rolling
merge_precip.csvit was always meant to use.
Validation — the 2026-09-11 flood
The training data ends 2026-09-09, so this event is genuinely unseen. Observed peak 100 cm
at 17:20. Replayed 5 minutes at a time with replay_flood_event.py --model m6:
| previous model (50/60/75) | this model (60/75/90) | |
|---|---|---|
| max forecast +60 min | 130.7 cm | 100.5 cm |
| max forecast +120 min | 107.4 cm | 100.3 cm |
| Inundação fired | 15:05, river at 61 cm | 16:30, 5 min after the 90 cm crossing |
The previous model forecast up to 130.7 cm for a flood that reached 100 cm — the
flood-window training bias made the linear model extrapolate far past anything observed —
and declared Inundação with the river at 61 cm. This model peaks at 100.5 cm, essentially
on the observed peak, and fires 5 to 10 minutes after each crossing: the behaviour the
dissertation reports out of sample, now reproduced on an event it had never seen.
Assets
model6_mlr.pkl— the deployed modelmodel6_mlr.meta.json— provenance: SHA-256 of every input CSV, library versions,
hyper-parameters, effectiveTRAIN_END, git commitmodel6-mlr-v2.0.0-training-code.tar.gz— the full chain that produces and serves it:
the training script, the backtestconfig.py/real_utils.py, the catalogue builder,
the 3-pass level filter, the production inference and alert code, and the parity testpredictive-models-v2.0.0-training-data.tar.gz— the input CSVs, raw and processed,
with aLEIAME_DADOS.mdexplaining each file and how to reproduce (same snapshot as the
Model4-LGBM v2.0.0 release; the two models share one data pipeline)
Data snapshot fidelity
Captured on the training day itself, from the same paths the script read; the per-file
SHA-256 hashes in model6_mlr.meta.json let you verify it before retraining.
/home/ilha3d/SAPI/data/merge_precip.csv is deliberately not in the snapshot and must
never be used for training: it is a rolling file the nightly refresh_merge_csv.py cron
rewrites with about six days of rain, and it exists only as an inference fallback. The
frozen MERGE history is merge_20260404.csv.
Rollback
The previous model is kept on the host as model6_mlr_v1_20260516.pkl. Restoring it is a
rename plus reverting THRESHOLDS in config_production_m6.py; the cron picks it up within
one 5-minute cycle and there is no service to restart.
Provenance note: git_commit in the .meta.json (0f2b1d9) refers to the private development repository. The same code is in this tag under predictive-models/; the training-code tarball is attached unchanged.
Licence: AGPL-3.0-or-later — see LICENSE and NOTICE.
Imported from the private development repository, where this release was first published. Issue numbers such as #31 refer to that repository. The source of this tag is in this repository at predictive-models/.
Model4-LGBM v2.0.0 — retrained on the full record, thresholds 60/75/90
Production retrain of the M4 LightGBM model. Deployed 2026-09-12 13:08 UTC to
/home/ilha3d/SAPI/LightGBM_Production/models/model4_lgbm.pkl, replacing the model trained
on 2026-05-10.
Training
| Trained | 2026-09-12 12:58 UTC |
| Trained by | train_production_from_backtest.py --model m4 --train-end 2026-09-10 |
| Training set | 177,154 steps of 5 min, 10 contiguous segments, 2024-12-30 → 2026-09-09 |
| Mode | TRAIN_MODE = "continuous" — the whole record, floods and dry weather |
| Hyper-parameters | lags=[-24,-18,-12,-9,-6,-3,-1], lags_past_covariates=[-18,-12,-6,-1], output_chunk_length=24, n_estimators=500, num_leaves=31, random_state=42 |
| Libraries | darts 0.43.0, lightgbm 4.6.0, scikit-learn 1.8.0, pandas 2.3.3, numpy 2.4.3, Python 3.13.5 |
| Thresholds | Atenção 60 / Alerta 75 / Inundação 90 cm, both stations |
The model is trained by the same code that produced the published backtest results —
train_production_from_backtest.py imports config and real_utils from
results_final_m4_lgbm/ rather than re-implementing the pipeline. The previous production
trainer was a separate implementation that had drifted and is now deprecated.
TRAIN_END is pushed past the end of the record at runtime, so the deployed model is fitted
on every available sample. The chronological hold-out at 2026-05-01 exists to evaluate;
the published out-of-sample metrics therefore describe a model trained on less data than
this one, and are a conservative floor for it rather than a description of these weights.
Issues resolved
#223— continuous training instead of flood-window-only slicing#225— thresholds recalibrated 50/60/75 → 60/75/90 after the ~18 cm baseline drift#227— the 9 sensor-artifact windows masked out of the level series#124— Station-02 rain spread over 3 five-minute steps, in training and inference
Validation — the 2026-09-11 flood
The training data ends 2026-09-09, so this event is genuinely unseen. Observed peak 100 cm
at 17:20. Replayed 5 minutes at a time with replay_flood_event.py --model m4:
| previous model (50/60/75) | this model (60/75/90) | |
|---|---|---|
| max forecast +60 min | 90.7 cm | 96.4 cm |
| max forecast +120 min | 78.0 cm | 92.1 cm |
| Inundação fired | 15:15, river at 64 cm | 16:35, 10 min after the 90 cm crossing |
The previous model could not express a long-horizon flood at all, topping out at 78.0 cm at
+120 min, and declared Inundação with the river at 64 cm because its Inundação threshold was
75 cm. Note that this does not contradict the dissertation's structural-saturation finding,
which describes the model trained to 2026-05-01 with 43 steps above 90 cm: the sample
changed here, not the model type.
Assets
model4_lgbm.pkl— the deployed modelmodel4_lgbm.meta.json— provenance: SHA-256 of every input CSV, library versions,
hyper-parameters, effectiveTRAIN_END, git commitmodel4-lgbm-v2.0.0-training-code.tar.gz— the full chain that produces and serves it:
the training script, the backtestconfig.py/real_utils.py, the catalogue builder,
the 3-pass level filter, the production inference and alert code, and both test scriptspredictive-models-v2.0.0-training-data.tar.gz— the input CSVs, raw and processed,
with aLEIAME_DADOS.mdexplaining each file and how to reproduce
Data snapshot fidelity
The snapshot was captured on the training day itself, from the same paths the script
read, and the per-file SHA-256 hashes in model4_lgbm.meta.json let you verify it before
retraining.
One caveat worth stating explicitly: /home/ilha3d/SAPI/data/merge_precip.csv is not
in the snapshot and must never be used for training. It is a rolling file that the nightly
refresh_merge_csv.py cron rewrites with about six days of rain, and it exists only as an
inference fallback. The frozen MERGE history is merge_20260404.csv, which is what the
snapshot and the backtest config both use.
Rollback
The previous model is kept on the host as model4_lgbm_v1_20260510.pkl. Restoring it is a
rename plus reverting THRESHOLDS in config_production.py; the cron picks it up within
one 5-minute cycle and there is no service to restart.
Provenance note: git_commit in the .meta.json (0f2b1d9) refers to the private development repository. The same code is in this tag under predictive-models/; the training-code tarball is attached unchanged.
Licence: AGPL-3.0-or-later — see LICENSE and NOTICE.
Imported from the private development repository, where this release was first published. Issue numbers such as #31 refer to that repository. The source of this tag is in this repository at predictive-models/.
MERGE2MySQL v1.0.0 — MERGE/CPTEC data acquisition pipeline
MERGE/CPTEC Data Acquisition Pipeline
FTP sync → GRIB2/CDO processing → MySQL import pipeline for MERGE/CPTEC
precipitation data (ftp.cptec.inpe.br/modelos/tempo/MERGE/GPM/HOURLY/).
Components:
merge_crontab_wrapper.sh— hourly crontab wrapper0-sincroniza_repo_merge_crontab.sh— FTP sync vialftp2-processa_dia_completo.sh+2-processa_dia_completo_crontab.sh— GRIB2→NetCDF conversion (CDO) + ROI extraction + orchestration3-importar_dados_crontab.sh— incremental MySQL import3-importar_dados_lote.sh— batch import (single day / full month / full year), used for historical reconstruction and gap-fillingpython/importar_csv_mysql.py— upsert logic (UPDATEvalor_3if a row for the timestamp exists, otherwise INSERT)python/importar_csv_mysql_incremental.py—INSERT IGNOREvariantpython/config_mysql_SAMPLE.py— connection config template (real credentials are gitignored and NOT distributed)
Deployment / validation date: Cross-platform compatibility (x86_64 Ubuntu + Raspberry Pi 4 ARM64) validated 2026-02-03 in PR #9 — full pipeline tested end-to-end (sync → process → import) on both architectures. CDO 2.4.x uses variable name prec, CDO 2.5.x (ARM64) uses rdp; script auto-detects which to use.
Issues resolved: #162 (this pipeline was operational but had never gone through the Production Release Procedure — no tag/release existed until now)
Fidelity note: This is a source-code release. No trained model or binary is attached — the tarball is a snapshot of the scripts as of this tag for replication on any Linux server (only the target lat/lon needs to be reparametrized to point at a different watershed covered by MERGE/CPTEC).
Asset: merge2mysql-v1.0.0.tar.gz is regenerated from this repository's tag (same scripts, plus the licence headers).
Licence: AGPL-3.0-or-later — see LICENSE and NOTICE. This release predates the project's formal licence; as sole copyright holder, the author extends AGPL-3.0-or-later to it retroactively (2026-07-28).
Imported from the private development repository, where this release was first published. Issue numbers such as #31 refer to that repository. The source of this tag is in this repository at pipelines/merge2mysql/.