Skip to content

v0.9.0 — runs locally from a fresh clone

Pre-release
Pre-release

Choose a tag to compare

View all tags
@CieriS CieriS released this 12 Oct 08:29
· 71 commits to main since this release
v0.9.0
10015f9
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

First tagged version. The whole pipeline runs locally from a fresh clone; the cloud target is implemented but has not run on a real Google Cloud project yet, which is what v1.0.0 waits for (milestone v1.0.0).

What it does

  • Ingestion (Rust, aq-ingest): ARPAE air quality measurements, station registries and Open-Meteo weather into partitioned Parquet, idempotent, with a 30-day window reprocessed on every run to pick up revised values.
  • Models (dbt on DuckDB, portable to BigQuery): staging, intermediate and five marts — exceedances of the legal limits, multi-year trends, traffic against background, PM10 against wind and rain, data completeness — with 107 data tests and a unit test.
  • Orchestration (Dagster): daily schedule, backfills by range, asset lineage, freshness and completeness checks.
  • Dashboard (Streamlit) on the marts.
  • CI: Rust, Python, dbt on fixtures, the BigQuery dialect on an emulator, the Docker images.

Try it

With Docker only:

git clone https://github.com/CieriS/aria-er && cd aria-er
make up    # Dagster on http://localhost:3000, dashboard on http://localhost:8501

About six minutes after the start the dashboard shows ten years of measurements. Checked on a clean clone on 2026-10-12.

Checked against ARPAE

  • PM10 exceedance days of the Bologna stations equal the ARPAE annual report for every year from 2016 to 2025.
  • For 2026, PM10 exceedance days equal the ARPAE daily bulletin of 8 October for all 64 stations it lists, and NO2 hours for all 65.

Changed since the last state of main without a version

  • Measurements come from the archive of the ARPAE REST service instead of the CKAN datastore, which stopped on 2026-09-17 (#33, ADR 0008). The whole of 2026 can now be ingested, with 22 more stations.
  • Exceedances are counted on values rounded to a whole number, as ARPAE does (#33).
  • PM10 and PM2.5 of the current year were attributed to the following day; fixed with the new source (#33).
  • The Docker images build again: they had moved to a Python version the projects do not support (#35).
  • The dashboard and the README credit ARPAE and Open-Meteo as their licences ask (#34).
  • The code is under the MIT licence; ADR 0007 (no streaming) and ADR 0008 are accepted (#39).

Known limits

  • The REST archive had no value newer than 2026-10-09 11:00 on 2026-10-12 (#40).
  • Ozone exceedance days of 2026 differ from the ARPAE bulletin by 1 to 3 days for a third of the stations (#44).
  • A first start loads the last 30 days of the current year, not all of it: make ingest FROM=2026-01-01 loads the rest.
  • Everything else: docs/limitations.md.
Assets 2
Loading