OpenPVMapper — source code

This repository is code for replication only. The OpenPVMapper datasets themselves (the data you'd actually want to use) are on Zenodo at https://zenodo.org/records/21534856 — go there if you just want the data. This repo is for regenerating that data from scratch, or auditing how it was built.

Code that reproduces:

1. latest-dpvm / latest_patched_*.geojson — the single-vintage, DPVM-only export used in the Joule paper (also released on Zenodo as latest-dpvm.geojson.zip).
2. The enriched, multi-source national database (enriched-national.geojson / dpvm_enriched.geojson) — DeepPVMapper + OpenStreetMap + FRPV + manual corrections, resolved and characterized.
3. The precision/recall validation reported in the paper (Technical Validation section) — FRPV threshold calibration and the two-axis stratified precision sample.

This repository is code-only. A companion archive, resources.zip, holds everything this code needs that isn't code (model weights, reference tables, manual-annotation ground truth, a few project-specific shapefiles), deposited on Zenodo:

Download: https://doi.org/10.5281/zenodo.21611312

Unzip it as a sibling folder of this repository, named resources/:

your-projects-folder/
├── source-openpvmapper/      <- this repo
│   ├── 01_raw_to_latest/
│   ├── 02_patches/
│   ├── 03_multi_source_aggregation/
│   ├── 04_gpu_segmentation/
│   └── 05_validation/
└── resources/                 <- unzip resources.zip here

Every hardcoded path in this codebase that points at resources/ assumes exactly this layout (see "Paths into resources/" below). If you place resources/ somewhere else, override the relevant path via each script's own CLI flag — none of these paths are truly hardcoded, just defaulted.

Pipeline overview

raw/D0xx_YYYY/arrays_characteristics_D0xx_YYYY.geojson   (input -- one per department x vintage)
        │
        ├──────────────────────────────────────────────────────┐
        │                                                       │
        ▼                                                       ▼
01_raw_to_latest/                                    03_multi_source_aggregation/
  build_last_vintage.py                                build_full_release.py (orchestrator)
  + 02_patches/patch.py (ad-hoc tile reruns,              -> sources/dpvm_union.py
    merged in by build_last_vintage_patched.py)           -> sources/frpv.py, osm.py, corrections.py
        │                                                 -> matching/overlap.py
        ▼                                                 -> geometry_resolution.py
postprocessed/latest_patched_{date}.geojson                -> finalize/flag_false_positives.py
  = "latest-dpvm" export                                    -> finalize/recover_false_negatives.py
                                                              -> enrichment/characterize_from_polygon.py
                                                              -> enrichment/characterize_from_point.py
                                                                 (+ 04_gpu_segmentation/ for point-only
                                                                 candidates, run separately on a GPU pod)
                                                              -> finalize/finalize_national.py
                                                              -> finalize/drop_commune_name.py
                                                                       │
                                                                       ▼
                                                     enriched-national.geojson
                                                       = "enriched" export
                                                                       │
                                                                       ▼
                                                     05_validation/
                                                       build_precision_sample.py --analyze
                                                       -> precision/recall numbers reported in
                                                          the paper's Technical Validation section

Folder structure

• 01_raw_to_latest/ — build_last_vintage.py (assembles the most recent DeepPVMapper vintage per department from raw/, fetches communes and RNB building IDs) and build_last_vintage_patched.py (same pipeline, merging in the ad-hoc tile reruns from 02_patches/ before running). Produces latest_patched_{date}.geojson.
• 02_patches/ — code for re-processing individual tiles that were missing or wrong in an original department run (patch.py + its standalone copies of the characterization/aggregation helpers), plus README.md documenting the tilt-LUT NaN bug this patch mechanism exists to fix. Reads its data (dalles, communes, patched output) from resources/; topo/ (BD TOPO) is not bundled, deposit it yourself under resources/patches_topo/ (see below).
• 03_multi_source_aggregation/ — the full multi-source pipeline (build_full_release.py orchestrates sources/, matching/, geometry_resolution.py, enrichment/, and finalize/ in sequence). config.yml is included as a reference/example — update the paths for your own environment before running.
• 04_gpu_segmentation/ — self-contained GPU segmentation step for point-only candidates (FRPV promotions, confirmed false negatives) that have no polygon yet; run separately on a GPU pod, see its own README.
• 05_validation/ — the code behind the paper's Technical Validation numbers: frpv_threshold_calibration.py (FRPV promotion threshold, precision-recall curve with bootstrap CIs) and build_precision_sample.py (the two independent stratified precision samples -- by source combination and by power class -- and the population-weighted global precision estimate, --analyze). export_anomalies.py and verify_regional_power.py are smaller sanity checks kept alongside for completeness. These are genuinely part of reproducing the paper, not just the dataset: the Technical Validation section reports numbers that only this code (plus the annotated ground truth in resources/validation_data/) can regenerate.

What's in resources.zip

Path	Used by	Size	What it is
model_weights/model_bdappv_cls.pth	DeepPVMapper classification stage (Inception-v3)	~93 MB	Trained classification weights, BDAPPV project.
model_weights/model_bdappv_seg.pth	DeepPVMapper segmentation stage (DeepLab-v3), and 04_gpu_segmentation/	~233 MB	Trained segmentation weights, BDAPPV project.
pypvroof_data/bdappv-metadata.csv	enrichment/_pypvroof_bootstrap.py, 02_patches/scripts/pypvroof_patch.py	~430 KB	Calibration table pypvroof needs at runtime; not always shipped by the pip package install.
patches_dalles/D{dept}_{year}/ (5 depts: 029_2024, 050_2025, 059_2025, 062_2024, 066_2024)	02_patches/patch.py	~5.4 MB	Re-flown tile shapefiles (dalles.shp + mosaic .vrt) for each patched department -- project-specific reruns, not a public dataset.
patches_data/	Reference output of patch.py for the 5 patched departments	~23 MB	Kept as ground truth to check a rerun of patch.py reproduces the same output.
patches_communes/COMMUNES_2021/	02_patches/patch.py, 01_raw_to_latest/*, 03_multi_source_aggregation/{sources,finalize}/*	~314 MB	National communes reference, 2021 vintage. Re-downloadable from IGN ADMIN EXPRESS (see table below), but the exact vintage matters for reproducibility, so the file actually used is bundled here.
validation_data/precision_sample_annote.geojson	05_validation/build_precision_sample.py --analyze	~2.5 MB	The 1,862-installation manually annotated precision sample (two independent stratified draws) -- the ground truth behind the paper's ~74-75% precision figure.
validation_data/precision_sample_strata.csv	same	~64 KB	Stratum/population crosswalk for the sample above (needed for the population-weighted estimator).
validation_data/frpv_threshold/*	05_validation/frpv_threshold_calibration.py	~1.8 MB	The ~1,200-point manually annotated sample and precision-recall curve used to calibrate the FRPV promotion threshold (0.5).

patches_topo/ is expected but not provided even in resources.zip: BD TOPO (BATIMENT.shp) is too large and department-specific to bundle; deposit it yourself at resources/patches_topo/D{dept}_2026/**/BATIMENT.shp before running 02_patches/patch.py.

No other non-fetchable files were identified (the tilt lookup table referenced in 02_patches/README.md, lut_france.json, ships inside the pypvroof package itself, not in this repository).

Paths into resources/

The following defaults assume resources/ is a sibling folder of this repository (see layout above). All are overridable via their own CLI flag:

• 02_patches/patch.py (file lives in 02_patches/, i.e. one level below the repo root): DATA_DIR/DALLES_DIR/COMMUNE_DIR point at ../../resources/patches_data|patches_dalles|patches_communes; TOPO_DIR points at ../../resources/patches_topo (still user-provided) -- two levels up (out of 02_patches/, out of the repo root) to reach the sibling resources/.
• 03_multi_source_aggregation/sources/dpvm_union.py, sources/osm.py, 03_multi_source_aggregation/finalize/finalize_national.py (all three levels below the repo root): --communes / DEFAULT_COMMUNES_PATH point at ../../../resources/patches_communes/COMMUNES_2021/... -- three levels up.
• 03_multi_source_aggregation/enrichment/_pypvroof_bootstrap.py (also three levels below the repo root): the bundled pypvroof CSV resolves from ../../../resources/pypvroof_data/; the LUT patch import resolves from ../../02_patches/scripts/ (code, two levels up then back down into 02_patches/, stays inside this repo).
• 03_multi_source_aggregation/enrichment/characterize_from_point.py (same depth): --model-dir points at ../../../resources/model_weights.

--raw (input, e.g. dpvm_union.py) and --in/--out defaults in 05_validation/*.py (pointing at a live output/ directory) are untouched by any of this: they refer to the live pipeline's working data, not to anything in resources.zip, and are always meant to be overridden per-run.

External sources needed to run this code

None of these are included in this repository or in resources.zip — they must be fetched independently before running the corresponding stage. Where the exact vintage matters for reproducing the paper's numbers exactly (not just "a" database, but the one described), it's pinned below.

Needed for	Source	Where to get it	Version / vintage actually used
raw/ itself (DeepPVMapper detections, upstream of this repo)	IGN BD ORTHO aerial imagery (20 cm/pixel)	https://geoservices.ign.fr/bdortho	Varies per department/vintage — see raw/index.json in the data release for the exact capture year used per department.
01_raw_to_latest/, finalize/drop_commune_name.py (communes)	IGN ADMIN EXPRESS-COG-CARTO.LATEST (WFS, fetched automatically by the scripts)	https://geoservices.ign.fr/adminexpress	Not pinned -- .LATEST always resolves to whatever is current when the script runs. build_last_vintage.py snapshots its own fetch into postprocessed/reference/ with the run date in the filename specifically to compensate for this (see its docstring); the exact date used for the paper's release should be read off that snapshot filename, not assumed.
01_raw_to_latest/ (rnb_id field)	RNB (Référentiel National des Bâtiments), bulk export	https://rnb-fr.beta.gouv.fr/ (exports also on data.gouv.fr)	Not pinned, same caveat as ADMIN EXPRESS above -- snapshotted alongside it in postprocessed/reference/.
02_patches/ (resources/patches_topo/D{dept}_2026/BATIMENT.shp)	IGN BD TOPO (bâtiments), per department	https://geoservices.ign.fr/bdtopo	2026 vintage (BD TOPO isn't versioned by imagery capture year, always latest at patch time -- see 02_patches/README.md).
sources/osm.py	OpenStreetMap France extract (.osm.pbf)	https://download.geofabrik.de/europe/france.html	france-260716.osm.pbf -- the Geofabrik dated France extract from 2026-07-16 (filename encodes YYMMDD). Re-running against a different/later extract will not reproduce the exact OSM-sourced counts.
sources/frpv.py	FRPV dataset (per-building PV probability)	NEROT, Boris; THEBAULT, Martin, 2024, "FRPV - Presence of Rooftop Photovoltaic (RPV) systems on French buildings.", https://doi.org/10.57745/BXXYW4, Recherche Data Gouv	V3 specifically -- earlier versions cover fewer departments (this release uses 95/96, cf V3 changelog on Recherche Data Gouv).
enrichment/* (characterization)	pypvroof Python package	pip install pypvroof; method described in Trémenbert et al., arXiv:2309.07143 (https://arxiv.org/abs/2309.07143)	Not pinned to a specific PyPI release here -- pin one yourself (pip freeze after a working install) if you need bit-exact reproduction; the characterization method itself hasn't changed since the arXiv paper.
raw/ generation (upstream, not part of this repo)	DeepPVMapper detection/classification/segmentation pipeline	https://github.com/gabrielkasmi/deeppvmapper/	Not pinned to a commit here -- see that repo's own releases/tags.

Model weights, the pypvroof calibration table, the patched-department shapefiles/data, the communes reference, and the validation ground truth are not public — see resources.zip above instead.

Not included

• scripts/reorganize.py: builds raw/ from the original per-tile DeepPVMapper output — upstream of raw/, which is treated as a given input here.
• patches/aux|tmp/ (generated caches, rebuilt automatically on first run): not included.
• Any other data file: raw/, intermediate/, output/, postprocessed/ (live pipeline working directories, not part of a fixed release).