Getting started

Getting started

The repository README is the canonical setup reference. This wiki page is maintained manually.

Requirements

• Python 3.11 or newer; development uses Python 3.12
• Git
• uv

Install and check the package

git clone https://github.com/valeford-de/spacement-pvextent.git
cd spacement-pvextent
uv sync --locked --extra import --extra geo --extra train --extra baseline
uv run --locked --extra import --extra geo --extra train --extra baseline pvextent --help
uv run --locked --extra import --extra geo --extra train --extra baseline python -m unittest discover -s tests -v

uv manages the local environment; no separate activation command is needed. The lockfile records the project's dependency versions. These four extras enable import, geographic preparation, smoke training and pretrained baseline training. Encoder weights are prepared separately with an explicit download flag.

Validate a dataset

Prepare a dataset using the dataset format and annotation guide, then run:

uv run --locked pvextent data validate /path/to/dataset --json

The validator checks metadata, file hashes, image/mask alignment, mask classes, and declared split boundaries. A passing report does not certify annotation truth, usage rights, or geographic independence.

Try the synthetic format example

uv run --locked python examples/create_demo_dataset.py data/demo
uv run --locked pvextent data validate data/demo --json

This generates synthetic pixels without downloading imagery. Its labels are unreviewed, and its capture dates and georeferences are unknown, so a valid report includes warnings. It is a structural example, not a training dataset.

Import the first public segmentation dataset

uv sync --locked --extra import
uv run --locked --extra import pvextent data import-pv4ger-seg \
  data/raw/pv4ger-seg-v0.9.1 data/pv4ger-seg-v0.9.1 --download
uv run --locked --extra import python -m unittest discover -s tests -v

See PV4GER-SEG import for pinned source files, offline use, licenses, and unassigned geographic groups. The import audit records 4,027 valid image/mask pairs. Their labels have not been locally reviewed.

Geographic splits and label review

uv sync --locked --extra import --extra geo
uv run --locked --extra geo pvextent data split \
  data/pv4ger-seg-v0.9.1 data/pv4ger-seg-geographic-v1 --json
uv run --locked pvextent data review \
  data/pv4ger-seg-v0.9.1 data/review-pv4ger-seg-v0.9.1 --json

Open the review directory's index.html in a browser. Review evidence downloads as JSON; source labels and status are not automatically changed. See geographic splits, label review, and unreviewed NRW candidates. All outputs require a new path; open existing review bundles directly instead of re-exporting over them.

Run the full test suite with uv run --locked --extra import --extra geo --extra train --extra baseline python -m unittest discover -s tests -v.

Experimental negative curation

The negative curation guide documents the dedicated JSON format for two independent AI visual reviews, explicitly authorized for experimental training. Only the intersection of approved background polygons becomes label 0; other pixels stay ignore 255. Gallery decision downloads use a separate format and are not curation inputs. Original source snapshots remain unchanged, and AI provenance is retained without claiming human review.

The executed collection contains 11 partial negative masks with 217,276 background pixels and four unresolved, all-ignore masks. Its new training snapshot has 3,293 training, 278 validation, 421 test and 42 unassigned records. All validation/test declarations and inherited rasters are preserved; the augmenter checked a minimum 1 km centre buffer without opening holdout pixels. The curation audit records the exact evidence and a separate negative-only training smoke check. A fresh candidate baseline has since trained on this snapshot; the original reference artifacts and scores remain unchanged. See the comparison guide.

After preparing the curated collection, create a new training snapshot with:

uv run --locked --extra train --extra geo pvextent data augment-training \
  data/pv4ger-seg-geographic-v1 data/pvextent-negative-curation-v1 \
  data/training-augmented-new \
  --dataset-version training-augmented-new --buffer-m 1000 --json

The local datasets and review evidence are not included in Git. Follow the curation guide to prepare inputs and keep a new output path. Inherited pixel files use hard links, so all input and output snapshots must remain immutable. This prepares data for a separate training experiment; it does not establish a new model-quality result or an independent benchmark.

Training smoke test

uv sync --locked --extra train
uv run --locked --extra train pvextent train smoke \
  data/pv4ger-seg-geographic-v1 runs/smoke-001 \
  --device auto --steps 8 --samples 8 --batch-size 2 --json

This uses a tiny randomly initialized U-Net and selected training pixels. See training smoke for checkpoint continuation and the measured audit for the executed CPU/MPS checks. Losses are training diagnostics, not accuracy results.

Pretrained U-Net/ResNet18 baseline

uv run --locked --extra baseline pvextent train baseline \
  data/pv4ger-seg-geographic-v1 runs/baseline-001 \
  --config configs/baseline-resnet18.toml \
  --weights-dir artifacts/weights/resnet18-a1-in1k \
  --download-weights --device auto --json

The configuration fixes three epochs and uses the complete declared training and validation partitions. The workflow normalizes native RGB crops, fine-tunes the complete model, accumulates global validation pixel IoU, Dice, precision and recall, and selects the best completed epoch by validation IoU. Test imagery is not opened.

See baseline training for configuration, continuation, failure recovery and a validation preview command; pretrained weights documents the pinned encoder and its separate license. The first configured three-epoch full run completed successfully with 3,282 training and 278 validation examples. Its best checkpoint was selected at epoch 3. The baseline audit records the internal validation results and artifact fingerprints; these are not an independent test result.

Compare the augmented training run

The fresh candidate completed the same three-epoch protocol on 3,293 training records, with 4,941 optimizer updates. The reference used 3,282 records and 4,923 updates. Both selected epoch 3 at threshold 0.5 on the unchanged 278 validation crops. All original 421 PV4GER test examples remain unopened.

Internal validation measure	Reference	Candidate
Pixel IoU	0.91572	0.91903
Precision	0.95477	0.96085
Recall	0.95724	0.95479

The candidate produced 39,287 fewer false-positive pixels and 14,951 more false-negative pixels. Paired crop IoU improved on 139 images, worsened on 138 and tied on one. This is a mixed development result; the experimental candidate does not automatically replace the original baseline. Changed shuffling, augmentations, batches and update counts prevent attributing the difference solely to the new masks. No roof-level false-positive rate or independent benchmark result has been established.

After producing each checkpoint's validation analysis against its own training snapshot, compare the retained reports without reopening raster files:

uv run --locked pvextent evaluate compare-validations \
  artifacts/baseline-resnet18-errors-001 \
  artifacts/baseline-resnet18-negatives-errors-001 \
  artifacts/baseline-resnet18-negative-comparison-001 --json

Use a new output path; existing completed analyses and comparisons are not overwritten. The comparison guide gives the complete training and analysis commands and pairing checks. The comparison audit records measured results, artifact hashes and the limited diagnostic on added training negatives. No trained model weights have been released.

Continue the unchanged candidate to six epochs

The registered duration control resumes the augmented candidate's epoch-three last.pt and changes only the total epoch target. It uses the same 3,293 training and 278 internal validation records, configuration, optimizer/RNG state, preprocessing, encoder, package versions and device. The total target includes earlier epochs, so this command adds epochs 4–6:

uv run --locked --extra baseline pvextent train baseline \
  data/pvextent-training-v2 runs/baseline-resnet18-continued-001 \
  --config configs/baseline-resnet18-continued6.toml \
  --weights-dir data/pretrained/resnet18.a1_in1k \
  --device mps --resume runs/baseline-resnet18-negatives-001/last.pt --json

The completed run has 9,882 total optimizer updates. Its best checkpoint is epoch five, with internal validation pixel IoU 0.92183825 and Dice 0.95932969, an IoU gain of 0.280326 percentage points over the parent best. The fixed final epoch six has lower IoU 0.91783037. Start-validation counts match the parent's final counts exactly; best-checkpoint reload also reproduces its recorded validation counts. Original checkpoints and data stay unchanged. No original test or external-pilot raster was opened, and no new external inference was performed to select this checkpoint or threshold. These internal metrics do not demonstrate that the earlier transfer failure has been resolved.

Use a new output directory for a reproduction. best.pt is for inference and cannot resume training; use a verified last.pt with its checksum companion. Regular continuation deliberately rejects changed dataset or split fingerprints. The baseline guide and continuation audit record the contract, hashes, per-epoch results and limitations.

Prepare separate future NRW training data

The new training-review acquisition contains 22 native 320 × 320 RGB crops at verified 0.1 m/pixel: eight Schleiden, eight Dülmen and six Marsberg-area cases. Registered per-region acquisitions retain the original 24 slots after two combined attempts failed without publishing. Two Marsberg slots lacked full-crop metadata coverage and were excluded. Full identity exclusions cover all 4,054 supplied records; distance protects 761 validation, test and unassigned centres, including the complete external twenty-case pilot.

Two independent blind AI reviews agree on seven positive proposals and defer fifteen cases. Their intersections contain 8,938 proposed PV pixels and 20,899 proposed background pixels. These are partial experimental proposals, not human-confirmed truth or imported canonical labels. All 22 canonical masks remain entirely 255, unreviewed and unassigned, with training use disabled. None was added to the six-epoch continuation.

Positive-mask curation, an immutable training snapshot and a separately registered NRW-enriched experiment remain next steps. Record target-source sampling and supervised-pixel shares explicitly: a few new positive cases can otherwise be overwhelmed by 3,293 uniformly sampled examples. The existing negative-only augmenter rejects PV masks; regular resume cannot introduce a changed dataset. See the continuation audit for actual acquisition provenance and proposed-label limits.

Explore the separate negative holdout

The negative holdout guide documents acquisition from three new NRW regions, metadata-only exclusion against all 4,034 existing records, blind AI review and freezing before model predictions. V1's 50% whole-crop background gate admitted zero examples. A registered V2 amendment reduced it to 5% before inference, keeping the same images, polygons, reviews and models. The failed V1 and its evidence are preserved.

V2 has seven new test crops and thirteen unassigned records. Both fixed models were evaluated on 150,374 consensus background pixels: the reference produced 31 FP pixels and the candidate zero. Their 220 and 93 PV predictions in ignored areas remain unscored. These small AI-reviewed partial masks do not establish human benchmark truth, recall, roof-level false-positive rates or general model superiority. The original 421 PV4GER test rasters remain unopened.

After completing the documented acquisition, review and freeze steps:

uv run --locked --extra baseline pvextent evaluate negative-holdout \
  data/negative-holdout-frozen-v2 \
  runs/baseline-resnet18-001/best.pt \
  runs/baseline-resnet18-negatives-001/best.pt \
  artifacts/negative-holdout-evaluation-v2 --device mps --json

Existing outputs are not overwritten; keep the executed output or choose a fresh reproduction directory. The new holdout, candidate pool and curation lineage stay reserved from training, including unassigned curation records. See the pilot audit for measured coverage, protocol history, checkpoint identities and limitations.

Explore the separate positive holdout

The positive holdout guide extends the same external acquisition cohort using exactly the thirteen cases not previously evaluated. Two new blind AI reviews inspected all thirteen originals. Three crops pass the fixed 100-consensus-PV-pixel gate; ten remain entirely ignored and unassigned. The protocol, image hashes, checkpoint identities and threshold 0.5 were registered before these annotations and predictions. All twenty acquisition cases remain reserved from training and tuning.

Both fixed models were evaluated once on 3,577 confirmed PV pixels. The reference has zero TP and 3,577 FN (0% recall); the candidate has 145 TP and 3,432 FN (4.053676% recall). There are no confirmed background pixels, so zero FP/TN counts have an empty denominator and the background FP rate is undefined. The candidate's 457 PV predictions in ignored areas remain unscored. These are partial AI labels, not complete-array/roof recall or human benchmark truth. The earlier negative pilot remains a separate diagnostic; the class-screened subsets cannot be combined into a representative confusion matrix.

After the documented independent reviews and freeze step, the command is:

uv run --locked --extra baseline pvextent evaluate positive-holdout \
  data/positive-holdout-frozen-v1 \
  runs/baseline-resnet18-001/best.pt \
  runs/baseline-resnet18-negatives-001/best.pt \
  artifacts/positive-holdout-evaluation-v1 --device mps --json

Use a new output path for a reproduction; the retained pilot is not re-inferred to tune thresholds or models. Because recall was unexpectedly poor, a separate technical control used the first two sorted, already evaluated validation crops. Both models' constructed input tensors, saved predictions and TP/FN/FP/TN counts matched exactly: four model/crop comparisons on two distinct images. The pilot was not re-inferred, its threshold and models stayed fixed, and the original 421 PV4GER test images remained unopened. No input-path or checkpoint-loading defect was demonstrated; the cause of the source difference remains unresolved.

The positive audit records the failure and control. Next development work concerns source differences and separate NRW training positives with fuller labels. Neither checkpoint is ready for an operational detection claim; the current pilot stays outside training and tuning.

Inspect frozen validation errors

uv run --locked --extra baseline pvextent evaluate validation \
  data/pv4ger-seg-geographic-v1 runs/baseline-resnet18-001/best.pt \
  artifacts/baseline-resnet18-errors-001 --device mps --top-k 12 --json

The executed analysis covers all 278 validation crops and reproduces the checkpoint's pixel confusion counts in the original execution context. Its offline gallery contains 30 difficult examples selected from error rankings; these examples are not a representative performance sample. The model, threshold, upstream source masks and annotation status remain unchanged.

See frozen validation errors for the protocol and the analysis audit for measured evidence. All original 421 PV4GER test examples remain unopened.

Inspect unannotated candidate predictions

uv run --locked --extra baseline pvextent evaluate candidates \
  data/nrw-roof-candidates-dortmund-v1 runs/baseline-resnet18-001/best.pt \
  artifacts/nrw-candidate-predictions --device auto --json

The candidate exporter has run on all seven prepared NRW crops. Their masks remain entirely 255 ignore pixels, with unreviewed status and pending human decisions. The gallery shows original RGB images first and starts prediction panels closed. It reports predicted-pixel counts without accuracy or false-positive/false-negative scores. The separate negative curation workflow creates new partial masks without changing these original all-ignore source annotations.

Both commands require a new output directory. Open an existing bundle's index.html directly instead of overwriting it. The original baseline ran on MPS; use the same device and package versions for the closest validation reproduction. See candidate predictions and the label-review workflow before approving or correcting labels.

Package boundaries

Module	Responsibility
pvextent.data	Dataset loading, labels, provenance, and preprocessing
pvextent.models	Model construction and architecture adapters
pvextent.training	Training and checkpoint creation
pvextent.evaluation	Segmentation and array detection evaluation
pvextent.inference	Predictions from images and trained checkpoints

The data module provides preparation and lazy train/validation tensor views. Model and training modules provide both the tiny smoke U-Net and the pretrained U-Net/ResNet18 baseline. Pixel evaluation and checkpoint selection are implemented for validation, with frozen error analysis, bounded candidate prediction export and exploratory positive/negative holdout diagnostics. General inference and a representative independent benchmark remain outstanding. No trained model has been released.

Continue with the research roadmap.