Repository navigation
Training the Year Built Model
Step by step, from an empty building_data to a retrained OrtoBuildingDetectionModel.mlnet.
Read Year-Built-Prediction first — in particular the section on what the label actually is, because it decides what a good result looks like.
| SDK | .NET 10 |
| Trainer |
mlnet CLI — dotnet tool install -g mlnet-win-x64
|
| Data | Feature columns populated in building_data for every county being trained on |
| Labels | the stored User year built column of building_data for those counties, written by the Year Built building-data update (DiGi-Organization/DiGi.GIS.PostgreSQL#94) |
Important
Model Builder in Visual Studio 2026 cannot train this. Its Microsoft.ML.AutoML.dll binds Microsoft.CodeAnalysis.CSharp 4.9.0.0 and ships no copy of it; VS 2026 ships Roslyn 5.900. Hitting Train fails with Could not load file or assembly 'Microsoft.CodeAnalysis.CSharp, Version=4.9.0.0'. This is not fixable from the .mbconfig — Entity.FromExpression sits in the sweepable-pipeline core, not in the tuner, so changing Tuner does not avoid it. Use the CLI, which carries its own dependency closure.
The regressor reads 172 feature columns. Ninety of them (the detection columns) and eighteen (the population columns) are created by runs, not by code, so a database that has never had those runs looks complete and yields a training table of defaults.
PowerShell -ExecutionPolicy Bypass -File "DiGi.Maintenance/Scripts/CheckBuildingDataColumns.ps1" -CountyId <id> -GroupExpect TOTAL 172 / 172. If detection or population read 0 / 90 or 0 / 18, stop and do Step 2.
Note
buildingdata/coveragebycountyid cannot answer this. It returns row counts only — Building2DCount, MissingReferenceCount and so on — and says nothing about which columns exist or how full they are.
Per county, two independent runs in either order.
Detections — run the pipeline with the scoring and year-built writes off, so the model being replaced never scores anything and its answer is not written next to its own training features:
{ "CountyIds": [<id>], "ScratchDirectory": "scratch",
"ExportImages": true, "RunPrediction": true, "UpdateDetections": true,
"Score": false, "UpdateYearBuiltData": false, "UpdatePredictedYearBuilt": false }See Year-Built-Prediction for how to invoke the runner.
Population — in the DiGi.GIS.PostgreSQL.UI tray application, on the database machine, start Update building data, select update type Statistical and the counties. It reads statistical_data_collection, which has no controller, so the task log is the only evidence the Population data is present.
Re-run Step 1 before continuing.
cd <repo>\DiGi.GIS.ML
dotnet bin\YearBuiltPredictionTrainingTableConsoleApp.dll `
--output "user files\reports\YearBuiltTraining.tsv" `
--counties 104106,80328,75125,204,5It reads the label as a paged read of the stored User year built column of building_data, projects features through DiGi.GIS.IO.Query.YearBuiltPredictionInputColumns — the same allow-list the inference path uses, so training and scoring cannot drift — and writes a fixed 174-column schema: Reference, 172 features, Year built.
Two guards built into it:
- The label is read, not re-derived. The column was written once by the Year Built building-data update from the most frequent exact user year; a county the update has not reached returns no labels and is skipped with a warning rather than trained on.
- Every allow-list column is materialised, defaulted when the source omitted it, because detection columns exist per county only for the years that county's imagery covers. Counties would not otherwise concatenate.
Read the warning it prints. N of 174 columns never vary is the check that Step 2 actually landed — after a good run only a handful should be constant (single-county administrative values, and the two dead population years).
mlnet regression `
--dataset "<repo>\DiGi.Test\user files\reports\training_all.tsv" `
--label-col "Year built" `
--ignore-cols "Reference" `
--has-header true `
--split-ratio 0.2 `
--train-time 3600 `
--name YearBuiltPrediction -o <output folder>--split-ratio 0.2 matches the incumbent's TrainValidateSplitValidationOption. The CLI writes a .mlnet, a .mbconfig and the generated consumption / evaluate / training C# files.
Or train from a config, as the 2026-10-08 model was. mlnet train --training-config <file>.mbconfig reruns a recorded configuration exactly: the column purposes, formats and categorical flags, the label, the split and the time budget. Copy the committed .mbconfig next to the output, set DataSource.FilePath to the new table, empty RunHistory.Trials, and point RunHistory.ModelFilePath at the new output, so it does not overwrite a model you still need. The committed .mbconfig records the configuration the shipped model was trained with. Since 2026-10-08 that config does not mark the text columns categorical (AutoML featurizes them as text) and converts Is residential / Is occupied from booleans. That changes the generated ModelOutput, so carry the regenerated output types into consumption.cs and keep its hand-maintained MLNetModelPath resolver.
Train a twin with the same settings on the non-holdout FNV-1a rows only (1 in 5 of the references, hash % 5 == 0 is the holdout): the shipped model is trained on every row and cannot be honestly scored on any of them, so the twin is the measurement vehicle the bar is applied to — the shipped model's own holdout figure is kept, flagged contaminated, for the contrast.
Observed on 20 241 rows: LightGbm leads throughout and the search plateaus early — the best R² moved from 0.8858 at 18 models to 0.8892 at 323. The tuner spends most of its budget on FastForest (166 of 186 trials in one sample) while every leading result is LightGbm, so a longer --train-time buys little.
The generated ModelInput binds by display name, so the members come out as Floor_area, Municipality_population_2019, Year_built. The legacy Polpulation misspelling disappears on its own, because the column names in the new table are correct.
This is the step that decides whether the retrain was worth it, and the CLI cannot do it. AutoML reports R² on its own internal random split; it cannot score the incumbent on the same rows and it knows nothing about a grouped split.
Score both models on the same holdouts, and report Create.YearBuiltPredictionAccuracyResult for each against:
- constant 2008
- first detection year
- the incumbent, through the deployed path (the model installed as
OrtoBuildingDetectionModel.mlnet— run the evaluation before swapping the committed file, or the row scores the new model against itself)
Read every model through the deployed path. --model is scored through ML.NET's TextLoader (the retrained row). For one and the same model, that reads about 0.07–0.09 years higher MAE than the production binding, with RMSE and R² within 0.01; the cause is not yet established. To score the twin the way production will, copy it as OrtoBuildingDetectionModel.mlnet into an empty folder and run the evaluation app from there: the model resolver checks the current directory first, so the deployed path row is then the twin.
on two splits:
- random 20 % (by reference) — FNV-1a of the reference, 1 in 5, comparable with the incumbent's methodology
- grouped 20 % (by subdivision) — FNV-1a of the subdivision identifier, no subdivision spanning train and holdout, the control for memorised neighbourhoods. On the 2026-09-29 table the heuristic scores R² 0.652 random against 0.644 grouped, so a model that holds up on the random split and collapses on the grouped one is reading the map, not the imagery. The grouped row is not valid for a twin trained on the non-holdout references: most rows of a subdivision carve were in its training data (the 2026-10-08 twin scored a contaminated 0.134 there). A grouped measurement needs its own twin, trained without the grouped carve
The bar is the first-detection heuristic's figure on the FNV-1a holdout — beat MAE 0.434, RMSE 1.730 and R² 0.786 simultaneously, not "beat 0.9010" and not "beat the incumbent's historical figure". Beating R² alone is not sufficient (DiGi-Organization/DiGi.GIS.ML#4). Those figures were the heuristic on train8 detections. After a detector change, compute the heuristic on the same detections the model is trained on. The 2026-10-08 retrain on train9_fresh faced a stronger heuristic (MAE 0.285 on the same carve) and beat it at 0.277 / 1.129 / 0.909 (Year-Built-Prediction).
Beside the .mlnet, record: the feature list and its hash, the county identifiers and names, the row count, both split definitions, the metric table from Step 5, the categorical encoding decisions, the date, and the source commit. Without it the next person cannot tell which data a model came from.
Keep the previous .mlnet — it is needed to measure the next retrain against something real. It is committed, so git is a sufficient backup.
- Year-Built-Prediction — what the model does, requirements, running the pipeline
-
DiGi.GIS.IO —
YearBuiltPredictionInputColumns, the feature allow-list