Repository navigation
OSM extracts
The walking and cycling legs, the direct walking and cycling trips and, when UseTomTomNetworkForCars is FALSE, the car use the road network of OpenStreetMap (OSM). The model reads it from the free shapefiles that Geofabrik publishes per region, in %NetworkModel_Dir%/Infrastructuur/OSM/<date>, and makes from them one network cache per extract: %LocalDataProjDir%/OSM/Final_Network_<date><revision>_<hash>.mmd. This page describes how a new extract becomes such a cache.
The extract that the model reads follows the analysis moment: the column OSM_file_date of AnalysisMomentPresets, in group 7 of ModelParameters.dms. The steps below add an extract. Their example is the extract of 1 June 2026, 20260601, under the analysis moment Y2026. What the model does with the roads is on Private transport and Walking and cycling legs; how the cache is named and checked is under OSM network cache.
- Step 1: choose the date and the regions
- Step 2: download the extracts
- Step 3: unpack them into the folder of the date
- Step 4: set the date for the analysis moment
- Step 5: check that the model reads them
- Step 6: make the network cache
- Which caches the new extract replaces
Take an extract from a date close to the analysis date of the analysis moment, so that the roads match the timetable. The date in yyyymmdd form becomes the name of the folder and of the cache.
The model reads the Netherlands, with Belgium and the German regions along the border. Which regions follow from the column OSM_NL_prov of the analysis moment, and the lists are in ModelParameters/Advanced/OSM_impl:
OSM_NL_prov |
List | Regions |
|---|---|---|
TRUE |
InProvinces |
the twelve Dutch provinces, Belgium, Niedersachsen and the Regierungsbezirke Düsseldorf, Köln and Münster of Nordrhein-Westfalen |
FALSE |
InOneFile |
the Netherlands as one file, Belgium, Niedersachsen and Nordrhein-Westfalen |
All analysis moments with an analysis date (Y2015 and Y2023 to Y2026) read per province.
Download the free shapefile (.shp.zip) of each region from Geofabrik. The provinces are under Netherlands, Belgium under Europe, Niedersachsen and Nordrhein-Westfalen under Germany, and the three Regierungsbezirke on the page of Nordrhein-Westfalen.
Geofabrik names the extract of the day <region>-latest-free.shp.zip, for example drenthe-latest-free.shp.zip. Older extracts are under their date, as <region>-<yymmdd>-free.shp.zip, in the raw directory index linked from the page of each region, as far back as Geofabrik keeps them; drenthe-260601-free.shp.zip is the extract of Drenthe of 1 June 2026. Take the same date for all regions.
Make the folder %NetworkModel_Dir%/Infrastructuur/OSM/<date>, for example Infrastructuur/OSM/20260601. Unpack each zip into a folder of its own in it, named after the zip without .zip: drenthe-260601-free.shp, belgium-260601-free.shp, duesseldorf-regbez-260601-free.shp and so on.
For each region, SourceData/Infrastructure/OSM/Prepare_SourceData/<region>/RoadsFile takes <region>-<yymmdd>-free.shp/gis_osm_roads_free_1.shp in the folder of the date, with the yymmdd of that folder, when it exists. Otherwise it takes <region>-latest-free.shp/gis_osm_roads_free_1.shp (#180). An extract downloaded as -latest can therefore keep that name, but give it the date of the folder when you can: then the name of the folder says which extract it is. The case of the letters does not matter.
The model reads only the road layer, gis_osm_roads_free_1 (.shp, .shx, .dbf and .prj). The other layers of the zip may stay or be left out.
Put the date in the column OSM_file_date of the analysis moments that use the extract, in AnalysisMomentPresets (group 7 of ModelParameters.dms), and set OSM_NL_prov to the list of step 1. A new analysis moment gets a new row there, with its other sources, and nrofrows of AnalysisMomentPresets goes up by one; see The GTFS feed for its analysis date and feed.
The analysis moment of a run is ModelParameters/AnalysisMoment. A call of UpdateAll.cmd can give another one with -set AnalysisMoment <name>, and a GeoDmsRun of its own with the environment variable GEODMS_Overridable_AnalysisMoment.
Read the roads of a region, in a GeoDmsRun of its own. Open a command prompt in the folder NetworkModel_PBL:
set "GEODMS_EXE=C:\Program Files\ObjectVision\GeoDms20.28.0.m\GeoDmsRun.exe"
set "GEODMS_Overridable_AnalysisMoment=Y2026"
set "OSM=/SourceData/Infrastructure/OSM/Prepare_SourceData"
if not exist batch\log md batch\log
"%GEODMS_EXE%" /Lbatch\log\OSM_read.log "%CD%\cfg\main.dms" @statistics %OSM%/Drenthe/Read_Roads_shp/fclass @statistics %OSM%/Drenthe/Read_Roads_shp/NrUnmappedFclass
The region in the item is the name of the list with _ for -: Drenthe, Noord_Holland, Duesseldorf_regbez. For each region the log has a line Read .../Read_Roads_shp from <file>, which shows which folder it read. Count of fclass is the number of roads; for Drenthe in the extract of 20260601 it is 130,822. NrUnmappedFclass counts the roads whose road type (fclass) is not in Classifications/OSM/roadType; the model only lets pedestrians use them. It is 0 for the extracts in use, so a higher number means that OSM has a new road type, which then needs a row in that classification. No line of the log may start with [E].
Make the network cache with the section sources of UpdateAll.cmd:
batch\UpdateAll.cmd sources -set AnalysisMoment Y2026
It makes the cache only when it is missing. For a new date it is always missing, because the date is in its name. The OSM network takes a few minutes and about 19 GB of memory (see Hardware requirements). Then batch\UpdateAll.cmd checks -set AnalysisMoment Y2026 reports PASS: the OSM network cache <path> exists.
The model makes every cache that depends on the extract again by itself, because the name of each holds it:
- the network cache holds the date of the extract and a hash of
OSM/Signature; - the pre- and post-transport caches, the direct walking and cycling trips and the car caches on the OSM network name the network cache by its date, revision and hash (
Impl/OSMNetwork_Tag); - the output holds the signatures of those caches.
So nothing has to be removed. The caches of the old extract stay in %LocalDataProjDir% until you remove them.
A new delivery under a date that is already used, for example a region downloaded again, is not noticed, because the names hold the date and the settings, not the content. Then remove the folder of the network cache, OSM/Final_Network_<date>_...mmd, and also the caches and output made from it: in IntermediateResults the Legs_... files and, for the car on OSM, the DirectCar_OSM<date>_... folders, and in Output the folders of the runs with that analysis moment. The next run makes them again.
NetworkModel PBL documentation