Components and merging

A run of the model computes components. A component is a direct trip from origin to destination (walking, cycling or by car), or public transport with one type of pre-transport and one type of post-transport. Each component has its own output folder. The merge combines the components of a variant into one Pareto front per origin-destination (OD) pair. Which components a run computes is set with the switches of the variant; see The variant on Model parameters.

Contents

• The components
• The direct trips in the output
• Car and bicycle ownership as criteria
• Merging the components

---

The components

Classifications/Components lists every component (#117, #133):

• the direct trips WW (walking), CC (cycling) and Car (AA in ModeUsed, with start and parking time);
• the public transport components <pre-transport code>_PT_<post-transport code>: every combination of the five pre-transport types (W, C, Crm, Crmt, Cr) and the six post-transport types (W, C, rmC, rmtC, ovfC, rC), 30 in all, such as W_PT_W, C_PT_ovfC or Crm_PT_W. The types are described on Walking and cycling legs.

A direct trip is a component of its own and never part of a public transport component. It needs no chains. The car keeps several routes per pair (Advanced/ParetoLegs_Car), and walking and cycling do not depend on the departure moment.

One component per run step

One GeoDmsRun computes one component, Impl/ComponentName. It follows from ConfigSettings/Overridable/RunComponent, which batch\UpdateAll.cmd sets per step in the environment variable GEODMS_Overridable_RunComponent. In GeoDmsGui, ModelParameters/GuiComponent takes its place. Everything else follows from the name: the pre- and post-transport type, the direct trips, the output folder and its signature.

What a component needs:

• A public transport component reads the pre- and post-transport caches and the chain cache (Classifications/Components/WithPT). The output step joins them as described on From chains to journeys.
• Car reads the car caches (DirectCar).
• WW and CC compute their trips in the output step itself.

UpdateAll.cmd therefore skips the chain cache for the direct components, and the car caches for every component but Car. The public transport components of the variant share the chain cache. Its last boarding moment, and the _v in its signature, follow the longest pre-transport leg over their pre-transport types (Impl/PT_Chains_Max_V_Time), not that of one component. The signature of the output of a public transport component holds that of the chain cache. A change of that longest leg, for example by a pre-transport type with a longer maximum time, therefore also gives the other public transport components a new output folder.

Output folder and departure moments

The output folder of a component is <Analysis_date>_<PartOfDay>_<OutputLabel>_<component>_<hash> (NetworkSetup/GenerateOutput/CompOutput_T/Dir). It has _fb after the component when the component cycles on the network of the Fietsersbond (Impl/CyclingNetwork_DirTag). At the end is a hash of eight characters of the signature of the output, as the caches have it (Templates/StringHash_T). Because of this hash, output with other settings gets a folder of its own beside the earlier one, and switching back to the earlier settings finds the earlier output again. For WW and CC, the origin set (Impl/OrgSet_PathTag, for example b2025) takes the place of OutputLabel. Their output does not depend on the part of the day, so every part of the day and every variant label reads the same folder. The subfolder PerBlock holds tt_<moment>_<n>of<N>.parquet (or .csv). The date and the component are in the folder name, and the settings in signature.txt; this keeps the paths short.

WW and CC do not depend on the departure moment (Classifications/Components/TimeInvariant). They compute one moment per block, and their file applies to every departure moment of every part of the day: tt_alle_<n>of<N>.parquet (Classifications/Components/FileMoment). Car on the OSM network also computes one moment per block: the congestion moment of the part of the day (Classifications/Components/SingleDepartureMoment, 07:30 in the morning rush hour). Its speeds do not depend on that moment, but its file and its signature (_dep07h30m) carry it. On the TomTom network the car trip is always time-dependent (Impl/CarTimeDependent follows UseTomTomNetworkForCars), and Car computes every departure moment.

The signature of the output folder names the chain cache and the public transport settings only for a public transport component, and the car settings only for Car. It holds every setting that changes the output of the component and is in no cache that the component reads. It holds such a setting with its value, also when that is the default. It holds no setting that does not change the output, so that runs that differ only in such a setting share the output:

• for WW and CC: the maximum time of the trip, the network with its speed, the OSM network cache, the junction penalties and the clean-up of the network, and for CC the start time. Their costs per minute, and whether these count in Price, are in it only with csv, which writes the price. With parquet, the merge computes the price of WW and CC, and these costs are in the signature of the merge (ComponentMerge_Def/DirectPriceTag). WW and CC have one route per pair, so the criteria and the epsilon of a front are not in their signature either;
• for a public transport component: MaxTravelTime, the wait at home (Advanced/MaxWaitingTimeAtOrigin, IncludeWaitingAtHomeInTravelTime) and whether the costs of walking and cycling count in Price. Per pre- and post-transport type it holds the network, the maximum time of the base set and that of the type, the start, parking and penalty times of cycling, and the price per minute or the rent of the OV-fiets with the search distance to its stops (Classifications/TimeInvariantTypes/OutputSignatureTag);
• for Car: Car_StartTime and Car_ParkingTime, which are added to the travel time when the car cache is read;
• with csv the exported columns (Export_TravelledDistance, Export_PriceInformation); parquet always has all columns of its kind;
• the departure moment of Car on the OSM network, Advanced/MakeMedian, and the settings that place the origins and destinations (Impl/OrgDest_Tag: the weight of the centroids, the CBS year of a region set without a year of its own, the BAG date, the public transport nodes as destinations).

Every run computes the signature of every component in the same way (CompOutput_T). The merge, Output/FrontCheck and Output/OD_PairRoutes therefore find the folder of each component they read. For this, the signatures of the chain cache and the car cache follow from the departure moments of the part of the day (Impl/PartOfDay_Moments), and the criteria follow from MinimiseCriteria (Classifications/ComponentParetoAttr), not from the component of the run.

The direct trips in the output

The direct components go through the same output step as the public transport components (PerDepartureMoment_T), without pre-transport, chains and post-transport (Impl/WithPT FALSE): their options (AllOptions) are the direct trips of the component.

1. The trips come from PublicTransport_Prep/Direct: Walking and Cycling (see step 5 of Walking and cycling legs) and the car per congestion moment (see Private transport).
2. PublicTransport_Prep/Direct_OD holds the direct trips of the component that do not depend on the departure moment, with duration, distance, price, augmented price and mode. NetworkSetup/impl/Direct_OD adds, once for all blocks, the places of origin and destination, the ownership that the trip needs (NeedsCar, NeedsBike), and the origin block and part of each row. Each part selects its rows.
3. On the TomTom network the car trip depends on the departure moment (Impl/CarTimeDependent) and is not in Direct_OD. Each departure moment reads the rows of its origins from the car cache of that moment (CarRows). The cache already holds the front on travel time and distance per OD pair; DirectCar (template CarFront_T) adds the start and parking time and the price to its routes.
4. A direct trip leaves at the departure moment and arrives after its duration. Its time, distance and price are in the pre-transport columns (PretransTime, TravelDist_V, PretransPrice); public transport and post-transport are zero. ModeUsed is WW, CC or AA.
5. The two condensations of step 7 of From chains to journeys keep the front per pair. Walking and cycling have one route per pair. The car keeps the routes of its front on travel time and distance, whose price follows from the distance.

The price of a direct trip:

Trip	Price	Price_augm
walking	Direct_Walking_Time_Costs per minute with Advanced/IncludeWalkingCostsInPrice, else 0	Direct_Walking_Time_Costs per minute
cycling	Direct_Cycling_Time_Costs per minute with Advanced/IncludeCyclingCostsInPrice, else 0	Direct_Cycling_Time_Costs per minute
car	the distance times Car_CostPerKm plus Car_FixedCosts_PerTrip, with Advanced/IncludeCarCostsInPrice	equal to Price

Car and bicycle ownership as criteria

MinimiseCriteria, and in the merge MergeCriteria, can also name Car and Bike (#117). Each adds a criterion that is true or false:

• NeedsCar: the direct car trip (AA). There is no park-and-ride pre-transport, so no public transport journey needs a car.
• NeedsBike: the direct cycling trip (CC), and a journey with a pre- or post-transport leg on one's own bicycle. The OV-fiets is rented, so ovfC does not need one's own bicycle (Classifications/TimeInvariantTypes/NeedsOwnBike).

The criteria are minimised, false before true. An option that needs neither a car nor a bicycle is therefore never dominated by one that does. A public transport journey that is more expensive and slower than the car trip stays next to it. A car trip that is more expensive and slower than a public transport journey still drops out. One front then serves every group of travellers:

• with a car and a bicycle: all rows;
• without a car: the rows with NeedsCar 0;
• without a car and a bicycle: the rows with NeedsCar and NeedsBike both 0.

Without these criteria, such journeys drop out. An analysis of travellers without a car then finds no option for those pairs, or a slower one than exists. Output/PostAnalysis/FindMedian, for example, leaves out the car trips (ExcludedModes = 'AA') to describe accessibility without a car.

Rules:

• Within one component, each criterion is the same for every option, so in the run of a component it does not change the front. The criteria matter in the merge. The defaults of MinimiseCriteria and MergeCriteria (both 'Price,Time') do not name them.
• A criterion that cannot differ between the options is left out (Classifications/OwnershipParetoAttr_Candidates/IsRelevant), also from the signature. NeedsCar counts only with the direct car trip. NeedsBike counts only with the direct cycling trip or a pre- or post-transport type on one's own bicycle, and in the merge only when a merged component needs one's own bicycle (Classifications/Components/NeedsBike).
• The chain cache does not depend on them: the chain joiner computes public transport only.
• The signature of the output carries the extra criteria in its MinCrit part, as the letters of ModeUsed: MinCrit-Price_Time_A_C.
• Both columns, NeedsCar and NeedsBike (0 or 1), are in the csv output of a component and in the merged output, whatever the criteria; in the parquet output of a component they are constant and left out.
• The criteria need the true-or-false criteria of pareto_optimal_eps (GeoDMS #1287), which every engine that the model accepts has.

How many options of each component the criteria keep in a merge is shown under MergeCriteria.

Merging the components

Merging is optional, and it computes no journeys itself. With ModelParameters/Merge on, batch\UpdateAll.cmd ends with the merge, as the component Merged, also when the variant has one component. Per origin block and departure moment, the merge combines the output of the components of the variant into one Pareto front per OD pair (NetworkSetup/ComponentMerge). With one component, it still fills in the columns that the parquet of the component leaves out, and it applies MergeCriteria. So every variant has its final table in a Merged folder, with the same columns (#148). With 'Time', for example, only the fastest of the car routes of Car stays.

Several merges, each with its own components and criteria, are rows of ModelParameters/MergeVariants. UpdateAll.cmd all -variant all makes them one after the other. It computes each component only once, because a component with the same settings has one output folder in every merge. It writes every merge to its own Merged<n> folder (see Merge variants and settings per call).

The steps

1. The merge reads the parquet files of the components (<Analysis_date>_<PartOfDay>_<OutputLabel>_<component>_<hash>/PerBlock, see Signature), with a reader per kind of component (Classifications/Components/ExportClass: Direct, Car or PT). The components must therefore be written with OutputFormat 'parquet'; ChecksBeforeRunning/Check_OutputFormat stops a run with a merge before it computes anything when they are not. The parquet has the times in whole seconds and the columns of The parquet files. That format is in the signature of a component (_parquet-s3-), and the merge reads only a folder whose signature equals the one it computes. So it never reads an older output. Such an output has the times in minutes, which GDAL would read as seconds without a warning, or it lacks the distances and prices of pre- and post-transport or the walking and waiting at transfers. A component with one departure moment (WW, CC, and Car on the OSM network) serves every departure moment of the merge.
2. It fills in the columns that are constant in a component or follow from its columns: the price of WW and CC from the travel time and that of Car from the distance, as in PublicTransport_Prep/Direct; ModeUsed WW, CC or AA; NeedsCar and NeedsBike from the component; zeros for the missing times and distances. A direct trip is its own pre-transport, as in the csv of a component: its time, distance and price are also PretransTime, TravelDist_V and PretransPrice. Every row gets the column Component, the component it comes from.
3. It splits time, price and distance per part and per mode, as the csv of a component does (see The csv files). PTTime is Traveltime minus PretransTime and PosttransTime, and minus WaitingTime when Advanced/IncludeWaitingAtHomeInTravelTime is on. InVehicleTime is PTTime minus TransferWalkTime and PTWaitTime. PTPrice is Price minus PretransPrice and PosttransPrice, in cents. Walking, cycling and car get the time, price and distance of the pre- and post-transport with that mode. The mode of the pre- and post-transport follows from the component (Classifications/Components/PreNetworkType and PostNetworkType); the OV-fiets counts as cycling.
4. Per OD pair it keeps the rows that are not dominated, with pareto_optimal_eps on Classifications/OutputParetoAttr. These are the same operator and epsilons as in a run, with price and time in cents and seconds. The rows are in the order of the components in Classifications/Components, direct trips first. Of rows that are equal on every criterion, the first stays.
5. It writes the result in OutputFormat, parquet or csv, to <Analysis_date>_<PartOfDay>_<OutputLabel>_Merged<MergeRunNr>_<hash>, one block per phase. The counts per block go to Tellingen, and NetworkSetup/GenerateOutput/OD_Counts_FromFiles adds them up. Both formats have the same columns in the same order, whatever the components of the variant; a column that does not apply to a row is 0. The times are in minutes, in the parquet as float32, while the parquet of a component has them in whole seconds (#149). The columns of the split come after Component, so that a reader that reads the csv by position keeps the earlier columns.

Criteria

The merge has its own criteria, MergeCriteria (default 'Price,Time'), apart from MinimiseCriteria of the components. They can be any subset of Price (or Price_Augm), Time, Car and Bike, with at least Price or Time (ChecksBeforeRunning/Check_MergeCriteria). With 'Time' or 'Price' alone, each pair and departure moment keeps exactly one row, the fastest or the cheapest. Car and Bike count only as far as a merged component can need them.

With IncludeWalkingCostsInPrice FALSE, the default, walking is free. The direct walking trip then stays in the merged front of every pair within walking range, and an option that is not more than 1% faster drops out against it.

Signature

The folder of a merge is <Analysis_date>_<PartOfDay>_<OutputLabel>_Merged<MergeRunNr>_<hash>, with _fb before the hash when a component cycles on the network of the Fietsersbond (ComponentMerge_Def/Dir). Its signature lists the components, each with the hash of the signature of its output ([W_PT_W:<hash>]). It also holds the criteria, the format (<OutputFormat>-m6) and the costs of WW and CC, whose price the merge computes. The signature therefore changes when a component is computed with other settings, and a merge with other criteria or other components gets a folder of its own; MergeRunNr only labels the merge. The full signatures of the components are in their own folders. In the signature of the merge, they would make it too long for the hash.

The merge computes the folder of every component from the current settings (ComponentMerge_Def/PerComp, with NetworkSetup/GenerateOutput/CompOutput_T). It reads a component only when the signature.txt in that folder equals the signature that it computes (ComponentMerge_Def/CompDir). Otherwise it stops with an error that names the component: that component has not been computed with the current settings.

Merging against one run

The 1% rule of the front is not transitive: an option can drop out against an option that drops out itself. The merge selects in two rounds, first within each component and then over the components. Compared with one front over all options of the variant at once, it can therefore keep or drop a few options that are at most about twice the epsilon (2%) slower than a cheaper option. The cheapest option of every OD pair stays in both.