Skip to content

GeoDms 20.20.0

Latest

Choose a tag to compare

@MaartenHilferink MaartenHilferink released this 10 Sep 22:09
Immutable release. Only release title and notes can be modified.

GeoDms 20.20.0 covers everything since 20.17.0; the 20.18 and 20.19.x builds in between were interim pre-releases and their notes are folded in here. Headlines: a stored item is now read by an operator like any calculated item (#587), which is what lets a check refer to it and a fence hold it; a fifth polygon family of GeoDMS's own, dms_*, with exact arithmetic and no validity requirement (#1214); .mmd stores that read back where they used to promise data they did not hold (#1247, #1264, #1267, #1245); an attribute declared without a domain finds its unit through a grouping container (#1244); deadlocks that used to park a computation at 0% CPU now report themselves (#1233); a refused memory commit is an error on the item instead of an access violation per thread (#1269); the XML configuration notation reads back what the writer emits (#1261); and the teardown, allocation and connect work of the 20.19 line. The .g (GLOBIO) flavour ships alongside .m, .c and .l.

Flavour Asset
.m — Windows, MSBuild GeoDms20.20.0.m-Setup-x64.exe
.c — Windows, CMake GeoDms20.20.0.c-Setup-x64.exe
.g — Windows, GLOBIO compatibility GeoDms20.20.0.g-Setup-x64.exe
.l — Linux, Ubuntu 24.04 GeoDms20.20.0.l-linux-x64.deb, .tar.gz (+ .sha256, .sha256.p7s)

Test report

Things a modeller must act on

  • A ring's role is read relative to the first ring, not to the compass (#1212). The GEOS and CGAL readers judged each ring absolutely — shell iff clockwise — so a feature whose rings are uniformly counter-clockwise, which is what a source listing coordinates in latitude/longitude order gives you, lost its shell and promoted the lake behind it. Nothing was rejected and nothing was empty: ordinary-looking geometry with the wrong area. A configuration's areas may change on upgrade; the older numbers were the wrong ones. Measured on a real model in #1230: in the RSopen regression, 105,051 BAG building footprints stored counter-clockwise were read as empty and are now read correctly, while 232 CBS land-use polygons whose rings are only partly flipped were read 2,298 ha too large and now match their stored area. Both directions occur; a feature that is only partly flipped still needs fix_winding_order.
  • The two remaining composition deprecations become errors in GeoDms 21 (#1038), and say so: a declared ValueComposition that differs from the calculated one, and a geometry argument whose composition is not what the operator means. A (polygon) declared on a points2sequence result — which computes an arc — now writes a polyline layer through gdalwrite.vect, as it always silently did; the message tells you which item to fix.
  • A .mmd store written below a DisableStorage case parameter is refused at write time (#1245) rather than producing a store that fails in the reader with "Domain mismatch between the specified Domain (/store) and the domain of the results". TreeItem::Copy returned early for a case parameter before carrying the configured properties over, so the DisableStorage never reached the instantiated parameter, and the writer accepted the shape.

Storages

  • A stored item is read by an operator (#587). A table unit read from a storage gets the calculator storage_read_table(…) right after its storage manager's UpdateTree, its stored attributes subitem(<that key>, attr), and a stored parameter storage_read_value(…); every storage manager reads through these — shp, dbf, odbc, xyz, FSS, cfs, strfiles, tif, bmp/pal, gdal.grid and gdal.vect. A stored item thus has a raw key like a calculated one. Two things that were impossible before follow from that: an IntegrityCheck on a stored item may refer to that item (a self-referring check is not a supplier of the item it checks), and a PhaseContainer may fence a container holding stored members — it asks the item that carries the data whether a member is ready. The gdal.vect attributes of a table are read in one pass over its features; a stored member that its table's read does not name reads on its own.
  • A shapefile is written with closed rings. Both writers stored a polygon's rings exactly as the value held them, but a GeoDMS polygon need not repeat a ring's first point at the end and the ESRI specification requires it. gdal.vect — the engine's own default reader for the file it just wrote — refused such a file with "Non closed ring detected".
  • gdalwrite.vect recognises a geometry by its composition, not by the name geometry (#1232). A geometry column called anything else was measured against the driver's creation field types, so the write was refused with "driver ESRI Shapefile does not support writing of values" and the layer was created without a geometry field.
  • A layer's companion columns come into play at write time (#1229).
  • Missing VAT when exporting to dbf or gpkg (#973) — four separate defects, among them a native shapefile branch that compared against the wrong driver name and had been dead since 2023, with the resulting storage failure reported as "Export ready". A unit carrying an attribute of another domain exports as a table again (#1145).
  • Storage readers validate their input. Bytes from a shapefile, a GDAL layer, a TIFF, a DBF, a BMP or an xyz file were trusted through assertions that are optimiser assumptions in a Release build; a malformed file could read past a vector or overflow a buffer through a mismatching column type. Each such condition is now a check that names the file. The TIFF reader takes a missing SampleFormat from the configured type's category instead of reporting an unknown pixel type, and reads the ModelTransformation tag with the right width.

.mmd dictionaries

  • A store materialises every attribute it declares (#1247). Two items of one store can resolve to the same thing — an aliased table, an attribute repeating another's source, two items with one expression — and the store held such content once while the dictionary declared every claimant, so it promised data it did not have. The write exited 0; the failure surfaced on first use, possibly on another machine months later, as "Data not found in .MMD storage folder". An attribute whose rule names anything outside the store is now written.
  • A store-local rule is recorded as a rule, not as bytes (#1264). Where an attribute's rule names only identifiers inside the store, the dictionary carries the rule and the store holds no bytes for it; the reader re-applies it.
  • A tiled domain reads back (#1267). The dictionary recorded a domain by its Range alone, so a reader tiled it the default way — 65,536 per tile — and the per-tile .dat and .seq files no longer matched; a store over a TiledUnit could not be read at all. The tiling is now recorded as a rule.
  • A zero-length attribute file reads (#1246), and the dictionary's unit-range guard no longer takes an interest it then has to satisfy by starting a computation while it writes (#1266).

Polygons

  • The dms_* polygon family (#1214) — GeoDMS's own backend next to bp_, bg_, cgal_ and geos_: dms_intersect, dms_union, dms_xor, dms_difference for all six point types, plus dms_polygon, dms_union_polygon, dms_overlay_polygon, dms_polygon_connectivity and the two Minkowski operators. It does not require valid input: each sequence is read as one closed walk under the even-odd rule, so self-intersecting, self-touching, overlapping and unclosed rings all have a defined meaning. The dissolve nodes every element once on one lattice and finds crossings by an exact plane sweep, in the shape boost::polygon uses; on the 124,662 Amsterdam buildings it agrees with geos_ to 3·10⁻¹⁰ in area and runs in a fifth of the time.
  • Nesting-based winding-order operators (#302) on GEOS: fix_winding_order, fix_polygon, has_correct_winding, geos_polygons_by_nesting. #1219points2polygon no longer requires a closed ring the readers close themselves. #917 — Minkowski sums take the kernel as an argument, deprecating 48 bp_* names.

Allocation

  • A land use type that is no option no longer fails the allocation. Measuring the distance from optimum — which only builds the DiscrAlloc completed … status line — added the shadow price to an undefined suitability, where the bidding loop itself skips such a type. On a model with null suitabilities that ended the whole run with numeric overflow in the shadow price arithmetic. The same addition used to wrap silently, so a type that was never an option compared as a better bid: the "at most N from optimum due to M better options" figures have been wrong for such models for as long as they have existed, and are now right.
  • The Simulation-of-Simplicity perturbation range is checked once per run, before the solve starts rather than partway through an allocation.
  • discrete_alloc arithmetic audited (#1196). The claim aggregates were summed in UInt32: two land use types at the "no limit" maximum claim sum to exactly 2^32, read back as zero, and the run was rejected as infeasible — four cells are enough. Shadow prices go through checked arithmetic naming the component that overflowed, and six discrete_alloc_*_pi64 names offer a 64-bit perturbation.

Robustness

  • Deadlocks report themselves instead of parking (#1233). Each showed the same thing: a computation at 0% CPU, no error, no log line. The one that bit hardest is in deterministic tile accumulation, where a tile that unwinds — a read error, a cancellation, a failed allocation — advanced nothing and every later tile waited for it forever. Every function that acquires a lock now declares the outermost acquire it makes and is checked on entry against what the caller holds; a static pass over those declarations runs in analyze.bat. That turned up and fixed eight more silent parks along the way, the last being a report of a failed allocation issued under the token registry's own exclusive hold, and a NotifyTargetCount outer to that registry.
  • A refused memory commit is an error on the item, not an access violation per thread (#1269). At a commit limit VirtualAlloc(MEM_COMMIT) left the pages reserved and inaccessible, the allocator handed the store out regardless, and every worker walked into the same wall — 23 faults under an 8 GiB job cap. It now raises MemoryAllocFailure, so the run ends with [E] memory allocation failed on the item. Every thread starts with 1 MB of committed stack.
  • Memory under a deferred commit (#1259). A producer whose result would not fit is walked once per pass, starts at once, and the ledger charges the work in flight rather than only what is already produced; parse_xml runs on the worker pool now that the token registry scales, and GeoDmsRun releases an item's interest when that item is done instead of holding the whole subtree until exit. On a real BAG extract with fifty filesets: 151 s where the serial shape took 310 s, at 28 GB peak.
  • A Release-build fault leaves a minidump (#1241). The handler used to run only after Windows had unwound every frame, so there was no stack to capture and the window simply vanished. The dump is now written in the exception filter, on a parked writer thread with a stack of its own — which is what makes a stack overflow dumpable at all.
  • A map view of an irregularly tiled grid no longer terminates the GUI (#1242) when the cursor leaves the tiles; such a location answers an undefined row.
  • GDALAllRegister is serialized (#1234), and the issued-warnings set is guarded (#1258). GeoDmsRun died with 0xC0000374 or 0xC0000005, no [E] line and part of a tif set written, when ten gdalwrite.grid items were pulled by one ExplicitSuppliers trigger.
  • A derived values unit stays alive (#1237). The single-argument area / arc_length / mls_length derives its result unit from the coordinate unit, but a data item holds its values unit only weakly.
  • A process no longer aborts at exit with a corrupted heap. Reporting the final allocator summary allocates, and allocating can post to the main-thread operation queue — which, by the time a static destructor of the same library runs, has itself been destroyed. It cost roughly one run in ten on a loaded machine and nothing on an idle one. The summary now comes from an explicit terminate step while the process is still whole.
  • Runtime-core and viewer hygiene from doc/code-fixes.md: a moved-from interest holder no longer keeps the old interest alive for as long as the temporary lives; fail reasons, the garbage can, the task group and the registry checked where they were assumed; GDI handles and WM_COPYDATA payloads validated; a circular diversity kernel counted wrongly from radius 4 on. Fifteen confirmed defects in the first phase alone.
  • #1191 — what aborts the process is named, and the cancellation paths no longer abort it. #1206 — GDAL cleanup ordering at shutdown. #1225 — a GUI closed with work abandoned no longer ends in uncaught DmsException: Cannot find Domain unit .. #1226 / #1227 — the export dialog no longer hangs on a cold item; GetName() no longer hands out a lock on the token registry, and a self-deadlock on it is reported rather than parked on. #1249 — the item schema writes its nodes again, and the failure walk finds its ancestors.

IntegrityChecks and fences

  • A check now guards an item read from a storage (#1209) — consumers were handed an unguarded source reference and the check ran only after data preparation. With #587 a check may also refer to the stored item it guards.
  • A check on an ExplicitSuppliers item guards the item that declares it (#1218). Selecting either item in the TreeView previously ran no check at all.
  • Indirect expressions behind a fence are rejected (#1199), and the phase scan honours the static argument policy (#1224).

Charts and views

  • The map view's class breaks are an operator application (#1248), demanded like any other item — the last piece of view work that ran as a special item writer. View work that awaits an item is polled by its DataView (#1255) instead of parked on a detached thread that blocked in a read lock.
  • A chart's axis labels are centred on the positions they name (#1263) — they began at the middle of their bar and ran into the next one. A new chart no longer waits for its own classification (#1221), position axes fit their data instead of forcing zero into view (#1222), row-number axes are labelled from the domain's Label attribute (#1207) and tick labels take the thousand separator (#1223).
  • Each chart kind has its own icon (#1211), the View and Window menus and window titles are drawn from the TreeView's icon set (#1220). The active TreeView item is visible again (#1235). Every DataView caption leads with the kind of view, and a missing SpatialReference is named (#1238).
  • Editing a calculated class break, colour or label copies the values first (#634); a classification can be copied from one map view and applied in another (#734); the EventLog filter panel is no longer squeezed (#1213); the legend-width menu items name the keys that actually work (#1217); dragging a row height distributes the cell pitch, so a bordered symbol column no longer drifts out of step with the rest (#1208). The main window's placement is recorded while it is up, so a session ending in a crash no longer restores a placement from days ago.

Networks

  • connect accepts its two geometries the other way round (#1228). Passing the points first turns them into a choice set: every arc gets the nearest of them, measured to the nearest location on that arc; a reversed connect_info names its relation member point_rel. The filtered search reports when it found nothing, and the reversed search is capped at maxSqrDist instead of being cancelled.
  • connect_matrix / dist_matrix (#1236) — every point within a maximum distance, as a sparse matrix: a fresh domain of (feature, point) pairs carrying arc_rel, point_rel and dist, plus the cut-point members for connect_matrix. Both have _eq and _ne variants.
  • Bi-criteria (pareto) impedance_matrix (#856), and od:StartPoint_rel is written (#1210) — NYI since 2022, requesting it yielded uninitialised memory.

Configuration language and diagnostics

  • An attribute declared without a domain takes its nearest ancestor unit (#1244). The parser stores . for such an attribute, and . names the parent — so an attribute grouped in a container inside a unit (meta, gebruiksdoelen, woningtype in the BAG toolkit) failed with Cannot find Domain unit ., and every attribute in such a group had to spell (..), or the group had to carry a unit of its own. The search now continues upward to the first ancestor that is a unit. Nothing that worked changes: an attribute whose parent is a unit still gets that parent, and a container was never a domain.
  • iif, union, union_data and lookup keep the geometry composition of their arguments (#1240). cond ? poly_a : poly_b produced an arc whatever its arguments were. One rule serves all four: the first value argument seeds the composition, arguments that agree keep it, and arguments that genuinely disagree are an error. The composition deprecation names the calculation rule that produced it, and its advice covers multipoint (#1243).
  • The XML configuration notation reads back what the writer emits (#1261). A reader that had not been exercised in years asserted on malformed input before it threw, dropped a function, a value array and the layout of element text, and disagreed between != and ==. It now reports malformed input, and a round-trip battery over every testcases configuration keeps it that way. An entity reference stops on its ; so the next character survives (#1260), and an included .xml fragment pops back to the item it was included into (#1254).
  • A configuration dump is self-contained (#1253), writes one line end per line end (#1251), dumps a signature-typed function parameter as the source wrote it (#1252), carries no digit grouping in a range (#1256), and reads only what is resolved, resolving beforehand, so it computes nothing while it writes (#1268).
  • Every path that resolves a name reports a case mix-up (#1262), not only the path that creates the token — and the engine no longer reports such mix-ups about its own names (#1161): the regression battery went from 2747 case warnings, ten of every eleven the engine disagreeing with itself, to 40. unit, value, item, param, attr and nrofrows are canonically lower case.
  • union_data warns when its arguments do not match the parts of its result domain (#990), the legacy brace range spelling is caught on either bound (#1165), table_spec and stable sort_index helpers (#421), editor navigation into a template instantiation (#471), a worker thread's internal error names the item it was working on (#1202), the referred-item chain no longer yields shadowed sub-items (#1200), and a configuration can decide whether its hidden items are shown (#694).
  • GeoDmsRun exits 1 when the item of a @statistics or @valueinfo request fails to compute (#1270), as the exit-code table on the wiki promises; it used to log the failure and exit 0.

Packaging and build

  • The .g (GLOBIO) flavour, built from the same solution with a committed GeoDmsGlobio switch against the older GDAL/GEOS stack. Install it only where that stack is required; .m remains the ordinary Windows build. It is gated like the others (#1231): the unit suite, the testcases battery and the shipped-content test all pass before the setup script installs.
  • The installer ships examples\testcases\data (#1271), the fixtures seventeen of the shipped battery's cases read; both NSIS scripts packaged the folder with a flat *.* that did not descend. In this release the .g and .l setups carry the folder; the .m and .c setups were packaged minutes before that fix landed and still lack it, so under an installed .m or .c those seventeen cases fail with File Open Error on ..\data\<fixture>. The engine itself is identical across all four. A deleted resource stops shipping (#1031)CopyResources mirrors the shipped trees instead of accumulating stale copies.
  • The VS Code extension ships with the setup, with editor language definitions generated from the operator registry. A GUI test script no longer aborts on a byte outside ASCII (#1239). Multi-version Python bindings (#1105).
  • A failing Debug-build assertion reports itself and stops (#1265) — writes the assertion to stderr and every open log, then exits 3 — instead of opening a modal dialog that pumps the engine while it waits. Debug builds only.
  • Every text blob in the repository is stored with LF. Source only; nothing about the shipped binaries changes.