Skip to content

0.7.0

Choose a tag to compare

@mapsmith-ai mapsmith-ai released this 26 Sep 14:20
· 124 commits to main since this release

This release changes answers earlier releases gave, and that comes first.
From 0.4.0 to 0.6.1 every position MapSmith read from a point-registered raster
(AREA_OR_POINT=Point — the USGS elevation products and the Copernicus DEM)
was half a cell north-west of the sample: GDAL already corrects for the tag, and
MapSmith corrected a second time. The same premise sat in the correctness suite
MapSmith is graded by, which scored the wrong answer as a pass; it has published
an erratum.
Terrain operations now accept point-registered DEMs, which they refused
outright. Two capabilities arrive on existing tools — weighted zonal statistics
and the gradient along a line — with a Copernicus notebook. No tool was added.
Two records change shape, both below: elevation_profile across two CRSs uses
the specification's source_crs/target_crs, and terrain operations record
the raster registration in crs_decisions.

Records declare spec_version 1.0.0-draft.8, as in 0.6.1.

Added

  • A Copernicus notebook. examples/04_copernicus_terrain_vegetation.ipynb,
    on Monte Baldo: a Copernicus DEM in degrees refused for slope and reprojected,
    a Sentinel-2 NDVI, and the treeline read off both. Two clips ship in
    examples/fixtures/ under their own licences, with the script that rebuilds
    them. It found both fixes above, and one fact about the data: for the item it
    uses, the Sentinel-2 catalogue declares a reflectance offset of -0.1 that the
    pixels already contain — deep water would read -7% — so the clip declares the
    scale alone, and the script re-checks it every time it runs.

  • Weighted zonal statistics. zonal_statistics takes weights_path, a
    raster of weights, and weighted_mean, weighted_sum, weighted_stdev,
    weighted_variance and weighted_frac: the mean heat of a district weighted
    by the population living in each cell, which used to take three operations
    and a division and is a different number from the plain mean wherever people
    are not spread evenly. Both rules were measured on the engine first.

    One is in the record: a cell with a value and no weight -- nodata or NaN --
    counts with weight 0, because exactextract's own default turns the zone's
    weighted statistics into NaN for a single missing weight. The record carries
    weight_of_a_cell_with_no_weight: 0.0 and the weights by name
    (weights_path) as well as in inputs[], and a non-critical check,
    x-mapsmith:every_valued_cell_has_a_weight, names the zones where it
    happened.

    The other is a refusal, so a record exists only when it held: the weights
    must sit on the value raster's own grid -- same CRS, cells (compared relative
    to the cell size) and registration -- because exactextract accepts a coarser
    grid by repeating each coarse cell whole in every fine one, which inflates a
    weighted sum of counts, and never compares the two coordinate systems. A
    weights raster off the grid, with two bands, or with a negative weight inside
    a zone is refused with how to align it; a negative weight outside every zone
    changes no number and is not.

    With two rasters in one record, each one's georeferencing facts stay with
    it: the top-level environment describes inputs[0], and the weights' are
    on their own entry under x-mapsmith:environment. Records of operations
    that read one raster are unchanged. Reached through the same tool; the tool
    count is unchanged.

  • The gradient along a line. elevation_profile takes grade_base_length
    and, optionally, grade_threshold_percent: every point carries
    grade_percent, the rise over the window of that length starting there, and
    the result gives per line the steepest window and where it starts, the total
    ascent and descent, and the stretches above the threshold. It closes four
    railway requests that each took a plan -- "the steepest section in percent",
    "the maximum gradient over any 100 m", "does any part exceed 2.5%" -- and
    three traps are decided rather than defaulted: this is not slope, which is
    the ground's steepest direction and on a track crossing a hillside at 45
    degrees says 2% where the track climbs 1.41%; the window slides by the
    spacing, so it answers "any 100 m stretch" and not "every 100 m from the
    start"; and the base length has no default and must be a whole number of
    steps. A window never crosses the gap between two parts of a multi-part line
    that do not touch (each point carries run_index), nor a point the raster
    could not read; a stretch is one direction of travel, so a climb and the
    descent after it are two. A line whose unit is not the metre needs
    grade_height_unit, because the rise comes from the raster's values, which
    state no unit, and metres over US survey feet is 3.28 times too steep. A check
    recomputes every stored grade from the profile as written and fails on a
    grade missing where a window fits, with the count in its detail.
    The new search phrasings moved discovery figures the right way: words alone
    under BM25 at 76 entries from 50% to 55%.

Fixed

  • Every position read from a point-registered raster was half a cell off,
    since 0.4.0.
    On a file declaring AREA_OR_POINT=Point MapSmith moved each
    sample half a cell north-west of where GDAL puts it: sampling,
    elevation_profile, line_of_sight, locate_extreme_cell, least_cost_path,
    and the zones of zonal_statistics, which were shifted for the coverage
    computation. The premise was that GDAL reports the tag and leaves the
    geotransform alone. It does the opposite — since RFC 33 the GTiff driver moves
    a PixelIsPoint tie point half a cell on write and on read, so its geotransform
    already centres each cell on its sample — and the correction was made twice.
    Found by a Copernicus DEM, whose samples fall on whole arc-seconds by its own
    documentation: 10.8 by GDAL, 10.79986 by MapSmith. Positions now come from
    GDAL's geotransform whatever the tag says; the tag is still recorded, because
    it says what a value represents. The same premise was in Argleton trap 024,
    which scored MapSmith's wrong answer as correct for 26 days; the suite has
    published an erratum.
    A weights raster is no longer refused for a registration different from the
    values': on the same geotransform the samples are in the same places. And the
    fix no longer depends on the environment: GTIFF_POINT_GEO_IGNORE=TRUE, which
    reverts GDAL to its pre-2010 reading, moved every answer half a cell back with
    nothing in the record to say why, so MapSmith now pins it to FALSE on import,
    whether or not remote paths are allowed — which also overrides an operator's
    setting in a process that imports mapsmith as a library, deliberately.

  • The terrain operations refused every point-registered DEM, the Copernicus
    DEM included.
    The engine reads the stored tie point as a cell corner, and
    the guard that compares its grid with GDAL's caught the half cell and refused,
    so the numbers were never wrong and twelve operations were unusable on the
    most used global DEM (the eleven that write a raster, and contour_lines). The engine now gets a copy declared area-registered on
    GDAL's own geotransform — the same pattern as the TIFF predictor, disclosed in
    the manifest — and the output gets the input's registration back. Their
    manifests record the registration in crs_decisions, as the raster writers
    already did, and say when MapSmith retagged an output after the engine wrote
    it, since those bytes are no longer the engine's alone.
    contour_lines loses a branch for point-registered DEMs that corrected half a
    cell the wrong way and could not be reached.

  • elevation_profile measured the spacing in the DEM's units, and on a DEM
    in degrees a 1000 m line came back as one point.
    The line was reprojected
    to the raster's CRS before its positions were computed, so spacing=100
    meant 100 degrees, and the only trace was a GeoPandas warning on the console.
    Positions are now measured along the line in its own projected CRS -- a
    geographic line was already refused -- and only the sample points go to the
    raster's CRS to be read. The output is now in the line's CRS: when the two
    differed it used to be in the raster's, and a plan step reading it must now
    expect the line's (the plan validator does). The record says what was
    transformed in the specification's own keys: source_crs the line's,
    target_crs the raster's, and the transformation the sample points took to
    be read, while output.crs says where the result is. It used to list the line
    under x-mapsmith:inputs_reprojected from its own CRS, which that key's
    definition makes a contradiction. When line and DEM share a CRS nothing
    changes. Found while building a gradient on top of it.

  • elevation_profile stopped short of the far end of a line. A line whose
    length is not a whole number of steps ended on the last whole step: 15 m at a
    step of 4 ended at 12, and a summit in the last 3 m was not in the profile,
    while the docstring and the name of the test said both ends were included --
    and the test asserted the opposite. The far end is now a last, shorter step,
    and the stepping check requires it.