Repository navigation
0.7.0
This release changes answers earlier releases gave, and that comes first.
From 0.4.0 to 0.6.1 every position MapSmith read from a point-registered raster
(AREA_OR_POINT=Point — the USGS elevation products and the Copernicus DEM)
was half a cell north-west of the sample: GDAL already corrects for the tag, and
MapSmith corrected a second time. The same premise sat in the correctness suite
MapSmith is graded by, which scored the wrong answer as a pass; it has published
an erratum.
Terrain operations now accept point-registered DEMs, which they refused
outright. Two capabilities arrive on existing tools — weighted zonal statistics
and the gradient along a line — with a Copernicus notebook. No tool was added.
Two records change shape, both below: elevation_profile across two CRSs uses
the specification's source_crs/target_crs, and terrain operations record
the raster registration in crs_decisions.
Records declare spec_version 1.0.0-draft.8, as in 0.6.1.
Added
-
A Copernicus notebook.
examples/04_copernicus_terrain_vegetation.ipynb,
on Monte Baldo: a Copernicus DEM in degrees refused for slope and reprojected,
a Sentinel-2 NDVI, and the treeline read off both. Two clips ship in
examples/fixtures/under their own licences, with the script that rebuilds
them. It found both fixes above, and one fact about the data: for the item it
uses, the Sentinel-2 catalogue declares a reflectance offset of -0.1 that the
pixels already contain — deep water would read -7% — so the clip declares the
scale alone, and the script re-checks it every time it runs. -
Weighted zonal statistics.
zonal_statisticstakesweights_path, a
raster of weights, andweighted_mean,weighted_sum,weighted_stdev,
weighted_varianceandweighted_frac: the mean heat of a district weighted
by the population living in each cell, which used to take three operations
and a division and is a different number from the plain mean wherever people
are not spread evenly. Both rules were measured on the engine first.One is in the record: a cell with a value and no weight -- nodata or NaN --
counts with weight 0, because exactextract's own default turns the zone's
weighted statistics into NaN for a single missing weight. The record carries
weight_of_a_cell_with_no_weight: 0.0and the weights by name
(weights_path) as well as ininputs[], and a non-critical check,
x-mapsmith:every_valued_cell_has_a_weight, names the zones where it
happened.The other is a refusal, so a record exists only when it held: the weights
must sit on the value raster's own grid -- same CRS, cells (compared relative
to the cell size) and registration -- because exactextract accepts a coarser
grid by repeating each coarse cell whole in every fine one, which inflates a
weighted sum of counts, and never compares the two coordinate systems. A
weights raster off the grid, with two bands, or with a negative weight inside
a zone is refused with how to align it; a negative weight outside every zone
changes no number and is not.With two rasters in one record, each one's georeferencing facts stay with
it: the top-levelenvironmentdescribesinputs[0], and the weights' are
on their own entry underx-mapsmith:environment. Records of operations
that read one raster are unchanged. Reached through the same tool; the tool
count is unchanged. -
The gradient along a line.
elevation_profiletakesgrade_base_length
and, optionally,grade_threshold_percent: every point carries
grade_percent, the rise over the window of that length starting there, and
the result gives per line the steepest window and where it starts, the total
ascent and descent, and the stretches above the threshold. It closes four
railway requests that each took a plan -- "the steepest section in percent",
"the maximum gradient over any 100 m", "does any part exceed 2.5%" -- and
three traps are decided rather than defaulted: this is notslope, which is
the ground's steepest direction and on a track crossing a hillside at 45
degrees says 2% where the track climbs 1.41%; the window slides by the
spacing, so it answers "any 100 m stretch" and not "every 100 m from the
start"; and the base length has no default and must be a whole number of
steps. A window never crosses the gap between two parts of a multi-part line
that do not touch (each point carriesrun_index), nor a point the raster
could not read; a stretch is one direction of travel, so a climb and the
descent after it are two. A line whose unit is not the metre needs
grade_height_unit, because the rise comes from the raster's values, which
state no unit, and metres over US survey feet is 3.28 times too steep. A check
recomputes every stored grade from the profile as written and fails on a
grade missing where a window fits, with the count in its detail.
The new search phrasings moved discovery figures the right way: words alone
under BM25 at 76 entries from 50% to 55%.
Fixed
-
Every position read from a point-registered raster was half a cell off,
since 0.4.0. On a file declaringAREA_OR_POINT=PointMapSmith moved each
sample half a cell north-west of where GDAL puts it: sampling,
elevation_profile,line_of_sight,locate_extreme_cell,least_cost_path,
and the zones ofzonal_statistics, which were shifted for the coverage
computation. The premise was that GDAL reports the tag and leaves the
geotransform alone. It does the opposite — since RFC 33 the GTiff driver moves
a PixelIsPoint tie point half a cell on write and on read, so its geotransform
already centres each cell on its sample — and the correction was made twice.
Found by a Copernicus DEM, whose samples fall on whole arc-seconds by its own
documentation: 10.8 by GDAL, 10.79986 by MapSmith. Positions now come from
GDAL's geotransform whatever the tag says; the tag is still recorded, because
it says what a value represents. The same premise was in Argleton trap 024,
which scored MapSmith's wrong answer as correct for 26 days; the suite has
published an erratum.
A weights raster is no longer refused for a registration different from the
values': on the same geotransform the samples are in the same places. And the
fix no longer depends on the environment:GTIFF_POINT_GEO_IGNORE=TRUE, which
reverts GDAL to its pre-2010 reading, moved every answer half a cell back with
nothing in the record to say why, so MapSmith now pins it toFALSEon import,
whether or not remote paths are allowed — which also overrides an operator's
setting in a process that importsmapsmithas a library, deliberately. -
The terrain operations refused every point-registered DEM, the Copernicus
DEM included. The engine reads the stored tie point as a cell corner, and
the guard that compares its grid with GDAL's caught the half cell and refused,
so the numbers were never wrong and twelve operations were unusable on the
most used global DEM (the eleven that write a raster, andcontour_lines). The engine now gets a copy declared area-registered on
GDAL's own geotransform — the same pattern as the TIFF predictor, disclosed in
the manifest — and the output gets the input's registration back. Their
manifests record the registration incrs_decisions, as the raster writers
already did, and say when MapSmith retagged an output after the engine wrote
it, since those bytes are no longer the engine's alone.
contour_linesloses a branch for point-registered DEMs that corrected half a
cell the wrong way and could not be reached. -
elevation_profilemeasured the spacing in the DEM's units, and on a DEM
in degrees a 1000 m line came back as one point. The line was reprojected
to the raster's CRS before its positions were computed, sospacing=100
meant 100 degrees, and the only trace was a GeoPandas warning on the console.
Positions are now measured along the line in its own projected CRS -- a
geographic line was already refused -- and only the sample points go to the
raster's CRS to be read. The output is now in the line's CRS: when the two
differed it used to be in the raster's, and a plan step reading it must now
expect the line's (the plan validator does). The record says what was
transformed in the specification's own keys:source_crsthe line's,
target_crsthe raster's, and thetransformationthe sample points took to
be read, whileoutput.crssays where the result is. It used to list the line
underx-mapsmith:inputs_reprojectedfrom its own CRS, which that key's
definition makes a contradiction. When line and DEM share a CRS nothing
changes. Found while building a gradient on top of it. -
elevation_profilestopped short of the far end of a line. A line whose
length is not a whole number of steps ended on the last whole step: 15 m at a
step of 4 ended at 12, and a summit in the last 3 m was not in the profile,
while the docstring and the name of the test said both ends were included --
and the test asserted the opposite. The far end is now a last, shorter step,
and the stepping check requires it.