Repository navigation
Releases
MapSmith 0.12.1
Records continue to declare spec_version 1.0.0-draft.10; no manifest key is new or renamed in this release.
Fixed
- Compare every declared geometry CRS semantically in
calculate_field. Primary and secondary GeoParquet CRS declarations, including native GeoArrow declarations, must match the input. Equivalent PROJJSON serializations remain accepted, while an actual CRS change fails verification and retains the manifest. - Correct the public showcases. Include 0.12.x in the draft.10 range, describe Argleton's current multi-step scope without calling every trap single-step, and correct the current trap count.
MapSmith 0.12.0
Records continue to declare spec_version 1.0.0-draft.10; no manifest key is new or renamed in this release.
Added
- Restricted, deterministic field calculation.
calculate_fieldcreates or replaces one numeric or boolean attribute throughrun_operation, with arithmetic, comparisons, boolean logic,CASE,COALESCEandNULLIF. DuckDB's parsed expression tree is validated fail-closed; geometries, all other attributes and CRS stay unchanged, and non-finite calculated values are refused with the provenance record retained.
Fixed
- Declare the workspace's SELECT-only behavior in the
run_sqlcatalog summary. - Correct the multi-step method-choice benchmark. Expose output cardinality and geometry constraints, permit independent repeated-operation branches, separate unsupported requests from method-choice scoring, and accept minimal defensible sequences. Unsupported outcomes now have exhaustive, explicitly disclosed classes.
- Clarify preview and unsupported planning contracts. The
preview_mapsummary declares EPSG:4326 without changing its discovery terms, and the planner refuses read-only approximations of writes, per-feature iteration and multi-engine requests.
MapSmith 0.11.0
Records continue to declare spec_version 1.0.0-draft.10; no manifest key is
new or renamed in this release.
Added
- An executable agentic pilot and a measured fine-tuning decision.
examples/agentic_pilot.pydemonstrates typed planning, a validated brief,
QGIS execution, overlays and a closed-form 1,300 m² answer with its plan
manifest. The current mixed method-choice baseline does not justify
proprietary-model fine-tuning; revisit only after a complete baseline shows a
real advantage and explicit cost. - QGIS Processing through a GPL-isolated subprocess, starting with
native:buffer.qgis_processingexchanges JSON withqgis_process, never
imports QGIS, pins buffer semantics and returns a MapSmith-written,
MapSmith-verified output with the actual QGIS version in the manifest.
Distances remain explicitly in the input CRS's units. ArcGIS Pro and QGIS
integrations are exercised locally, with engine identity recorded in provenance. - Vector/raster conversion chosen from measured gaps in public training
exercises.rasterize_layerburns geometries or a numeric field onto a
reference raster's exact grid, refusing CRS mismatches and invalid field
values.polygonize_rasterexcludes nodata, supports declared 4/8
connectivity, bounds feature counts, emits stable row order and records
mechanical geometry repairs. Both are available throughrun_operation, with
catalog contracts, provenance and closed-form round-trip tests. - Bounded, protocol-level agent repair for verification failures (issue #26).
execute_planreturns structured checks with stable names, arguments,
observed/expected values and hints. An agent may submit a corrected plan with
the previous plan manifest, up to three attempts total; every attempt is
linked on disk, reusing an earlier output is refused to preserve provenance,
and a fourth attempt is refused before the engine runs. MapSmith does not
choose repairs or execute arbitrary repair scripts. - Deterministic project briefs derived only from validated plans (D-063).
brief_planrefuses invalid plans before producing project text; every brief
statement carries an operation, parameter, validation, engine or installation
source. Sixteen executable plan fixtures verify rendering, execution,
plan-level manifests and closed-form results on real vector and raster engines. - A multi-step method-choice benchmark with 20 public cases and no change to
Argleton's meaning.benchmarks/method_choice.pymeasures missing and
reordered operations, wrong tools, output contracts, required assumptions,
planning-silent errors and reproducibility, with defensible alternatives and
separate calibration/evaluation splits (10 cases each). Cases come from
paraphrased exercises in the QGIS Training Manual (GNU FDL 1.3,
docs.qgis.org); repeated-tool exercises that need SQL arguments are outside
this first method-only bench. API failures, invalid responses, stale prompts
and unattempted cases have separate counts rather than becoming wrong plans. - Machine-checkable argument contracts in the catalog and in plan validation
(issue #27). Choices, numeric bounds, cardinality, extensions and
cross-parameter rules are declared once incatalog.py, exposed by
list_operations(detail=true)and enforced before execution with
INVALID_ARGUMENT_VALUEmessages naming the rule and the offending value.
resample_rasteralso accepts warp-only methods that its runtime already
used,convert_formatrefuses unsupported extensions statically, and
spatial_joinrefuses unknown engine names before routing.
Security
- PROJ resource resolution is confined at startup, and SQL replacement scans
cannot hide dataset paths through CTE scopes. pyproj, rasterio and pyogrio
receive explicit absolute search paths without discarding grid-only
directories; the process working directory then moves inside the workspace so
DuckDB's PROJ cannot use the old cwd for explicit or EPSG-selected grids.
DuckDB has no search-path setter, sorun_sqlalso refuses explicitly named
external grids. SQL path analysis now follows CTE visibility by scope and by
recursive branch, including replacement scans without a file extension. - SQLite SRS catalogues are bounded and read case-insensitively. A catalogue
is limited to 20,000 rows, 512 referenced systems, 64 KiB per definition and
16 MiB overall. Reference views disable selective parsing, numeric text ids
follow SQLite affinity, and every row gets the textual path check. Legacy
SpatiaLitesrs_wktis read alongsidesrtextandproj4text. - JSON, text, XML, sidecar and ZIP-member parsing is bounded before whole-file
reads. Inputs over 32 MiB are refused under a workspace instead of being
materialised and parsed to find a coordinate system; a safe-looking prefix is
never accepted..prj,.aux.xmlandSRSNametext is bounded to 64 KiB,
with oversized declarations refused in both modes. - Only CRS refusals, never permissions, are cached by file metadata. An
allowed input is checked again, so replacing its bytes while preserving size
and modification time cannot reuse a stale permission. Per-text CRS verdicts
remain cached. - Audit sidecars stay inside the workspace and cannot replace one another.
Provenance, failure and plan manifests reject external symbolic links and are
replaced atomically. Plan validation and bounded repair reserve every output
together with its three audit sidecars before an engine runs, so a step cannot
overwrite another step's dataset or audit trail. Lineage reads use the same
containment rule.
Fixed
- An ordinary SpatiaLite EPSG catalogue no longer receives thousands of full
PROJ parses per input. Only coordinate systems referenced by layers are
parsed fully; an ordinary 5,804-row catalogue needs one in the regression
fixture. Per-text CRS verdicts and refusal caching remain in place. - A native Parquet CRS no longer becomes OGC:CRS84 merely because
geo.crs
is absent. Readers and CRS probes now use a native declaration when one
exists; an unresolved declaration stays unknown with its reason, while no
declaration at all keeps the GeoParquet default. Explicitgeo.crsvalues,
including null, retain their meaning on primary and secondary columns alike,
and multi-geometry files arbitrate byprimary_columnrather than the first
geometry in schema order. - The agent-facing discovery guidance no longer quotes a stale 51-operation
table.list_operationsnow explains the delivery mechanism without
duplicating measurements maintained in the published benchmark. Unused CRS
and scalar-transform helpers were removed after moving their remaining test
references onto the real public paths. - Rasterizing disjoint or empty geometries no longer returns silent success.
The background raster is still written, but a non-critical warning in the
result and provenance says that no cells contain an input geometry. - Malformed repair manifests and unknown engines fail structurally. Invalid
previous-plan sections produceINVALID_REPAIR_CHAINinstead of Python
attribute errors, and an unknown spatial-join engine no longer generates an
install command for an invented package extra.
0.10.1
Update if you run MapSmith under MAPSMITH_WORKSPACE. A dataset in a
format whose coordinate system MapSmith did not read itself -- GeoJSON with a
crs member, GML, FlatGeobuf and others -- could make PROJ open a file outside
the workspace, or a network share, before MapSmith refused it
(GHSA-36xg-cr27-6r4m).
This is the part 0.10.0's SECURITY.md and GHSA-3wvg-2qh8-8gg6 listed as still
open. Under a workspace, netCDF, HDF, JPEG 2000, ERDAS IMG and GRIB rasters are
now refused until MapSmith reads their coordinate system itself; without a
workspace nothing changes. Records still declare spec_version
1.0.0-draft.10, no key is new or renamed, and the answers are those of 0.10.0.
Security
- Under a workspace, the coordinate system of every dataset format MapSmith
accepts is read before GDAL opens the file, and a format it does not
recognise is refused. 0.10.0 read the declared SRS of GeoPackage,
shapefile and GeoParquet only; for the other formats GDAL read it first, and
PROJ opened any grid file it named, before MapSmith refused the file
(SECURITY.mdlisted this as open). The path gate now reads it from the
bytes for GeoJSON and Esri JSON (with a JSON parser, so an escaped key is
seen), GML and its.gfs, FlatGeobuf, GeoTIFF's GeoKey text, file
geodatabases, folders of shapefiles, zips of these, and a.prjbeside any
file. UnderMAPSMITH_WORKSPACEa format it does not recognise -- an ERDAS
.img, a netCDF, a JPEG 2000 -- is refused with the list of the ones it
does; without a workspace it is opened as before.run_sql'sST_Read
accepts every recognised format, where 0.10.0 accepted three. The verdict is
cached by file name, size and modification time, so one input is no longer
parsed by the gate, the probe and the reader in turn.
0.10.0
Update if you run MapSmith under MAPSMITH_WORKSPACE. A coordinate
reference system -- a tool argument, or the one a file declares -- could make
PROJ read a file outside the workspace or open a network share, and in
run_sql DuckDB's own PROJ could do the same, in every release since the
workspace and the refusal of network shares arrived (0.2.0). Both are closed
here (Security below, and SECURITY.md), and both are in one advisory,
GHSA-3wvg-2qh8-8gg6.
Records still declare spec_version 1.0.0-draft.10, and no key is renamed.
Two keys of MapSmith's own are new inside crs_decisions.transformation:
x-mapsmith:shift_from_crs_definition, when the datum shift is written into
the CRS itself (+towgs84), and x-mapsmith:default_was_ballpark, when the
engine's own choice would have applied no shift. Results move where an
operation that brings a vector input onto another CRS now chooses the datum
transformation as reproject_layer does (Monte Mario from Rome: up to about
80 m); nearest_neighbour_index refuses a feature with no position. Under a
workspace, run_sql runs only SELECT and takes coordinate systems and
dataset paths as literals.
Security
- A coordinate reference system that reaches MapSmith's own code can no
longer make PROJ read a file outside the workspace or contact a host.+nadgrids=,+geoidgrids=and the WKT
and PROJJSON spellings of a parameter file name grid files by path,+init=
names a file to parse, and a CRS reaches PROJ as a tool argument
(target_crs) and from the metadata of the files an agent reads. Under a
workspace a path outside it was read and its content moved the output; a
//host/sharegrid was opened over SMB; with PROJ networking on, an
http://grid made requests to any host (measured 2026-10-23, present since
reproject_layerexists). A CRS policy now refuses, wherever a CRS enters --
tool arguments, the CRS a file declares, every reader, the datum module, and
the admission of every input path, before GDAL probes the file -- any grid,
init or model name that is not a bare name from PROJ's data directories
(@nullallowed), any transformation written as a PROJ string, and any
declared SRS PROJ cannot read; the declared SRS is read as text from the
file's own catalogue (GeoPackage by its bytes, shapefile.prj, GeoParquet
geo), and the server switches PROJ networking off under a workspace at
startup, for pyproj's PROJ and GDAL's.SECURITY.mdrecords it, and names
what stays open: datasets in other GDAL formats, which GDAL opens first. run_sqlunder a workspace can no longer make DuckDB's own PROJ read a
file outside it. DuckDB's spatial extension carries a PROJ of its own,
which opens files natively and never met the connection sandbox:
ST_Transform(geom, 'EPSG:4326', '+init=<a file outside>:1')read that file
-- the output moved by the false easting written in it -- and so did
ST_SetCRSfollowed by a two-argumentST_Transform, whileread_texton
the same path was refused (measured 2026-10-25; every release with the
sandboxed connection, 0.2.0 to 0.9.0). Under a workspace the statement is
now read with DuckDB's own parser (json_serialize_sql) before it runs:
every CRS inST_Transform,ST_SetCRSor aGEOMETRY('…')type must be a
literal the CRS policy accepts, every path given toST_Read,
ST_Read_Meta,read_parquetor named inFROMmust be a literal the path
gate accepts (globs are expanded by the sandboxed connection and each match
is checked), what GDAL would open must be a GeoPackage, a shapefile's.shp
or a GeoParquet, recognised by its bytes -- FlatGeobuf, GeoJSON and the other
formats whose SRS GDAL reads before MapSmith can are refused inrun_sql--
and the rest is refused with how to write it: statements other than
SELECT(passoutput_pathto keep a result), SQL held in a string
(query()), GDAL open options, and a CRS or a path computed in SQL. Without
a workspace nothing changes: file access there is unconfined by design.- The path gate reads the CRS a Parquet file declares in the column's
nativeGEOMETRY/GEOGRAPHYtype, not only in thegeokey. A file
with the CRS there and nogeokey at all passed the gate with nothing to
check, and DuckDB read that CRS and handed it to its PROJ: through
run_sql, a grid outside the workspace was read (found by the audit of the
change above, before release). Theprojjson:<key>andsrid:<n>forms
are followed too. Every tool's input admission goes through the same gate.
Added
- 103 phrasings on 36 operations, taken from how the QGIS Training Manual
poses its exercises. Each is a paraphrase of a problem a lesson sets ("strip
along both banks of a stream", "distance along the roads, not as the crow
flies"), written from half of the 220 exercises extracted from that manual
(GNU FDL 1.3, docs.qgis.org) and measured on the other half, which the
phrasings never saw: BM25 found@3 on words alone 47% -> 59%, with the
caller's facts 40% -> 51%. On the 118 independent requests of the published
table every level moved up (words alone 29% -> 30%, fullest declaration
60% -> 62%; the embedding engine 18% -> 19% and 56% -> 62%), and the README,
docs/catalog-entry-spec.md, the site and the tool instructions an agent
reads say the new figures. The ranking code did not change; the order
list_operationsreturns for a given request can.
Fixed
run_sqlcan no longer crash the server withST_Readof a layer that
is not there. DuckDB's spatial extension (1.5.5) ends the process with a
segmentation fault onST_Read('roads.gpkg', layer := 'nope'), and the
MCP server with it: one line of agent-written SQL was a denial of service.
The defect is upstream (reproduced on a bare connection); MapSmith now
checks the layer, in either spelling (:=or=), against the names
ST_Read_Metalists and refuses a missing one with the list, in every
mode. A statement other thanSELECTcannot be checked that way, so one
that namesST_Readand a layer is refused with the advice to write the
SELECTand passoutput_path; that matters without a workspace, where
such statements run (CREATE TABLE … ASstill crashed after the first
version of this fix, found by the pre-release review). Found by the audit
of therun_sqlchange above.- A datum shift written into the CRS itself is no longer recorded as a
ballpark. For a CRS defined with+towgs84(or WKTTOWGS84[]) PROJ
builds a Helmert from those parameters and applies it in full, but reports
accuracy -1 for it -- the value it also reports for the ballpark -- and
everycrs_decisions.transformationread that value as "no datum shift
was applied":reproject_layer's since 0.3.0, every other operation's
since 0.5.0. Measured on Monte Mario with the seven IGM
parameters: the pipeline carried+proj=helmertand moved the data 79 m,
and the record saidis_ballpark: true. The verdict now comes from PROJ's
own name for the operation it used ("Ballpark geographic offset" is its
word, and a concatenation carries every step's) and, failing a name, from
the pipeline it ran: a shift step is a shift. The name of a shift a CRS
declares itself is written by whoever wrote the file, so it is cut out of
the description before the word is looked for: a Helmert called "Ballpark
geocentric translation" is still a shift (security review, same day). A CRS that carries its own
shift is also no longer rebuilt from a registry code on the way in, which
dropped the+towgs84. Such a record saysis_ballpark: false,accuracy_m: null(nobody has
stated one) and carriesx-mapsmith:shift_from_crs_definition: trueto say
why the two go together;docs/manifest-vocabulary.mdhas the entry. Data
and coordinates are unchanged -- only the record was wrong, in the direction
that hides a shift. Found by the pre-release review of 0.9.0. - Every operation that brings a vector input onto another CRS chooses the
datum transformation the wayreproject_layerdoes.reproject_layerhas chosen its transformation since 0.5.0 (datum.best_operation:
a published operation that covers the data, or a datum's Greenwich twin), but
the eight vector operations that bring a second input onto the first one's CRS
--clip_layer,overlay_layers,nearest_join,merge_layers,spatial_join,
count_in_polygons,summarize_points_in_polygons,apportion_by_area-- let
PROJ pick, throughto_crs. On a Monte Mario (Rome, EPSG:4806) layer PROJ's
pick is the ballpark, so a parcel sat 79.5 m apart inclip_layerand in
reproject_layer, each record true of its own run. The eight now go through
the same chooser and apply it themselves, andcrs_decisions.transformation
describes what was chosen, withx-mapsmith:default_was_ballparkwhere the
engine's own choice would have applied no shift. The twin route also serves a
target counted from another meridian (a WGS 84 mask onto a Rome layer), which
it did not. Operations that compute in an estimated UTM zone and come back are
unchanged: the way out and the way back largely cancel. The coordinates of the eight
change only where PROJ's default is the ballpark and a covering operation or a
twin exists; everywhere else the chosen transformation is PROJ's own. The
same chooser now serves every operation that brings a vector input onto
another CRS:snap_layer,line_intersections,transform_by_control_points,
compare_layers,nearest_neighbour_index, the points of
sample_raster_at_points,elevation_profileandleast_cost_path, the
zones and mask ofzonal_statisticsandclip_raster, and the pour points
and stations ofwatershedandviewshed. What still lets ...
MapSmith 0.9.0
Records still declare spec_version 1.0.0-draft.10, and no key is renamed.
Three keys of MapSmith's own are new, each declared with the reason it is not
a synonym of anything the specification has: x-mapsmith:areas_measured_on in
crs_decisions, from apportion_by_area, and x-mapsmith:ballpark_share and
x-mapsmith:outside_area_of_use inside crs_decisions.transformation. Three
check names are new, all from apportion_by_area:
x-mapsmith:extensive_total_preserved, x-mapsmith:every_source_value_placed
and x-mapsmith:no_zone_gives_more_than_it_holds. What
reproject_layer writes changes in two cases, both described under Fixed:
coordinates outside every published operation's area are no longer moved by
an operation from elsewhere, and a geographic CRS counted from another
meridian whose operations are published only for its Greenwich twin
(EPSG:4806, Monte Mario from Rome) goes through that twin. Shapely 2.1 and
pyarrow 23.0.1 or later are required.
Added
apportion_by_area: move counts and rates from the zones they are reported
in to zones that cross them. Cases by municipality to health districts,
census-tract population to school catchments. Each column is declared
extensive (a count, split by share of area) or intensive (a rate, averaged by
area); a call declaring neither is refused, because splitting a rate or
averaging a count gives a plausible wrong number. Each count comes with
_minand_maxthat hold however it is distributed inside the source
zones; rate bounds rest on the same uniformity the estimate assumes, and the
record says which is which. Zones are cut in the source zones' own CRS,
where a boundary both layers share is the same line, and a zone counts as
wholly inside a target by a topological test, not by a ratio of areas.
Shares are ratios of areas measured on an equal-area surface -- the source
CRS when it is one, otherwise a plane centred on the zones on their own
datum, named in the record -- so the pieces of each zone add up to it.
Whether targets overlap is decided zone by zone, so a few square metres of
overlapping boundaries are said in the record where they put a zone's value
in two targets, and a check fails if a zone with no overlap gives out more
than it holds. Invalid zones are repaired and the repair recorded; a zone
the repair leaves with no area or with a coordinate that is not a number,
zones spread over more than a hemisphere, layers writing longitude in two
conventions, and a target that the source CRS would tear at one of its
seams (the 180th meridian, a pole: caught by area and by position, whatever
the units) are refused; targets keep their own CRS; and the counts written
are checked against the pieces they came from.
Fixed
crs_decisions.transformationnow describes what PROJ did to the data's
own coordinates. Which operation PROJ selects depends on where a
coordinate is, and the record asked at one point chosen from the source CRS
alone -- for EPSG:4326, the middle of the world, in the Gulf of Guinea. It
was wrong in both directions: WGS 84 data in Europe moved to EPSG:3035, or in
England to EPSG:27700, was recorded as a ballpark (is_ballpark: true) where
PROJ applied a published 1 m or 2 m shift; and NAD27 coordinates in Italy were
recorded as a 7 m shift where PROJ applied none -- the error a ballpark record
exists to report, hidden by it. Every operation that moves an input now hands
over the moved layer, PROJ is asked what it did at a spread of the data's own
points, inside and outside the areas of use of the operations it has -- outside
them it applies no shift for some pairs (NAD27) and stretches a published
operation for others (DHDN, the Swiss grid), so the answer is PROJ's and not
inferred. A layer straddling the edge of a datum grid is recorded as a
ballpark withx-mapsmith:ballpark_sharesaying how many of its vertices got
no shift;x-mapsmith:outside_area_of_usesays how many lie outside the area
of every operation published for the pair, whatever PROJ then did with them;
a move of no coordinates at all records that nothing was transformed; and a
check with no data to hand asks where the two CRSs' areas of use overlap.reproject_layerno longer applies an operation outside its area of use,
and its output changes where it did. When PROJ's own choice at its probe
point was a ballpark it substituted the first published operation for the
pair, wherever the data was: WGS 84 data in Madrid moved into EPSG:3003 by
the Italian operation came out 84.5 m from PROJ's own result, and WGS 84
data in Italy moved into EPSG:27700 by the British one, 248 m. Now it
substitutes only an operation whose area of use contains every vertex, and
otherwise leaves each point where PROJ does. A geographic CRS counted from
another meridian whose operations are published only for its Greenwich twin
goes through that twin -- EPSG:4806 (Monte Mario, Rome) and EPSG:4819 (Nord
Sahara 1959, Paris) among the 23 tried; NTF Paris, EPSG:4807, has an
operation of its own and does not change. A station in Piedmont on EPSG:4806
now lands on the published 4 m operation's answer, where it was 6.3 m off.better_available_mnames only an operation that covers the data and is
not installed. It reported any published operation for the pair: EPSG:4806
data in Piedmont was told a 44 m operation needed its grid installed, when
that operation needs none and is valid in Sardinia only.- A record written on a failing path no longer claims an earlier run's
output. With a dataset already at the output path, a run refused by its
preconditions, or one that crashed before writing, recorded the digest of
the earlier bytes as its own output -- and a lineage walk by digest then
attributed them to the inputs of the run that failed -- and its record
replaced theirs, leaving them with none. Such a record now omitsoutput
and says why, and where a record on disk still describes the bytes at the
path it is left in place and the failing run's record is written beside it
as<output>.failed.provenance.json, which the error names. Bytes the
failing run did write keep their digest, including when a check of the
output crashes after the write.
Changed
- Shapely 2.1 or later is required (was 2.0):
apportion_by_areauses its
maximum inscribed circle to check that a reprojected target still holds its
own interior. - pyarrow 23.0.1 or later is required (was 21), the release fixing
CVE-2026-25087, a use-after-free reading Arrow IPC files; MapSmith reads none,
and the floor keeps an install at the minimum from carrying it. pytest 9.0.3
or later for the test extra.
MapSmith 0.8.0
Records declare spec_version 1.0.0-draft.10. draft.9 defines the digest
of a dataset made of several files, which is what the first fix below
implements; draft.10 gives each input a role and a configuration of its own.
One key moves: a second input's own georeferencing facts,
inputs[].x-mapsmith:environment on 0.7.x, are now inputs[].environment.
Added
-
cluster_points_by_distance: group points chained together within a
distance. "Group everything within half a kilometre" is single linkage, not
thinning: two points share a cluster when a chain of points joins them with
every link at most the distance long, so chains merge, and every point is kept
with itscluster_idandcluster_size.thin_pointskeeps one point per
crowd, and its count is close to the number of groups but not equal to it. A
check re-reads the written file and re-derives the chains by a second
algorithm. A geographic CRS is refused, and so are lines, polygons and
multipart points, which a point cluster has no single position for. -
list_operationstakes ageometry: what the caller's layer holds, as
point,lineorpolygon, or a list of them for a mixed layer. The
catalogue now has 31 operations taking one vector layer to another, one more
than the size it hands over whole, so a caller declaring everything it knew
got a shortlist back for the most common request there is; with the geometry
declared, the right operation is in the answer for 116 of the 118 benchmark
requests again. An entry declares the families it accepts only where its code
refuses the others (fifteen do), an entry without a declaration is kept for
every family, and a test calls each declared entry with every family it
leaves out and requires the refusal. -
benchmarks/choice_at_scale.py, with every choice it scores in
benchmarks/choice_at_scale.json: each of the 118 requests handed to Claude
Opus 5.5 in sets of 30, 40 and 50 candidates and compared with itself. From
40 it chose right 6.9 points less often than from 30, from 50 5.1 points
less, so the sizelist_operationshands over whole stays 30, and a test
holds it at or below the largest size measured without loss. -
Each input of an operation that reads more than one dataset says which
argument read it (inputs[].argument, specdraft.10). The order of
inputsis not significant, so until now a weighted zonal statistic with
values and weights swapped produced a record that differed only in file
names. Eighteen operations name their inputs (twenty code paths:
spatial_joinhas three engines), with the names the input checks
already use inverification[].argument, so the two can be joined; a
single-input record is unchanged. -
get_lineagetakes alayer. The history of one layer of a file
geodatabase now starts from the geodatabase itself: its record sits beside
the container (spec section 3.1), and the walk matches the layer at every hop
into a container. A layer given for a path that is not a container is
refused rather than ignored.
Changed
get_lineagesteps carryinputs[].argument,inputs[].environmentand
output.layerwhen the records hold them.- A shapefile output whose member files are ambiguous (
out.dbfbeside
out.DBF, possible only on a case-sensitive filesystem) is written with a
manifest that has nooutputdigest and a note saying why, instead of a
digest that would choose one of the two. inputs[].x-mapsmith:environmentisinputs[].environment, the field
specdraft.10defines for the configuration that changed how one input was
read. Same content; only the name moves, and a reader of 0.7.x records reads
the old one.- With more than one input, no file's facts sit on the record's
environment, the first input's included. A sidecar beside the first
raster was recorded there, which the format reads as true of the whole run,
since the order ofinputsmeans nothing: in aclip_rasterit read as true
of the mask. It is now on that input's ownenvironment, and only process
settings stay on the record. Seven operations can meet it (clip_raster,
sample_raster_at_points, elevation_profile, least_cost_path, viewshed,
watershed, zonal_statistics), and only when a sidecar is present. A
single-input record is unchanged.
Fixed
cluster_points_by_distanceis bounded. The pairs of points within the
distance grow with the square of the points when the distance is large, and
6,000 points at a distance of 1e9 took two minutes and 2.3 GB. The pairs are
now found in blocks and the call is refused once they pass
MAPSMITH_MAX_SAMPLES, the operator's limit, before memory is spent.
hot_spotswithweights="distance_band"uses the same query: its weights
were a double loop over every pair of features in Python, quadratic at any
band, and a band that is not a finite positive number is now refused.get_lineageno longer hashes every shapefile in the workspace. Since
the draft.9 digest rule it computed, while indexing, the other rule's digest
of every shapefile record's output, whatever the chain asked about. A
shapefile is now hashed only when a hop misses on it.- A Windows junction inside a
.gdbis not followed. The digest of a
directory container walked into junctions, whichis_symlink()does not
report, so files outside the container could enter its digest. Section 3.3 of
the specification says links are neither members nor followed. The size used
for the re-hash cap now counts the same files as the digest. - A single-layer
.gdbread by any operation records its layer. Every
operation now reads a single-layer file geodatabase, not only
extract_layer, and the input records the layer it read: without it, a
lineage walk meeting the record of that layer stopped with a false "records
claim this container's digest, but for other layers". get_lineagerefuses alayerholding:or a control character. On
Windows:in a file name opens an alternate data stream of another file.thin_pointsrefuses a distance that is not a finite positive number,
likecluster_points_by_distance; andleast_cost_pathaccepts a start or
end given as a single-part MultiPoint, which it used to count as no point.zonal_statisticsrefuses zones that are not polygons. Line zones were
accepted and returned the line's length in pixels as thecount, with a
mean over the cells it crossed: a plausible table, and not a zonal
statistic.count_in_polygonsandsummarize_points_in_polygonsrefuse a
polygon layer holding anything else the same way. Found by the test that
holds the new geometry declarations to the code.least_cost_pathis found by a caller holding points. It reads a cost
raster and two point layers but declared its input as raster only, so a
caller who said, correctly, that it held a vector layer never saw it. A start
or end layer holding a line or a polygon beside its point is now refused; it
used to pass with the extra feature ignored.snap_layersnaps to a polygon or multipart reference. Both crashed with
a rawNotImplementedErrorfrom shapely; every vertex of every part, holes
included, is now a target.- A feature with no geometry is refused, with a count, by
thin_points,
sample_raster_at_points,points_along_linesandline_intersections.
They crashed on it with anAttributeError, "cannot convert float NaN to
integer" and a failed sort. - A shapefile's digest covers its attributes and its CRS. An input or
output shapefile was digested as its.shpalone, so a record kept matching
a dataset whose.dbf(attributes) or.prj(CRS) had been rewritten, while
it saidsha256is of the bytes. The digest is now the listing of the
.shp,.shx,.dbf,.prjand.cpgfiles, as section 3.3 of the
specification defines it since draft.9; a file geodatabase input is digested
the same way, lock files excluded.get_lineagerecomputes digests with the
same function, so a chain written by this release still resolves. A record
written by an earlier release for a shapefile carries the old digest:
get_lineagecompares it under the rule it was written with, so an
unchanged shapefile is not reported as edited, and a chain that crosses
from a record written by 0.7.1 to one written now still resolves. A record
written before this release can only vouch for the.shp: an edit to the
attributes or the CRS made after it was written stays invisible to it, so
such a hop is reported asreverified_shp_only, notreverified, and a
history that contains one is notverified. extract_layerworks on a file geodatabase. The catalog listed.gdb
among its containers, and the input digest opened the path as a file: a
geodatabase is a directory, so the operation died before writing anything.get_lineagematches the layer when a digest belongs to a container.
Two records for two layers of one geodatabase share the container's digest
(spec section 6, draft.9); a hop into a container now resolves to the record
for the layer it names, and stops withother_layerrather than attribute
one layer to the operation that wrote another. It re-checks a layer's record
against the container it sits beside (<container>.<layer>.provenance.json,
spec section 3.1), and two layers of one container reached on two branches
are two histories.- A URI in a manifest path keeps its
//, so its credentials are masked.
The path normaliser collapsedhttps://user:pw@host/xtohttps:/user:pw@…,
and the redaction of URI credentials looks for://: the password would have
stayed in the record. Not reachable while remote paths are refused, which is
the default; found in the 0.7.0 audit. The weights raster of
`zonal_statistic...
0.7.1
Two fixes from the pre-release audit of 0.7.0, both present in earlier releases,
and the Argleton run scored against trap 024's corrected truth, which arrived
after the 0.7.0 tag: that release's PyPI page still quotes the recount of the
old run. No answer 0.7.0 got right changes, and no manifest key moves: records
declare spec_version 1.0.0-draft.8, as in 0.7.0.
Fixed
-
An output that is one of the inputs, or has an extension its writer cannot
honour, is refused before anything is written
(#31).zonal_statistics
withoutput_pathequal to its weights raster replaced the raster with a
directory of shapefiles — a vector writer given.tiffalls back to the
Shapefile driver — and failed before writing a manifest. The dedicated tools
now refuse both at the boundary, asrun_operationalready refused the
collision; and the plan validator turns an extension no writer can honour from
a warning into an error (OUTPUT_EXTENSION_REFUSED), keeping the warning for
writable but non-canonical ones such as.geojson. -
A spacing could ask for any number of points.
elevation_profileand
points_along_linesmaterialised every point before looking at how many
there were: a spacing of1e-6on a 90 m line is ninety million. Both now
count the points from the line lengths and refuse above
MAPSMITH_MAX_SAMPLES(default 1 000 000), an environment variable of the
server that no tool argument can move;SECURITY.mdsays what is and is not
limited.points_along_linesalso refuses a NaN or infinite spacing, which
spacing <= 0let through.
0.7.0
This release changes answers earlier releases gave, and that comes first.
From 0.4.0 to 0.6.1 every position MapSmith read from a point-registered raster
(AREA_OR_POINT=Point — the USGS elevation products and the Copernicus DEM)
was half a cell north-west of the sample: GDAL already corrects for the tag, and
MapSmith corrected a second time. The same premise sat in the correctness suite
MapSmith is graded by, which scored the wrong answer as a pass; it has published
an erratum.
Terrain operations now accept point-registered DEMs, which they refused
outright. Two capabilities arrive on existing tools — weighted zonal statistics
and the gradient along a line — with a Copernicus notebook. No tool was added.
Two records change shape, both below: elevation_profile across two CRSs uses
the specification's source_crs/target_crs, and terrain operations record
the raster registration in crs_decisions.
Records declare spec_version 1.0.0-draft.8, as in 0.6.1.
Added
-
A Copernicus notebook.
examples/04_copernicus_terrain_vegetation.ipynb,
on Monte Baldo: a Copernicus DEM in degrees refused for slope and reprojected,
a Sentinel-2 NDVI, and the treeline read off both. Two clips ship in
examples/fixtures/under their own licences, with the script that rebuilds
them. It found both fixes above, and one fact about the data: for the item it
uses, the Sentinel-2 catalogue declares a reflectance offset of -0.1 that the
pixels already contain — deep water would read -7% — so the clip declares the
scale alone, and the script re-checks it every time it runs. -
Weighted zonal statistics.
zonal_statisticstakesweights_path, a
raster of weights, andweighted_mean,weighted_sum,weighted_stdev,
weighted_varianceandweighted_frac: the mean heat of a district weighted
by the population living in each cell, which used to take three operations
and a division and is a different number from the plain mean wherever people
are not spread evenly. Both rules were measured on the engine first.One is in the record: a cell with a value and no weight -- nodata or NaN --
counts with weight 0, because exactextract's own default turns the zone's
weighted statistics into NaN for a single missing weight. The record carries
weight_of_a_cell_with_no_weight: 0.0and the weights by name
(weights_path) as well as ininputs[], and a non-critical check,
x-mapsmith:every_valued_cell_has_a_weight, names the zones where it
happened.The other is a refusal, so a record exists only when it held: the weights
must sit on the value raster's own grid -- same CRS, cells (compared relative
to the cell size) and registration -- because exactextract accepts a coarser
grid by repeating each coarse cell whole in every fine one, which inflates a
weighted sum of counts, and never compares the two coordinate systems. A
weights raster off the grid, with two bands, or with a negative weight inside
a zone is refused with how to align it; a negative weight outside every zone
changes no number and is not.With two rasters in one record, each one's georeferencing facts stay with
it: the top-levelenvironmentdescribesinputs[0], and the weights' are
on their own entry underx-mapsmith:environment. Records of operations
that read one raster are unchanged. Reached through the same tool; the tool
count is unchanged. -
The gradient along a line.
elevation_profiletakesgrade_base_length
and, optionally,grade_threshold_percent: every point carries
grade_percent, the rise over the window of that length starting there, and
the result gives per line the steepest window and where it starts, the total
ascent and descent, and the stretches above the threshold. It closes four
railway requests that each took a plan -- "the steepest section in percent",
"the maximum gradient over any 100 m", "does any part exceed 2.5%" -- and
three traps are decided rather than defaulted: this is notslope, which is
the ground's steepest direction and on a track crossing a hillside at 45
degrees says 2% where the track climbs 1.41%; the window slides by the
spacing, so it answers "any 100 m stretch" and not "every 100 m from the
start"; and the base length has no default and must be a whole number of
steps. A window never crosses the gap between two parts of a multi-part line
that do not touch (each point carriesrun_index), nor a point the raster
could not read; a stretch is one direction of travel, so a climb and the
descent after it are two. A line whose unit is not the metre needs
grade_height_unit, because the rise comes from the raster's values, which
state no unit, and metres over US survey feet is 3.28 times too steep. A check
recomputes every stored grade from the profile as written and fails on a
grade missing where a window fits, with the count in its detail.
The new search phrasings moved discovery figures the right way: words alone
under BM25 at 76 entries from 50% to 55%.
Fixed
-
Every position read from a point-registered raster was half a cell off,
since 0.4.0. On a file declaringAREA_OR_POINT=PointMapSmith moved each
sample half a cell north-west of where GDAL puts it: sampling,
elevation_profile,line_of_sight,locate_extreme_cell,least_cost_path,
and the zones ofzonal_statistics, which were shifted for the coverage
computation. The premise was that GDAL reports the tag and leaves the
geotransform alone. It does the opposite — since RFC 33 the GTiff driver moves
a PixelIsPoint tie point half a cell on write and on read, so its geotransform
already centres each cell on its sample — and the correction was made twice.
Found by a Copernicus DEM, whose samples fall on whole arc-seconds by its own
documentation: 10.8 by GDAL, 10.79986 by MapSmith. Positions now come from
GDAL's geotransform whatever the tag says; the tag is still recorded, because
it says what a value represents. The same premise was in Argleton trap 024,
which scored MapSmith's wrong answer as correct for 26 days; the suite has
published an erratum.
A weights raster is no longer refused for a registration different from the
values': on the same geotransform the samples are in the same places. And the
fix no longer depends on the environment:GTIFF_POINT_GEO_IGNORE=TRUE, which
reverts GDAL to its pre-2010 reading, moved every answer half a cell back with
nothing in the record to say why, so MapSmith now pins it toFALSEon import,
whether or not remote paths are allowed — which also overrides an operator's
setting in a process that importsmapsmithas a library, deliberately. -
The terrain operations refused every point-registered DEM, the Copernicus
DEM included. The engine reads the stored tie point as a cell corner, and
the guard that compares its grid with GDAL's caught the half cell and refused,
so the numbers were never wrong and twelve operations were unusable on the
most used global DEM (the eleven that write a raster, andcontour_lines). The engine now gets a copy declared area-registered on
GDAL's own geotransform — the same pattern as the TIFF predictor, disclosed in
the manifest — and the output gets the input's registration back. Their
manifests record the registration incrs_decisions, as the raster writers
already did, and say when MapSmith retagged an output after the engine wrote
it, since those bytes are no longer the engine's alone.
contour_linesloses a branch for point-registered DEMs that corrected half a
cell the wrong way and could not be reached. -
elevation_profilemeasured the spacing in the DEM's units, and on a DEM
in degrees a 1000 m line came back as one point. The line was reprojected
to the raster's CRS before its positions were computed, sospacing=100
meant 100 degrees, and the only trace was a GeoPandas warning on the console.
Positions are now measured along the line in its own projected CRS -- a
geographic line was already refused -- and only the sample points go to the
raster's CRS to be read. The output is now in the line's CRS: when the two
differed it used to be in the raster's, and a plan step reading it must now
expect the line's (the plan validator does). The record says what was
transformed in the specification's own keys:source_crsthe line's,
target_crsthe raster's, and thetransformationthe sample points took to
be read, whileoutput.crssays where the result is. It used to list the line
underx-mapsmith:inputs_reprojectedfrom its own CRS, which that key's
definition makes a contradiction. When line and DEM share a CRS nothing
changes. Found while building a gradient on top of it. -
elevation_profilestopped short of the far end of a line. A line whose
length is not a whole number of steps ended on the last whole step: 15 m at a
step of 4 ended at 12, and a summit in the last 3 m was not in the profile,
while the docstring and the name of the test said both ends were included --
and the test asserted the opposite. The far end is now a last, shorter step,
and the stepping check requires it.
MapSmith 0.6.1
A small release on the day of 0.6.0. It moves no manifest key and changes no
answer that 0.6.0 got right: it stops a refusal 0.6.0 introduced from firing on
a layer written in 0..360 longitudes, gives two refusals the right way out, and
declares the specification's current draft.
Changed
- Records declare
spec_version1.0.0-draft.8. Two drafts in one day,
both narrowings of where a producer may add a key: draft.7 requires every
key ofcrs_decisionsthe specification does not define to be named
x-<producer>:<name>, which it had left to one reading of an ambiguous
sentence, and draft.8 carries the rule insidetransformation,
round_tripandrepairs[]. MapSmith already named every one of them that
way, so no key moves: a reader keying on the label sees draft.8, a reader
keying on fields sees nothing change. The vendored schema and validator are
draft.8's, and the conformance sweep validates every writer's record
against them.
Fixed
- A layer written in 0..360 longitudes had its seam in the wrong place.
The antimeridian refusal of 0.6.0 took the edge of the plane to be -180/180,
so the world written asbox(0, -90, 360, 90)was refused, the way 0.6.0's
own review had found Antarctica refused in -180..180. For such a layer the
edge is 0/360, and the ring that really is drawn naively crosses the prime
meridian: the refusal now says so and tells its author to split there, where
it used to send them to 180. A layer is read as 0..360 when it has no
negative longitude and some beyond 180 -- not whenever one vertex passes
180, which the first version of this fix did, and which refused the world
beside a polygon overshooting 180 by 1e-7. Edges along either pair of seams
are exempt in every layer. - "No UTM zone" said the data was centred on the antimeridian, whatever the
data, and offered no way out. The realistic cause is latitude — UTM stops
at 84N and 80S — and the message now says that and names a polar CRS to use
(UPS, or a polar stereographic), for data that crosses 180 and for data that
does not.