Fix ESRIGeoJSON: handle Empty reply from server and paginate past maxRecordCount

[rywall]

Summary

• Empty reply from server: ESRIGeoJSON passed remote URLs straight to ogr2ogr, which uses GDAL's curl client. Some ArcGIS-on-IIS endpoints return an empty reply to that client (likely an HTTPS 1.1 negotiation issue — GDAL's driver list hints at this with MapInfo File' needs https 1.1), producing ERROR 1: Empty reply from server. Fix: fetch the URL with Ruby's `open-uri` into a tempfile, then run `ogr2ogr` on the local path.
• Truncation at maxRecordCount: ArcGIS query endpoints cap each response at the service's `maxRecordCount` (commonly 1000 or 2000 features) and signal `exceededTransferLimit` when more results exist. Fix: walk the pages with `resultOffset` and merge them into a single FeatureCollection before handing to `ogr2ogr`.
• Local-path behavior is unchanged.

Test plan

• Import features from an ArcGIS-on-IIS REST `query?...&f=geojson` URL that previously failed with `Empty reply from server`.
• Import features from an ArcGIS query URL whose result set exceeds `maxRecordCount` and confirm all features are returned (not just the first page).
• Import features from an ArcGIS query URL whose result set fits in one page and confirm a single request is still made.
• Import features from a local `.geojson` / `.json` file (regression check for the relative-path branch).

🤖 Generated with Claude Code

[njakobsen]

Both points addressed, and acting on them turned up two defects that were not visible before.

SpatialFeatures::GDAL

New module wrapping the GDAL command line tools. GDAL.capture(tool, *args) returns stdout, GDAL.run(tool, *args) returns the exit status. All three call sites now route through it — ogr2ogr in ESRIGeoJSON, and ogr2ogr plus gdalsrsinfo in Shapefile — where previously each rolled its own invocation in a different style (Open3.capture2, bare system, and a third form).

It also closes a hazard none of the individual call sites had guarded. system("one string") and Open3 both fall back to a shell when handed a lone string, so a future caller passing one argument would silently reintroduce shell parsing. The module always spawns via [[tool, tool], *args], which cannot fall back.

Download

Rather than a new module, the existing SpatialFeatures::Download gained what ESRIGeoJSON had grown privately: Download.read for a body as a String, a configurable Download.timeout applied to every remote fetch, and one UNREACHABLE_ERRORS rescue mapping network failures to ImportError. ESRIGeoJSON no longer carries its own copy of any of that.

Two defects this surfaced

Download.entries ran arbitrary commands. It used Kernel#open, which executes its argument when the string begins with a pipe — Download.entries("|touch /tmp/x") created the file. No caller inside the gem reaches it, but it is public API.

My first fix was wrong, and the spec I wrote to cover it caught that: URI.open hands anything that is not a URL straight to Kernel#open, so swapping one for the other changed nothing. A local path now goes to File.open, which cannot execute.

Centralising the rescue reintroduced a fixed disclosure. Errno::ENOENT is a SystemCallError, so a UNREACHABLE_ERRORS rescue covering local paths turned a missing upload into "This source could not be reached. No such file or directory @ rb_sysopen - /home/.../uploads/..." — putting the server path back in front of the submitter, which 225b8fa had removed. Caught by that commit's own spec. The rescue now applies only to remote URLs.

Verification

302 examples, 0 failures. New download_spec.rb covers the pipe execution on all three entry points, the timeout, the unreachable mapping, and that a missing local path still raises Errno::ENOENT rather than being reported as unreachable.

Mutation-tested: reverting the argv list to a shell string fails 2, routing a local path back through URI.open fails 1, and dropping the timeouts fails 2.

🤖 Generated with Claude Code

https://claude.ai/code/session_017uHh49RG66T3qNFNRnPY6s