Display custom map data from WMS endpoints

[1ec5]

#7510 added support for WMS tokens like bbox, height, and proj in custom backgrounds. This extends the custom background feature to be compatible with ArcGIS MapServers and ImageServers. However, for custom map data URL templates, iD only supports TMS and quadkey tokens and output in Mapbox Vector Tile format. It should also support WMS tokens and ArcGIS server JSON to make the feature compatible with more existing data sources.

The idea is merely to load ArcGIS FeatureServer layers in the same way that custom map data can already be loaded into iD: as a reference layer rather than a way to directly add data to OSM. I think further streamlining the workflow would be out of scope for this issue.

Rationale

The code for resolving URL templates should be shared between the custom background and custom map data modules to improve code coverage and consistency.

Basic WMS support can be seen as a baby step compared to the full ArcGIS FeatureServer integration proposed in #4164, but I think there is value in just getting WMS support to parity with the existing support for TMS and local files. The U.S. mapping community has discovered many appropriately licensed sources of aerial imagery from local government agencies; these sources have been a key ingredient in promoting local community involvement. But many of these agencies also provide non-imagery layers that we’ve yet to tap into, even in a bespoke, manual mapping workflow.

For example, an Ohio state agency provides a FeatureServer containing public domain address points for most of the state. Even though I’m not planning an automated import of this address data in the near future, I would like to be able to quickly look up the address of an individual building as I manually map it, since it would be nice to have this one address in the meantime, and my manual edit will make the building less likely to be included in an initial phase of a future building+address import. The same dataset comes with a MapServer, but it doesn’t label the address points and doesn’t have dynamic layers enabled, so I’m forced to fire up JOSM to load the FeatureServer. By comparison, a similar MapServer in Silicon Valley has saved us countless hours of manual hunting in our POI import there, but only because it was correctly styled with labels in the first place.

Beyond addresses, I’ve also come across other public domain FeatureServers that contain information like road names, speed limits, and boundaries that would be useful as a reference but not as an import source. I would like to be able to add unwieldy URLs for these FeatureServers to our local wiki page as resources for mappers who understand the difference between OSM tagging and the attributes in these FeatureServers but who are unlikely to use JOSM or QGIS as part of their day-to-day workflow.

Implementation notes

The custom background code contains a large chunk of code dedicated to URL template resolution:

iD/modules/renderer/background_source.js

Lines 113 to 228 in 2803cd7

	source.url = function(coord) {
	var result = _template;
	if (result === '') return result; // source 'none'
	// Guess a type based on the tokens present in the template
	// (This is for 'custom' source, where we don't know)
	if (!source.type) {
	if (/SERVICE=WMS|\{(proj|wkid|bbox)\}/.test(_template)) {
	source.type = 'wms';
	source.projection = 'EPSG:3857'; // guess
	} else if (/\{(x|y)\}/.test(_template)) {
	source.type = 'tms';
	} else if (/\{u\}/.test(_template)) {
	source.type = 'bing';
	}
	}
	if (source.type === 'wms') {
	var tileToProjectedCoords = (function(x, y, z) {
	//polyfill for IE11, PhantomJS
	var sinh = Math.sinh || function(x) {
	var y = Math.exp(x);
	return (y - 1 / y) / 2;
	};
	var zoomSize = Math.pow(2, z);
	var lon = x / zoomSize * Math.PI * 2 - Math.PI;
	var lat = Math.atan(sinh(Math.PI * (1 - 2 * y / zoomSize)));
	switch (source.projection) {
	case 'EPSG:4326':
	return {
	x: lon * 180 / Math.PI,
	y: lat * 180 / Math.PI
	};
	default: // EPSG:3857 and synonyms
	var mercCoords = d3_geoMercatorRaw(lon, lat);
	return {
	x: 20037508.34 / Math.PI * mercCoords[0],
	y: 20037508.34 / Math.PI * mercCoords[1]
	};
	}
	});
	var tileSize = source.tileSize;
	var projection = source.projection;
	var minXmaxY = tileToProjectedCoords(coord[0], coord[1], coord[2]);
	var maxXminY = tileToProjectedCoords(coord[0]+1, coord[1]+1, coord[2]);
	result = result.replace(/\{(\w+)\}/g, function (token, key) {
	switch (key) {
	case 'width':
	case 'height':
	return tileSize;
	case 'proj':
	return projection;
	case 'wkid':
	return projection.replace(/^EPSG:/, '');
	case 'bbox':
	// WMS 1.3 flips x/y for some coordinate systems including EPSG:4326 - #7557
	if (projection === 'EPSG:4326' &&
	// The CRS parameter implies version 1.3 (prior versions use SRS)
	/VERSION=1.3|CRS={proj}/.test(source.template())) {
	return maxXminY.y + ',' + minXmaxY.x + ',' + minXmaxY.y + ',' + maxXminY.x;
	} else {
	return minXmaxY.x + ',' + maxXminY.y + ',' + maxXminY.x + ',' + minXmaxY.y;
	}
	case 'w':
	return minXmaxY.x;
	case 's':
	return maxXminY.y;
	case 'n':
	return maxXminY.x;
	case 'e':
	return minXmaxY.y;
	default:
	return token;
	}
	});
	} else if (source.type === 'tms') {
	result = result
	.replace('{x}', coord[0])
	.replace('{y}', coord[1])
	// TMS-flipped y coordinate
	.replace(/\{[t-]y\}/, Math.pow(2, coord[2]) - coord[1] - 1)
	.replace(/\{z(oom)?\}/, coord[2])
	// only fetch retina tiles for retina screens
	.replace(/\{@2x\}|\{r\}/, isRetina ? '@2x' : '');
	} else if (source.type === 'bing') {
	result = result
	.replace('{u}', function() {
	var u = '';
	for (var zoom = coord[2]; zoom > 0; zoom--) {
	var b = 0;
	var mask = 1 << (zoom - 1);
	if ((coord[0] & mask) !== 0) b++;
	if ((coord[1] & mask) !== 0) b += 2;
	u += b.toString();
	}
	return u;
	});
	}
	// these apply to any type..
	result = result.replace(/\{switch:([^}]+)\}/, function(s, r) {
	var subdomains = r.split(',');
	return subdomains[(coord[0] + coord[1]) % subdomains.length];
	});
	return result;
	};

By comparison, the custom map data code has much simpler code to resolve URL templates:

iD/modules/services/vector_tile.js

Lines 98 to 107 in 2803cd7

	var url = source.template
	.replace('{x}', tile.xyz[0])
	.replace('{y}', tile.xyz[1])
	// TMS-flipped y coordinate
	.replace(/\{[t-]y\}/, Math.pow(2, tile.xyz[2]) - tile.xyz[1] - 1)
	.replace(/\{z(oom)?\}/, tile.xyz[2])
	.replace(/\{switch:([^}]+)\}/, function(s, r) {
	var subdomains = r.split(',');
	return subdomains[(tile.xyz[0] + tile.xyz[1]) % subdomains.length];
	});

I suspect the two code paths were originally mostly identical but diverged over time because there was more interest around custom backgrounds. We could factor out the custom background version into util.js for now; while it would be tempting to write a whole module for tile sources, there’s still a lot of functionality that doesn’t overlap between background_source.js and vector_tile.js.

The custom map data module currently supports local files in GPX, KML, and GeoJSON formats and tile servers in Mapbox Vector Tile format via togeojson. Every FeatureServer I’ve come across supports JSON formatted output for query operations, but it isn’t GeoJSON that iD expects from local JSON files. ArcGIS Server 10.4 and above can also output GeoJSON and KMZ. Maybe we could start by hooking up GeoJSON support, but I don’t think we can count on most FeatureServer endpoints allowing that output format. (For example, the address point FeatureServer mentioned above only outputs ArcGIS feature JSON.)