Filtering the output of the `map` API endpoint #124

tmcw opened this Issue Oct 11, 2012 · 9 comments


None yet

6 participants

tmcw commented Oct 11, 2012

The core API should support XAPI-style nodes-only, ways-only, etc queries. This might not be the same scope as XAPI, and as a read/write API has more issues, but I'm cool to look through them and see if this would work.

This would constitute a real benefit to limited editors, so that they can pull data from dense areas (read cities) and just edit the data they present (read POIs)

Also, this may be implemented in cgimap instead - @TomH & @zerebubuth have the wisdom there.


This is as really bad idea.

POI editor writers think this is what they want, but it isn't really what they need and it leads to thinks like MapZen POI Collector that encourage the adding of duplicate POIs because they only show nodes and ignore POIs which exist as areas.

@tomhughes tomhughes closed this Oct 11, 2012

Maybe I haven't understood what you're trying to say, but wouldn't using a (j)XAPI instance work just as well?

Firstly, jXAPI already has the functionality to do nodes-only / ways-only queries, including filtering by tag. jXAPI also supports minutely updates so it's unlikely that the data will be seriously out of sync. It seems that duplicating this functionality in the core API would be of little benefit.

Secondly, the OSM representation of a POI could be any one of a node, way or relation with a bunch of different tags, so a limited editor for POIs would need to understand and deal with the complexity of these concepts anyway. For a POI editor it might be the case that a proxy API to translate between "limited" representations and the full OSM data model would be a better solution.

tmcw commented Oct 11, 2012

@tomhughes I would appreciate you keeping this discussion open; even if you think this is a bad idea (it certainly might be), it could use some discussion rather than unilateral dismissal. You are not the only contributor or decision-maker for this project.

@zerebubuth The point of this would be to have a read/write API for editors; pulling data from one source and pushing to to another unsynchronized source doesn't seem doable to me, though I could be wrong. This also could amount to just a reliable hosted version of jXAPI that people could use.


Certainly Tom (H)'s point about the Mapzen POI Collector issue is a real one; essentially it fell down where a feature was present in the OSM database as a way outline, but wasn't rendered (either because of collisions or because it simply wasn't in the stylesheet). Since users didn't see it on the tile map display, they then proceeded to add a dupe. It's even more likely to be a problem these days because we have good aerial imagery, so people are increasingly mapping POIs as outlines.

A proxy read-only API is the most interesting idea I've yet heard of solving this, whether running off minutely diffs or chained to the main db servers via some magic replication thingy which is way beyond my paygrade.


I'll reopen this for you @tmcw but personally I really don't think that a bug tracker is the right place to have early phase discussions of potential enhancements.

@tomhughes tomhughes reopened this Oct 11, 2012
tmcw commented Oct 11, 2012

Okay, where would be a better place? We both know that IRC and dev@ are filled with flamewars, and the ewg meetings don't have nearly enough time for this sort of discussion.


@tmcw You say unsynchronised, I say loosely coupled ;-)

It's eminently doable - in fact many people already do download jXAPI data, load it into JOSM, edit it and upload it (generally for the purposes of mass automated editing - which I'm very much not in favour of, but on a technical level it's already happening). There's a small lag between jXAPI and the main API which is on the order of minutes, but recent work by Brett means that could come down to seconds.

In any case, the lag associated with data resting in the user's device / desktop is often greater than the lag between jXAPI and the main API. Even when that's not the case, the optimistic locking of the API allows the editor to detect these conflicts and prompt the user to reconcile them.


Re. "dev being filled with flamewars", I believe it's a matter of how you approach such a list. If you come barging in with "here's the improvements I want to push as part of the great kickoff (blog link) for our magnificent Knight initiative" then you might see more headwind than if you said "i've been thinking that these things would be nice to have as building blocks for future work; what do you think, and would the API be the right place for that?". It's something you can, and ultimately have to, learn if you want to work with the community. I think modesty might be the word.

Being more of a mailing list guy myself, I've responded to some of your issues on where these comments seem to be gated to unidirectionally; I'm sorry for the inconvenience but I really can't bring myself to having discussions in the github browser window.


On 10/11/2012 05:37 PM, Tom MacWright wrote:

@zerebubuth The point of this would be
to have a read/write API for editors; pulling data from one source and
pushing to to another unsynchronized source doesn't seem doable to me,
though I could be wrong. This also could amount to just a reliable
hosted version of jXAPI that people could use.

I am not sure that is that much of a problem. Assuming you have a
reliable jXAPI which consumes minutely diffs, then you shouldn't be more
than 2 or so minutes behind the main API. Most editors have more than 2
minutes between when they download the data and when they upload the
changes. So any conflict that can come from "unsynchronized" jXAPI can
happen anyway.

In order to detect any conflict you need to upload the version number of
the data you have and the API will only accept the upload if the data
hasn't changed in the meantime.

So having a different read and write API seems fine. Indeed people use
the JOSM mirror download plugin to use alternative servers for the map
call and still push back to the main API.

I suspect the bigger issue is that jXAPI hasn't been as stable as the
main API, so writing an editor that relies on it will equally be prone
to reduced availability. But that is a different issue.


Reply to this email directly or view it on GitHub
#124 (comment).

rails-dev mailing list

@tmcw tmcw closed this Jan 28, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment