Detect sidewalks and crossings that aren't tagged as such

[lxbarth]

We need to identify all highway=footway that are sidewalks or crosswalks but *not tagged accordingly with the sub tags footway=sidewalk or footway=crossing. This is a non-trivial task as there are many valid footways that are tagged highway=footway in OpenStreetMap - for instance a footway through a park.

Example

Look at this example (click for map):

This inconsistent tagging leads to issues on maps where we want to hide sidewalks (we always do, because sidewalks are horribly incomplete):

Analysis

Here's a sketch for an analysis:

Use Tile Reduce to look for

• parallel (within buffer) highway=footway without a footway=sidewalk or footway=crossing
• perpendicular (within buffer) highway=footway without a footway=crossing or footway=sidewalk

First result should be a map, can be of a small area.

cc @morganherlocker @tcql @planemad @mikelmaron

[tcql]

First (super simple, super naïve) pass is kinda-sorta doing the right things:

color	meaning
red	should be footway=crossing
green	should be footway=sidewalk
purple	part should be footway=crossing, part should be footway=sidewalk

There are some fun edges cases that I'm not handling yet, like bridges / tunnels:

seems like a crossing, because it intersects roads, but is actually a sidewalk along a bridge

sidewalk touches the road, so is flagged as sidewalk and crossing, but there's actually no crossing here

One possible approach here to make this everything more accurate is to work on segments instead of entire ways, so we can flag that only the section which actually crossing a road should be marked footway=crossing, while the other parts should be sidewalks. I should also take into account bearing of lines; sidewalks should be roughly parallel to a road, and remain close to the road, since footway=sidewalk is intended specifically for sidewalks alongside roads

[tcql]

Getting closer! still need to split segments - flagging an entire way as "should be a crossing" isn't as helpful as flagging just a crossing segment:

_this entire footway has been flagged as a crossing. we could segment the line and show specifically which part should be a crossing_

[planemad]

Very cool @tcql

Most of the sidewalk suggestions around the Lincoln memorial look like false positives, so its a nice area to see how to optimize the detection. Once the segment splitting is in this is going to be ready for the data team!

[tcql]

should offset footways be considered footway=sidewalk?

my hunch would be yes

/cc @planemad @lxbarth

[planemad]

@tcql for me, if its alongside the road kerb its a sidewalk, else remains a footway. @1ec5 ?

Its quite hard to define a standard offset under which we flag it as a sidewalk, maybe 3 lanes wide?

[1ec5]

@tcql for me, if its alongside the road kerb its a sidewalk, else remains a footway. @1ec5 ?

I disagree. It really depends: a single sidewalk can run right up to the curb for a stretch then move a few lane-widths away for another stretch but still remain a sidewalk. There is no standard offset, and it doesn’t even have to be completely straight. To me, it’s telling that in this example the walkway parallel to the road continues straight into a crossing at the intersection.

The curb right next to the buses should be tagged as a bus platform. If the buses hadn’t been present in the imagery, I would’ve just ignored that curb.

I’m not sure what to make of the connections between the sidewalk and the road. I’m inclined to call them sidewalks too, because they’re probably also part of the road’s easement and don’t go beyond the walkway that I’m sure is a sidewalk.

[1ec5]

Sometimes there are gray areas, for example when a walkway meanders so far away from a road that it would appear distinct from the road at any zoom level, or when it’s built a bit wider to accommodate bicycles (a lot of suburban “bike trails” are built this way).

[bhousel]

Yeah I am pretty liberal with adding footway=sidewalk tag. I use it for everything that is along a street (or parking area) or connects to the street, or is near the street.

I don't add the sidewalk tag for paths that lead up to building entrances, or go towards some other destination.

(Just my $0.02)

[tcql]

This is actually kind of complex, because it seems like OSM wiki's opinion is that sidewalks should be at the edges of roads, but if you're thinking of the reason for distinction between a plain footway and a sidewalk as a matter of purpose, then things like this should be a sidewalk:

As for distance from the road, OSM wiki makes this pretty clear:

The sidewalk (or pavement) is that part of a highway set aside for the use of pedestrians, often separated from the carriageway (or roadway) by a kerb (also curb). A sidewalk may be separated from the carriageway by only a kerb, by a road verge or alternatively may be at some distance from the road (but still associated with it) or separated from the road by some form of barrier, for example bushes or a line of trees. A road may have a sidewalk on only one side of the carriageway, or both side or have no sidewalks.

EDIT: so given the above I think my question is answered "Yes, those should be sidewalks"

[bhousel]

This is actually kind of complex, because it seems like OSM wiki's opinion is that sidewalks should be at the edges of roads, but if you're thinking of the reason for distinction between a plain footway and a sidewalk as a matter of purpose, then things like this should be a sidewalk:

Personally I would map that as a plain footway because it's not alongside a street. That said.... If somebody else mapped it as sidewalk, I wouldn't bother changing it - that's cool too.

Also, the OSM wiki is.. informative but not normative. It captures the opinion of the people who edit the wiki, not necessarily the opinion of the broader OSM community, or the needs of downstream data consumers, or the current tagging practice (and all of these things change constantly). Most of us who work on OSM tools have a kind of love-hate relationship with the wiki.

[tcql]

Per voice with @lxbarth this isn't super important to nail down at the moment. We'll flag things that are probably sidewalks (because they are really close to roads, stick mostly along the same path as the roadway), and we can discuss some broader questions about tagging obscure cases some other time.

Biggest hurdle we're going to be looking at now is how to check if we've "fixed" everything. Because there can be false positives, re-running this analysis after a big data team lift might still show a lot of potentially bad footways. If anybody has suggestions on how best to get feedback in the end, I'll take them.

[bhousel]

Per voice with @lxbarth this isn't super important to nail down at the moment.

agree 💯

[tcql]

looks like I did a dumb. Many of the segments I'm detecting as missing tags are actually fine, because, while the footway itself is not tagged as footway=crossing, there's a relation between the road and sidewalk that tags them as crossings.

For example:

[1ec5]

Yes, this is a common tagging style (and my personal preference, just out of expediency), although recent changes to iD’s presets are gently nudging me towards using footway=crossing ways too. By the way, call it a shared node. A relation is a much more complicated concept; you had me freaking out about associatedStreet relations when I first read that! 😄

[tcql]

okay, per voice with @lxbarth again, we'll concentrate on segments missing footway=sidewalk, and punt a bit on crossings. In an ideal world I could link up segments that cross roads, don't have footway=crossing and don't have shared nodes that indicate highway=crossing, but right now this is not high priority

[tcql]

Latest results I've got for DC area:

https://api.mapbox.com/styles/v1/tcql/cifpuldbphtbmr7lxw6tb2tay.html?title=true&access_token=pk.eyJ1IjoidGNxbCIsImEiOiJaSlZ6X3JZIn0.mPwXgf3BvAR4dPuBB3ypfA#13.605087499147213/38.89299098795752/-77.04346376737973/0

I'm working on running a bigger job, but having some issues with splitting up the job size / getting tile-reduce-watchbot to play nice. I'll still try and get something more put together tonight

[maning]

Whitelist

• foot=yes; highway=cycleway. See: https://www.openstreetmap.org/way/25964014/
• highway=footway; area=yes or area:highway=* https://www.openstreetmap.org/way/327400057/

[maning]

@tcql, I'm interested on how your detection magic works in this area

This is the park near BLR office. There is sidewalk and a footway inside the park.

[srividyacb]

The automatic matching of sidewalks is not detecting some of the sidewalks which have bridge=yes tag. Only in some parts its detecting the footways which are tagged along with bridge.
Link to area

[maning]

@tcql
Here's a few more false-positives:
tl;dr. It's tricky, adding a simple threshold buffer distance won't cut it. Also I'm not sure splitting footway to 250 feet is the best approach. Is it possible to split when it intersects another road? They way I normally map is that I only split a way if the tag is different. I assume this is generally the same for those who micromapped the roads.the >70% threshol on the bisector is OK though.

This goes along a cul-de-sac then on to the footway. https://www.openstreetmap.org/way/146481383/history#map=19/38.88012/-94.75477

Same as above. https://www.openstreetmap.org/way/146481383/history#map=19/38.88012/-94.75477

The initial segment is near the road, but igoes through the park. https://www.openstreetmap.org/way/113103914/history#map=19/38.90370/-94.75612

There is a parking space in between the road and the footway. https://www.openstreetmap.org/way/146481383/history#map=19/38.88012/-94.75477

The grass looks too wide, this to me should be just footway. https://www.openstreetmap.org/way/102647022/history#map=19/38.88391/-94.72316

Same as above. https://www.openstreetmap.org/way/102647046/history#map=18/38.88342/-94.72566

Partly a crossing but it goes through the building complex, so this should be just footway https://www.openstreetmap.org/way/102647022/history#map=19/38.88391/-94.72316

Footways enclosing a parking lot are not sidewalks. https://www.openstreetmap.org/way/47329241/history#map=18/38.91022/-94.77861

The grass looks too wide, this to me should be just footway.. https://www.openstreetmap.org/way/47329244/history#map=18/38.91025/-94.77997

The grass looks too wide, this to me should be just footway. https://www.openstreetmap.org/way/102647054/history#map=17/38.88381/-94.72702

While this one is close to the road, it is curvy and thus qualify as a footway. https://www.openstreetmap.org/way/102039940/history#map=18/38.88212/-94.75861

Same as above. https://www.openstreetmap.org/way/102039940/history#map=18/38.88212/-94.75861

[tcql]

@srividyacb interesting. bridges / tunnels should work - I've got code that checks the layer between the road and footway when testing intersections. I'll try and track down what's going on.

[srividyacb]

Came across some more false positives:

Images linked to OSM link

Another Example: https://www.openstreetmap.org/#map=18/32.892905/-97.282417
There is really lot of gap between the road and footway. These should be ignores while sidewalk detection.

based on the imagery here the footway looks like it belongs to the park inside and not the sidewalk. These are proper footways

Not a sidewalk: Footway going to parking lot

Similar to this here the footways inside a park are detected in sidewalk matching to mark as sidewalk.

Also another issues in one of the place is that in the beginning the sidewalk runs along with the highway and at one place it moves away from the sidewalk and later again joins the highway parallely.
Link to OSM

[1ec5]

At a cursory glance, some of Maning's examples look like sidewalks to me, although there's some gray area. In the U.S., the trend is to construct sidewalks set back from the street by increasing distances. Of course, if you see a creek running between the footway and the road, it's less likely to be a sidewalk.

[mourner]

@tcql would you be open to a potentially huge performance improvement of the sidewalk script? I have a cool idea.

Currently my linematch algorithm (which is used in the super-fast difference processor) essentially does line subtraction within a tolerance. Earlier I was thinking of making the algorithm more generic by extending it to support line intersection within a tolerance as well, meaning it would output segments of lines that go close to each other. I didn't have a use case for that at the time, but sidewalk detection seems like the perfect use case. It may be useful for other similar tasks in the future too.

[tcql]

@mourner oh that's awesome. Yea, something like that could be very awesome

[bryceco]

I didn't see this mentioned before, but footpaths in parks I've tagged with surface=dirt still show up as potential sidewalks. Sometimes real sidewalks are dirt but usually not (USA).

[tcql]

@bryceco thanks! we're still working on improving this, and surface checking is a great idea

[maning]

@tcql, here's a list of secondary tags that should be excluded:
area=yes
tunnel=yes
name=*
bicycle=yes
highway=cycleway
area:highway=*
surface=unpaved

[1ec5]

If tunnel is excluded, then so should bridge (skywalks, pedestrian bridges).

bicycle=yes

I don't think this tag should be excluded: in many localities, bikes are only allowed on the sidewalk, not the street, and the tagging would reflect that. I think highway=cycleway is sufficient for bike paths.

[tcql]

highway=cycleway

these shouldn't be getting picked up anyway; the analysis only looks at highway=footway

If tunnel is excluded, then so should bridge (skywalks, pedestrian bridges).

There are definitely bridges with what I would consider a sidewalks (and aren't tagged right now), such as the Arlington Memorial Bridge in DC

[jinalfoflia]

these shouldn't be getting picked up anyway; the analysis only looks at highway=footway

@tcql these get picked up when they are combined with highway=footway tag.
For example here where bicycle=yes and highway=footway tags are together.

In cases, where people cycle in these lanes, addition of footway=sidewalk tag will hide the lanes from the style. Therefore for now, we are not adding sidewalk tag for footways like these.

@nickidlugash, hiding sidewalks will lead to discontinuity in footways where the footway is also having bicycle=yes as a sub-tag. Ideally the footways like these should be rendered in cycling-oriented styles even though it's having the sidewalk tag along with it.

/cc @srividyacb @maning

[tcql]

For example here where bicycle=yes and highway=footway tags are together.

These should still be tagged as sidewalks though, if their function is as sidewalks. If the issue is that there are cycling-oriented styles that hide highway=footway + footway=sidewalk even when bicycle=yes, shouldn't the style be updated, instead of skipping things that should be corrected.

[srividyacb]

No next actions here. Closing.