Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Attic: Inconsistent Results way(id) vs. way[tag] #177

Closed
mmd-osm opened this issue Jan 10, 2015 · 6 comments
Closed

Attic: Inconsistent Results way(id) vs. way[tag] #177

mmd-osm opened this issue Jan 10, 2015 · 6 comments

Comments

@mmd-osm
Copy link
Contributor

mmd-osm commented Jan 10, 2015

There seems to be some kind of inconsistency when querying for way 60966410 at a certain point in time vs. querying all ways with one of its tags ("mtb:scale"). Please also see this forum thread: http://forum.openstreetmap.org/viewtopic.php?pid=476883#p476883

Test case 1:

[date:"2014-07-07T19:41:11Z"];
way(60966410);
out geom;

Returns:

pic1

<?xml version="1.0" encoding="UTF-8"?>
<osm version="0.6" generator="Overpass API">
<note>The data included in this document is from www.openstreetmap.org. The data is made available under ODbL.</note>
<meta osm_base="2015-01-10T10:16:02Z"/>

  <way id="60966410">
    <bounds minlat="48.5431916" minlon="9.3482167" maxlat="48.5451772" maxlon="9.3536466"/>
    <nd ref="251667573" lat="48.5432418" lon="9.3536466"/>
    <nd ref="251667574" lat="48.5432090" lon="9.3533588"/>
    <nd ref="568252984" lat="48.5431926" lon="9.3531511"/>
    <nd ref="251667575" lat="48.5432150" lon="9.3526754"/>
    <nd ref="251667576" lat="48.5431916" lon="9.3521809"/>
    <nd ref="2952899203" lat="48.5432048" lon="9.3518049"/>
    <nd ref="2952899205" lat="48.5432462" lon="9.3514781"/>
    <nd ref="251667577" lat="48.5433119" lon="9.3511877"/>
    <nd ref="251667578" lat="48.5435226" lon="9.3505242"/>
    <nd ref="251667579" lat="48.5437556" lon="9.3496745"/>
    <nd ref="251667580" lat="48.5440681" lon="9.3493054"/>
    <nd ref="251667581" lat="48.5443523" lon="9.3490736"/>
    <nd ref="251667582" lat="48.5446818" lon="9.3488247"/>
    <nd ref="251667583" lat="48.5450311" lon="9.3485909"/>
    <nd ref="2807533011" lat="48.5450952" lon="9.3485458"/>
    <nd ref="251667584" lat="48.5451354" lon="9.3484430"/>
    <nd ref="251667585" lat="48.5451772" lon="9.3482167"/>
    <tag k="highway" v="path"/>
    <tag k="mtb:scale" v="1"/>
  </way>

</osm>

Test case 2:

Overpass Turbo Link: http://overpass-turbo.eu/s/6X6

[date:"2014-07-07T19:41:11Z"];
way({{bbox}})["mtb:scale"];
out geom;

Returns:

pic2

Issue: Way 60966410 was not returned in the result.

Test case 3:

[date:"2014-07-07T19:41:11Z"];
way({{bbox}})["mtb:scale"~"."];
out geom;

Issue: Way 60966410 was not returned in the result.

Test case 4:

[date:"2014-07-07T19:41:11Z"];
way({{bbox}})["mtb:scale"~"^1$"];
out geom;

Issue: Way 60966410 was not returned in the result.

Test case 5:

[date:"2014-07-07T19:41:11Z"];
way({{bbox}})["mtb:scale"="1"];
out geom;

Way 60966410 is found:

pic3
@mmd-osm
Copy link
Contributor Author

mmd-osm commented Jan 11, 2015

Combined test case

I also tried a combined test case 2 + 5: http://overpass-turbo.eu/s/6Yb

[timeout:500]
[date:"2014-07-07T19:41:11Z"];
(
  way["mtb:scale"="0"]({{bbox}});
  way["mtb:scale"="1"]({{bbox}});
  way["mtb:scale"="2"]({{bbox}});
  way["mtb:scale"="3"]({{bbox}});
  way["mtb:scale"="4"]({{bbox}});  
  way["mtb:scale"="5"]({{bbox}});
)->.w1;

way({{bbox}})["mtb:scale"]->.w2;

(.w1; - .w2);
out geom;

Depending on the bbox, I either get no results at all (which is good), or some ways (may not occur), or in some cases it even triggered the following error:

<remark> runtime error: Way 287558455 cannot be expanded at timestamp 2014-08-25T14:55:02Z. </remark>
<remark> runtime error: Way 287558455 cannot be expanded at timestamp 2014-08-25T14:55:02Z. </remark>
<remark> runtime error: Way 287558455 cannot be expanded at timestamp 2014-08-25T14:55:02Z. </remark>
<remark> runtime error: Way 287558455 cannot be expanded at timestamp 2014-08-25T14:55:02Z. </remark>
<remark> runtime error: Way 287558455 cannot be expanded at timestamp 2014-08-25T14:55:02Z. </remark>
<remark> runtime error: Way 287558455 cannot be expanded at timestamp 2014-08-25T14:55:02Z. </remark>
<remark> runtime error: Way 287558455 cannot be expanded at timestamp 2014-08-25T14:55:02Z. </remark>

@mmd-osm
Copy link
Contributor Author

mmd-osm commented Feb 21, 2015

I can also reproduce the issue on the dev instance (--db-dir= ... /v0.7.51)

[timeout:500]
[bbox:48.4045911084346,9.123458862304688,48.746228791937774,9.855422973632812]
[date:"2014-07-07T19:41:11Z"];
(
  way["mtb:scale"="0"];
  way["mtb:scale"="1"];
  way["mtb:scale"="2"];
  way["mtb:scale"="3"];
  way["mtb:scale"="4"];  
  way["mtb:scale"="5"];
)->.w1;

way["mtb:scale"]->.w2;

(.w1; - .w2);
out geom;

List of affected ways in bbox (created using out ids; instead of out geom;):

Note: This list should be empty

<?xml version="1.0" encoding="UTF-8"?>
<osm version="0.6" generator="Overpass API">
<note>The data included in this document is from www.openstreetmap.org. The data is made available under ODbL.</note>
<meta osm_base="2015-01-24T00:17:02Z"/>

  <bounds minlat="48.4046" minlon="9.12346" maxlat="48.7462" maxlon="9.85542"/>

runtime error: Way 287558455 cannot be expanded at timestamp 2014-08-25T14:55:02Z.
runtime error: Way 287558455 cannot be expanded at timestamp 2014-08-25T14:55:02Z.
runtime error: Way 287558455 cannot be expanded at timestamp 2014-08-25T14:55:02Z.
runtime error: Way 287558455 cannot be expanded at timestamp 2014-08-25T14:55:02Z.
runtime error: Way 287558455 cannot be expanded at timestamp 2014-08-25T14:55:02Z.
runtime error: Way 287558455 cannot be expanded at timestamp 2014-08-25T14:55:02Z.
runtime error: Way 287558455 cannot be expanded at timestamp 2014-08-25T14:55:02Z.
  <way id="21636094"/>
  <way id="22938989"/>
  <way id="23335785"/>
  <way id="24308941"/>
  <way id="25691825"/>
  <way id="25691826"/>
  <way id="26327926"/>
  <way id="26503983"/>
  <way id="26707364"/>
  <way id="26799304"/>
  <way id="28798121"/>
  <way id="34117437"/>
  <way id="34117447"/>
  <way id="34117451"/>
  <way id="34312600"/>
  <way id="34730721"/>
  <way id="34730722"/>
  <way id="34730723"/>
  <way id="35159922"/>
  <way id="35266316"/>
  <way id="35552024"/>
  <way id="35553078"/>
  <way id="35815604"/>
  <way id="35939581"/>
  <way id="37739517"/>
  <way id="37954344"/>
  <way id="37954364"/>
  <way id="38421289"/>
  <way id="41060378"/>
  <way id="41062100"/>
  <way id="44818366"/>
  <way id="44846388"/>
  <way id="49084306"/>
  <way id="54266156"/>
  <way id="60966410"/>
  <way id="61694044"/>
  <way id="61975342"/>
  <way id="68494911"/>
  <way id="68494917"/>
  <way id="75349832"/>
  <way id="83399496"/>
  <way id="83824411"/>
  <way id="83824413"/>
  <way id="108191373"/>
  <way id="110939027"/>
  <way id="111685528"/>
  <way id="111685530"/>
  <way id="115664871"/>
  <way id="120727291"/>
  <way id="129129443"/>
  <way id="129129444"/>
  <way id="129129448"/>
  <way id="129139463"/>
  <way id="132758517"/>
  <way id="132761254"/>
  <way id="146912450"/>
  <way id="146912460"/>
  <way id="146912461"/>
  <way id="146976115"/>
  <way id="155781192"/>
  <way id="163728772"/>
  <way id="163728785"/>
  <way id="183632477"/>
  <way id="186952529"/>
  <way id="188673338"/>
  <way id="192902374"/>
  <way id="198942965"/>
  <way id="212734317"/>
  <way id="215326269"/>
  <way id="227911453"/>
  <way id="239078661"/>
  <way id="240612593"/>
  <way id="243585683"/>
  <way id="266681009"/>
  <way id="274510651"/>
  <way id="274920292"/>
  <way id="277782220"/>
  <way id="279962061"/>
  <way id="283423747"/>
  <way id="287518580"/>
  <way id="287520004"/>
  <way id="287540681"/>
  <way id="287576680"/>
  <way id="288501243"/>
  <way id="288501244"/>
  <way id="288937923"/>
  <way id="288937924"/>
  <way id="288937948"/>
  <way id="288937949"/>
  <way id="289991072"/>
  <way id="290833276"/>
  <way id="290833277"/>
  <way id="291733790"/>
  <way id="291751473"/>
  <way id="291751474"/>

</osm>

@drolbr
Copy link
Owner

drolbr commented Feb 24, 2015

Short information:
I'm investigating this bug along the bug causing the "Way X cannot be expanded" messages on the dev instance.

It is a bug in the backend because the database files show bogus duplicate attic objects that have the same timestamp and id as their regular counterparts. I don't know yet which parts of the update mechanism cause the problems. The only thing I know so far is that the objects in isolation don't trigger the bug.

I'm currently trying to reproduce the issue on a sufficiently small update step (300 diffs instead of thousands of diffs).

@mmd-osm
Copy link
Contributor Author

mmd-osm commented Feb 24, 2015

Thanks for looking into the example! I've found another example area in Switzerland, which wasn't hit that much by large scale split/combine activities in the past and may be easier to debug: http://overpass-turbo.eu/s/7Rt - this query will return a single affected way 28286609.

@drolbr
Copy link
Owner

drolbr commented Feb 28, 2015

The issue itself has been fixed with commit a4c6cfe

However, the damaged data will remain present until the database has been completely rebuild. This may take some months. To make this process more productive, I would like to investigate beforehand what causes the "Way cannot be expanded" errors.

@drolbr drolbr closed this as completed Feb 28, 2015
@mmd-osm
Copy link
Contributor Author

mmd-osm commented Mar 3, 2015

Great! Thanks a lot for fixing this tricky bug, Roland.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@drolbr @mmd-osm