Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clipping monster polygons #606

Closed
wants to merge 6 commits into from

Commits on Dec 5, 2023

  1. don't reclip large polygons that cover z6-z9 tiles

    Progress on systemed#605.
    
    The insight is that very large relations are expensive to clip,
    and we can often avoid clipping them at higher zooms.
    
    If a polygon clips to the full extent of a lower zoom tile, then
    it will do so for all higher zoom tiles.
    
    This PR adds a `largePolygons` map that tracks this. Any polygon
    that covers at least a z9 tile will be tracked, and short-circuited
    at higher zooms.
    
    It seems to speed up rendering z6/16/18 (Hudson Bay) dramatically.
    
    Neighboring tiles still have very high CPU, will dig into that next.
    cldellow committed Dec 5, 2023
    Configuration menu
    Copy the full SHA
    c19e3fa View commit details
    Browse the repository at this point in the history
  2. refactor, support the case where we're in a hole

    ...I think? This is the case where the size of the multipolygon is 0.
    
    This happened for Hudson Bay when it overlapped with land.
    
    A possible risk: are there valid cases where boost fails to produce a
    polygon at a lower zoom level and so returns 0, but would succeed on
    a higher zoom?
    
    Processing the Hudson Bay extract [1]:
    
    master:
    real	22m27.753s
    user	336m50.481s
    sys	0m25.314s
    
    this branch:
    real	3m46.838s
    user	45m33.959s
    sys	0m12.161s
    
    takes 3m46sec.
    
    [1]: https://extract.bbbike.org/?lang=en;sw_lng=-99.269;sw_lat=53.617;ne_lng=-65.244;ne_lat=64.616;format=osm.pbf;city=a%20big%20bay;email=cldellow%40gmail.com;as=76.54103151056417;pg=0.8698404751901266;coords=;oi=1;layers=B000T;submit=;expire=
    cldellow committed Dec 5, 2023
    Configuration menu
    Copy the full SHA
    e7a17cb View commit details
    Browse the repository at this point in the history
  3. build

    cldellow committed Dec 5, 2023
    Configuration menu
    Copy the full SHA
    280524c View commit details
    Browse the repository at this point in the history
  4. tweak

    cldellow committed Dec 5, 2023
    Configuration menu
    Copy the full SHA
    11cc6d5 View commit details
    Browse the repository at this point in the history
  5. update map for z9

    Oops, I was thinking z9 was the base zoom, so it didn't make
    sense to track tiles there--but z9 is the base zoom only for this
    bitset, not for the actual map.
    
    New timings:
    
    real	1m51.573s
    user	16m31.069s
    sys	0m9.744s
    
    This suggests that maybe it's worthwhile to track a secondary hierarchy
    from z10..z13, too.
    cldellow committed Dec 5, 2023
    Configuration menu
    Copy the full SHA
    c9480ab View commit details
    Browse the repository at this point in the history

Commits on Dec 6, 2023

  1. fix the bug, extend to z10..z13

    Unfortunately, the bug turned out to be a critical part of the speed. :)
    
    With it fixed, the new runtime is 11m5s. This is still a ~50% decrease,
    but not what I was hoping for.
    
    Some additional analysis shows that the frequency of a tile being entirely
    excluded or covered is rarer than I initially thought -- it looks like
    it's maybe 15-25% for Hudson Bay and James Bay.
    
    I suspect a better savings would be to also keep the unsimplified, uncombined
    clip from lower zooms around, and have higher zooms clip against them rather
    the raw polygon.
    
    I'll see if I can prototype that.
    cldellow committed Dec 6, 2023
    Configuration menu
    Copy the full SHA
    30fda8c View commit details
    Browse the repository at this point in the history