Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

layer dropping #113

Merged
merged 15 commits into from Jul 9, 2021
Merged

layer dropping #113

merged 15 commits into from Jul 9, 2021

Conversation

mapsam
Copy link
Member

@mapsam mapsam commented Jun 11, 2021

What changed?

  • Add layers option to the tile objects that allows user to specify which layers to keep in the final vector tile
  • the layers array must have at least one string, cannot be empty
  • if the layers array is not included, all layers are kept
  • add new validation and success tests
  • add layer dropping fixture to benchmarks

Housekeeping

  • Update README examples
  • Remove documentation auto generated docs
  • remove /docs which were copied from node-cpp-skel
  • remove a lot of README copied from node-cpp-skel
  • remove API.md which was copied from node-cpp-skel

cc @mapbox/tilesets-api @artemp

@mapsam
Copy link
Member Author

mapsam commented Jun 11, 2021

benchmarks round 1

Benchmarks look good. Mostly want to see no loss in performance with this addition, which these show. Looks like a slight increase, which is hard to believe, so I'll consider that mostly noise for now. No significant performance changes!

test 0.4.0 215e44e
1: single tile in/out 22422 runs/s (446ms) 24038 runs/s (416ms)
2: two different tiles at the same zoom level, zero buffer 5173 runs/s (1933ms) 5206 runs/s (1921ms)
3: two different tiles from different zoom levels (separated by one zoom), zero buffer 757 runs/s (13206ms) 876 runs/s (11419ms)
4: two different tiles from different zoom levels (separated by more than one zoom), zero buffer 1870 runs/s (5349ms) 1854 runs/s (5393ms)
5: tiles completely made of points, overzooming, no properties 10549 runs/s (948ms) 9242 runs/s (1082ms)
6: tiles completely made of points, same zoom, no properties 34602 runs/s (289ms) 32468 runs/s (308ms)
7: tiles completely made of points, overzoooming, lots of properties 7758 runs/s (1289ms) 7052 runs/s (1418ms)
8: tiles completely made of points, same zoom, lots of properties 27174 runs/s (368ms) 28653 runs/s (349ms)
9: buffer_size 128 - tiles completely made of points, same zoom, lots of properties 23095 runs/s (433ms) 23256 runs/s (430ms)
10: tiles completely made of linestrings, overzooming and lots of properties 1818 runs/s (5500ms) 1796 runs/s (5567ms)
11: tiles completely made of polygons, overzooming and lots of properties 362 runs/s (27616ms) 359 runs/s (27853ms)
12: tiles completely made of points and linestrings, overzooming and lots of properties 16103 runs/s (621ms) 13966 runs/s (716ms)
13: buffer_size 128 - tiles completely made of points and linestrings, overzooming and lots of properties 15823 runs/s (632ms) 12531 runs/s (798ms)
14: tiles completely made of points and linestrings, overzooming (2x) and lots of properties 21186 runs/s (472ms) 20161 runs/s (496ms)
15: tiles completely made of polygons, overzooming and lots of properties 1373 runs/s (7281ms) 1276 runs/s (7835ms)
16: tiles completely made of polygons, overzooming (2x) and lots of properties 2966 runs/s (3372ms) 3074 runs/s (3253ms)
17: buffer_size 4096 - tiles completely made of polygons, overzooming (2x) and lots of properties 1416 runs/s (7061ms) 1391 runs/s (7190ms)
18: single tile, dropping layers ... n/a 21413 runs/s (467ms)
peak mem (max_rss, max_heap, max_heap_total) 1.77GB 21.77MB 52.23MB 1.5GB 23.02MB 51.73MB

@mapsam mapsam marked this pull request as ready for review July 8, 2021 21:12
@mapsam mapsam requested a review from a team as a code owner July 8, 2021 21:12
Copy link
Contributor

@e-n-f e-n-f left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me in general, aside from a few questions inline

.travis.yml Show resolved Hide resolved
common.gypi Outdated Show resolved Hide resolved
: z{z0},
x{x0},
y{y0},
data{buffer.Data(), buffer.Length()},
buffer_ref{Napi::Persistent(buffer)}
buffer_ref{Napi::Persistent(buffer)},
layers{std::move(layers0)}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why does this move the layers0 argument instead of using a const reference?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good question! I'm open to ideas here - the clang tidy/format builds told me to use std::move here, so I'm admittedly just plugging that in. Can you show me how I'd use a const reference?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What I was talking about was 05cbd11bd093d, but you are right and I am wrong, and the TileObject needs to contain its own copy of the vector instead of holding a reference back to the caller's vector, which apparently goes out of scope before it is used. So please ignore this comment.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 thanks for the commit + trying it out @ericfischer!

src/vtcomposite.cpp Outdated Show resolved Hide resolved
@mapsam mapsam merged commit 13f697a into master Jul 9, 2021
@mapsam mapsam deleted the layer-dropping branch July 9, 2021 12:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants