stable_sort - refs #90

[springmeyer]

This fixes sorting of features to be stable across platforms. This fixes test failures in #90.

Before merging:

• Assess performance impact

[springmeyer]

Perf results locally on OS X:

stable-sort branch

Run 1:

~/projects/vtquery[stable-sort]$ node bench/vtquery.bench.js --iterations 500 --concurrency 1

1: pip: many building polygons ... 1157 runs/s (432ms)
2: pip: many building polygons, single layer ... 1253 runs/s (399ms)
3: query: many building polygons, single layer ... 877 runs/s (570ms)
4: query: linestrings, mapbox streets roads ... 1812 runs/s (276ms)
5: query: polygons, mapbox streets buildings ... 852 runs/s (587ms)
6: query: all things - dense single tile ... 463 runs/s (1081ms)
7: query: all things - dense nine tiles ... 63 runs/s (7968ms)
8: elevation: terrain tile nepal ... 960 runs/s (521ms)
9: geometry: 2000 points in a single tile, no properties ... 1938 runs/s (258ms)
10: geometry: 2000 points in a single tile, with properties ... 1075 runs/s (465ms)
11: geometry: 2000 linestrings in a single tile, no properties ... 1089 runs/s (459ms)
12: geometry: 2000 linestrings in a single tile, with properties ... 713 runs/s (701ms)
13: geometry: 2000 polygons in a single tile, no properties ... 691 runs/s (724ms)
14: geometry: 2000 polygons in a single tile, with properties ... 581 runs/s (861ms)

Run 2:

~/projects/vtquery[stable-sort]$ node bench/vtquery.bench.js --iterations 500 --concurrency 1

1: pip: many building polygons ... 1147 runs/s (436ms)
2: pip: many building polygons, single layer ... 1289 runs/s (388ms)
3: query: many building polygons, single layer ... 898 runs/s (557ms)
4: query: linestrings, mapbox streets roads ... 1792 runs/s (279ms)
5: query: polygons, mapbox streets buildings ... 887 runs/s (564ms)
6: query: all things - dense single tile ... 480 runs/s (1042ms)
7: query: all things - dense nine tiles ... 64 runs/s (7812ms)
8: elevation: terrain tile nepal ... 982 runs/s (509ms)
9: geometry: 2000 points in a single tile, no properties ... 2041 runs/s (245ms)
10: geometry: 2000 points in a single tile, with properties ... 1094 runs/s (457ms)
11: geometry: 2000 linestrings in a single tile, no properties ... 1126 runs/s (444ms)
12: geometry: 2000 linestrings in a single tile, with properties ... 769 runs/s (650ms)
13: geometry: 2000 polygons in a single tile, no properties ... 790 runs/s (633ms)
14: geometry: 2000 polygons in a single tile, with properties ... 590 runs/s (848ms)

feature-id branch

Using std::sort rather than std::stable_sort:

Run 1:

~/projects/vtquery[feature-id]$ node bench/vtquery.bench.js --iterations 500 --concurrency 1

1: pip: many building polygons ... 1134 runs/s (441ms)
2: pip: many building polygons, single layer ... 1250 runs/s (400ms)
3: query: many building polygons, single layer ... 849 runs/s (589ms)
4: query: linestrings, mapbox streets roads ... 1792 runs/s (279ms)
5: query: polygons, mapbox streets buildings ... 774 runs/s (646ms)
6: query: all things - dense single tile ... 448 runs/s (1117ms)
7: query: all things - dense nine tiles ... 61 runs/s (8215ms)
8: elevation: terrain tile nepal ... 938 runs/s (533ms)
9: geometry: 2000 points in a single tile, no properties ... 1984 runs/s (252ms)
10: geometry: 2000 points in a single tile, with properties ... 1053 runs/s (475ms)
11: geometry: 2000 linestrings in a single tile, no properties ... 1099 runs/s (455ms)
12: geometry: 2000 linestrings in a single tile, with properties ... 731 runs/s (684ms)
13: geometry: 2000 polygons in a single tile, no properties ... 739 runs/s (677ms)
14: geometry: 2000 polygons in a single tile, with properties ... 570 runs/s (877ms)

Run 2:

~/projects/vtquery[feature-id]$ node bench/vtquery.bench.js --iterations 500 --concurrency 1

1: pip: many building polygons ... 1126 runs/s (444ms)
2: pip: many building polygons, single layer ... 1222 runs/s (409ms)
3: query: many building polygons, single layer ... 856 runs/s (584ms)
4: query: linestrings, mapbox streets roads ... 1767 runs/s (283ms)
5: query: polygons, mapbox streets buildings ... 839 runs/s (596ms)
6: query: all things - dense single tile ... 443 runs/s (1129ms)
7: query: all things - dense nine tiles ... 62 runs/s (8082ms)
8: elevation: terrain tile nepal ... 947 runs/s (528ms)
9: geometry: 2000 points in a single tile, no properties ... 2000 runs/s (250ms)
10: geometry: 2000 points in a single tile, with properties ... 1092 runs/s (458ms)
11: geometry: 2000 linestrings in a single tile, no properties ... 1109 runs/s (451ms)
12: geometry: 2000 linestrings in a single tile, with properties ... 722 runs/s (693ms)
13: geometry: 2000 polygons in a single tile, no properties ... 756 runs/s (661ms)
14: geometry: 2000 polygons in a single tile, with properties ... 566 runs/s (884ms)

[sssoleileraaa]

To help with your gut-check, here are my local benchmarks:

macOS

stable-sort branch (using std::stable_sort)

➜  vtquery git:(stable-sort) ✗ node bench/vtquery.bench.js --iterations 500 --concurrency 1

1: pip: many building polygons ... 1089 runs/s (459ms)
2: pip: many building polygons, single layer ... 1222 runs/s (409ms)
3: query: many building polygons, single layer ... 849 runs/s (589ms)
4: query: linestrings, mapbox streets roads ... 1701 runs/s (294ms)
5: query: polygons, mapbox streets buildings ... 813 runs/s (615ms)
6: query: all things - dense single tile ... 394 runs/s (1268ms)
7: query: all things - dense nine tiles ... 61 runs/s (8154ms)
8: elevation: terrain tile nepal ... 926 runs/s (540ms)
9: geometry: 2000 points in a single tile, no properties ... 1901 runs/s (263ms)
10: geometry: 2000 points in a single tile, with properties ... 1059 runs/s (472ms)
11: geometry: 2000 linestrings in a single tile, no properties ... 1059 runs/s (472ms)
12: geometry: 2000 linestrings in a single tile, with properties ... 722 runs/s (693ms)
13: geometry: 2000 polygons in a single tile, no properties ... 754 runs/s (663ms)
14: geometry: 2000 polygons in a single tile, with properties ... 561 runs/s (891ms)

➜  vtquery git:(stable-sort) ✗ node bench/vtquery.bench.js --iterations 500 --concurrency 1

1: pip: many building polygons ... 1099 runs/s (455ms)
2: pip: many building polygons, single layer ... 1217 runs/s (411ms)
3: query: many building polygons, single layer ... 820 runs/s (610ms)
4: query: linestrings, mapbox streets roads ... 1678 runs/s (298ms)
5: query: polygons, mapbox streets buildings ... 826 runs/s (605ms)
6: query: all things - dense single tile ... 451 runs/s (1109ms)
7: query: all things - dense nine tiles ... 61 runs/s (8139ms)
8: elevation: terrain tile nepal ... 936 runs/s (534ms)
9: geometry: 2000 points in a single tile, no properties ... 1852 runs/s (270ms)
10: geometry: 2000 points in a single tile, with properties ... 1066 runs/s (469ms)
11: geometry: 2000 linestrings in a single tile, no properties ... 1075 runs/s (465ms)
12: geometry: 2000 linestrings in a single tile, with properties ... 719 runs/s (695ms)
13: geometry: 2000 polygons in a single tile, no properties ... 740 runs/s (676ms)
14: geometry: 2000 polygons in a single tile, with properties ... 565 runs/s (885ms)

feature-id branch (using std::sort)

➜  vtquery git:(feature-id) ✗ node bench/vtquery.bench.js --iterations 500 --concurrency 1

1: pip: many building polygons ... 1139 runs/s (439ms)
2: pip: many building polygons, single layer ... 1244 runs/s (402ms)
3: query: many building polygons, single layer ... 873 runs/s (573ms)
4: query: linestrings, mapbox streets roads ... 1730 runs/s (289ms)
5: query: polygons, mapbox streets buildings ... 849 runs/s (589ms)
6: query: all things - dense single tile ... 462 runs/s (1082ms)
7: query: all things - dense nine tiles ... 64 runs/s (7863ms)
8: elevation: terrain tile nepal ... 1002 runs/s (499ms)
9: geometry: 2000 points in a single tile, no properties ... 1908 runs/s (262ms)
10: geometry: 2000 points in a single tile, with properties ... 1035 runs/s (483ms)
11: geometry: 2000 linestrings in a single tile, no properties ... 1002 runs/s (499ms)
12: geometry: 2000 linestrings in a single tile, with properties ... 727 runs/s (688ms)
13: geometry: 2000 polygons in a single tile, no properties ... 720 runs/s (694ms)
14: geometry: 2000 polygons in a single tile, with properties ... 556 runs/s (899ms)

➜  vtquery git:(feature-id) ✗ node bench/vtquery.bench.js --iterations 500 --concurrency 1

1: pip: many building polygons ... 1094 runs/s (457ms)
2: pip: many building polygons, single layer ... 1238 runs/s (404ms)
3: query: many building polygons, single layer ... 856 runs/s (584ms)
4: query: linestrings, mapbox streets roads ... 1689 runs/s (296ms)
5: query: polygons, mapbox streets buildings ... 832 runs/s (601ms)
6: query: all things - dense single tile ... 462 runs/s (1082ms)
7: query: all things - dense nine tiles ... 61 runs/s (8141ms)
8: elevation: terrain tile nepal ... 978 runs/s (511ms)
9: geometry: 2000 points in a single tile, no properties ... 1701 runs/s (294ms)
10: geometry: 2000 points in a single tile, with properties ... 1048 runs/s (477ms)
11: geometry: 2000 linestrings in a single tile, no properties ... 1104 runs/s (453ms)
12: geometry: 2000 linestrings in a single tile, with properties ... 737 runs/s (678ms)
13: geometry: 2000 polygons in a single tile, no properties ... 688 runs/s (727ms)
14: geometry: 2000 polygons in a single tile, with properties ... 561 runs/s (892ms)

linux

stable-sort branch (using std::stable_sort)

alliecrevier@ubuntu:~/workspace/vtquery$ node bench/vtquery.bench.js --iterations 500 --concurrency 1

1: pip: many building polygons ... 1901 runs/s (263ms)
2: pip: many building polygons, single layer ... 2137 runs/s (234ms)
3: query: many building polygons, single layer ... 1603 runs/s (312ms)
4: query: linestrings, mapbox streets roads ... 2513 runs/s (199ms)
5: query: polygons, mapbox streets buildings ... 1488 runs/s (336ms)
6: query: all things - dense single tile ... 820 runs/s (610ms)
7: query: all things - dense nine tiles ... 113 runs/s (4425ms)
8: elevation: terrain tile nepal ... 1326 runs/s (377ms)
9: geometry: 2000 points in a single tile, no properties ... 2370 runs/s (211ms)
10: geometry: 2000 points in a single tile, with properties ... 1645 runs/s (304ms)
11: geometry: 2000 linestrings in a single tile, no properties ... 1754 runs/s (285ms)
12: geometry: 2000 linestrings in a single tile, with properties ... 1362 runs/s (367ms)
13: geometry: 2000 polygons in a single tile, no properties ... 1529 runs/s (327ms)
14: geometry: 2000 polygons in a single tile, with properties ... 1208 runs/s (414ms)
alliecrevier@ubuntu:~/workspace/vtquery$ node bench/vtquery.bench.js --iterations 500 --concurrency 1

1: pip: many building polygons ... 2000 runs/s (250ms)
2: pip: many building polygons, single layer ... 2242 runs/s (223ms)
3: query: many building polygons, single layer ... 1520 runs/s (329ms)
4: query: linestrings, mapbox streets roads ... 2513 runs/s (199ms)
5: query: polygons, mapbox streets buildings ... 1572 runs/s (318ms)
6: query: all things - dense single tile ... 797 runs/s (627ms)
7: query: all things - dense nine tiles ... 115 runs/s (4362ms)
8: elevation: terrain tile nepal ... 1319 runs/s (379ms)
9: geometry: 2000 points in a single tile, no properties ... 2283 runs/s (219ms)
10: geometry: 2000 points in a single tile, with properties ... 1712 runs/s (292ms)
11: geometry: 2000 linestrings in a single tile, no properties ... 1712 runs/s (292ms)
12: geometry: 2000 linestrings in a single tile, with properties ... 1080 runs/s (463ms)
13: geometry: 2000 polygons in a single tile, no properties ... 1272 runs/s (393ms)
14: geometry: 2000 polygons in a single tile, with properties ... 1205 runs/s (415ms)

feature-id branch (using std::sort)

alliecrevier@ubuntu:~/workspace/vtquery$ node bench/vtquery.bench.js --iterations 500 --concurrency 1

1: pip: many building polygons ... 1859 runs/s (269ms)
2: pip: many building polygons, single layer ... 2000 runs/s (250ms)
3: query: many building polygons, single layer ... 1553 runs/s (322ms)
4: query: linestrings, mapbox streets roads ... 2439 runs/s (205ms)
5: query: polygons, mapbox streets buildings ... 1553 runs/s (322ms)
6: query: all things - dense single tile ... 786 runs/s (636ms)
7: query: all things - dense nine tiles ... 114 runs/s (4385ms)
8: elevation: terrain tile nepal ... 1370 runs/s (365ms)
9: geometry: 2000 points in a single tile, no properties ... 2146 runs/s (233ms)
10: geometry: 2000 points in a single tile, with properties ... 1645 runs/s (304ms)
11: geometry: 2000 linestrings in a single tile, no properties ... 1748 runs/s (286ms)
12: geometry: 2000 linestrings in a single tile, with properties ... 1309 runs/s (382ms)
13: geometry: 2000 polygons in a single tile, no properties ... 1475 runs/s (339ms)
14: geometry: 2000 polygons in a single tile, with properties ... 1217 runs/s (411ms)
alliecrevier@ubuntu:~/workspace/vtquery$ node bench/vtquery.bench.js --iterations 500 --concurrency 1

1: pip: many building polygons ... 1923 runs/s (260ms)
2: pip: many building polygons, single layer ... 2137 runs/s (234ms)
3: query: many building polygons, single layer ... 1114 runs/s (449ms)
4: query: linestrings, mapbox streets roads ... 1969 runs/s (254ms)
5: query: polygons, mapbox streets buildings ... 1393 runs/s (359ms)
6: query: all things - dense single tile ... 791 runs/s (632ms)
7: query: all things - dense nine tiles ... 103 runs/s (4872ms)
8: elevation: terrain tile nepal ... 1305 runs/s (383ms)
9: geometry: 2000 points in a single tile, no properties ... 2294 runs/s (218ms)
10: geometry: 2000 points in a single tile, with properties ... 1661 runs/s (301ms)
11: geometry: 2000 linestrings in a single tile, no properties ... 1695 runs/s (295ms)
12: geometry: 2000 linestrings in a single tile, with properties ... 1232 runs/s (406ms)
13: geometry: 2000 polygons in a single tile, no properties ... 1479 runs/s (338ms)
14: geometry: 2000 polygons in a single tile, with properties ... 1179 runs/s (424ms)

[springmeyer]

Great, thanks @AllieOop. I'm not seeing a significant performance change. It seems like sometimes std::sort is even slightly slower, which means the noise is more significant that the differences. Going to merge now since this is more correct behavior and warrants, in my opinion, a slight perf hit.

[mapsam]

Thanks @alliecrevier @springmeyer - love seeing the benchmarks here, and totally agree the noise is more significant than the differences. Super enjoy the linux/macos benchmarks as well 👍