Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use Mapnik geojson plugin (rather than CSV) #5

Closed
springmeyer opened this issue Aug 20, 2014 · 2 comments
Closed

Use Mapnik geojson plugin (rather than CSV) #5

springmeyer opened this issue Aug 20, 2014 · 2 comments

Comments

@springmeyer
Copy link
Contributor

We should start targeting the Mapnik geojson native plugin, which like the CSV Plugin(#3) uses Mapnik's fast json parsing grammars. But unlike the CSV plugin we know the whole payload is geojson so there is less tokenization cost.

Specifically, after further optimization in Mapnik 3.x, profiling now shows the tokenization + quoting work to prepare to parse rows as geojson geometries is taking longer than parsing the actual geometries:

screen shot 2014-08-20 at 3 46 54 pm

NOTE: this is with this geojson file with lots of features: https://github.com/mapbox/tilelive-overlay/blob/bench-async/test/data/complex.geojson

The catch is that the geojson plugin only supports featurecollections. So in the case of single geometries, they will need to be wrapped. But this should have minimal overhead and the case of multiple features will then go faster.

@springmeyer
Copy link
Contributor Author

I'll add: in the case of single large geometries and not repeated features in a collection the CSV plugin is not likely much slower.

@tmcw
Copy link
Contributor

tmcw commented Aug 25, 2014

👍 done in m3 branch.

@tmcw tmcw closed this as completed Aug 25, 2014
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants