Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve serialization of geopandas objects and code cleaning #77

Merged
merged 1 commit into from
May 5, 2020

Conversation

mattijn
Copy link
Owner

@mattijn mattijn commented May 5, 2020

This PR mainly targets the topojson.core.extract.py file.

It refactors the parsing of GeoDataFrames and GeoSeries. Now a GeoDataFrame is serialized from gdf.to_dict(orient='records') and a GeoSeries from series.to_dict(). No need to go through the much slower __geo_interface__.

Also this PR prepend non public attributes and functions in the Extract class with a _.
And because I removed the options object out of the .options, Ihad to change some other classes and tests as well.

Also more docstrings are included.

@mattijn mattijn merged commit 39a2839 into master May 5, 2020
@mattijn
Copy link
Owner Author

mattijn commented May 5, 2020

A check with the geojson from #73 (comment) (44mb, 3826 geometries) results in:

  • a speedup of 14.7X [from 1160ms (±140ms) to 79ms (±1.67ms)]

And for naturalearth_lowres:

  • a speedup of 13.4X [from 78.8ms (±1.55ms) to 5.89ms (±0.12ms)]

!Speedup counts only for the Extract class

@mattijn
Copy link
Owner Author

mattijn commented May 5, 2020

As a side effect this PR also introduce support for input objects structured as:

data = [
    {'geometry': <shapely geometry>, 'y': 1, 'z': 2}, 
    {'geometry': <shapely geometry>, 'y': 2, 'z': 3}
]

and

data = {
    0: <shapely geometry>, 
    1: <shapely geometry>
}

@martinfleis
Copy link
Contributor

a speedup of 14.7X [from 1160ms (±140ms) to 79ms (±1.67ms)]

That is great!

@mattijn mattijn deleted the improve-extract-geopandas branch July 6, 2020 08:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants