Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ability to do read/write in feather/parquet #198

Closed
e-lo opened this issue Jul 29, 2020 · 2 comments
Closed

Ability to do read/write in feather/parquet #198

e-lo opened this issue Jul 29, 2020 · 2 comments
Labels
enhancement New feature or request
Milestone

Comments

@e-lo
Copy link
Collaborator

e-lo commented Jul 29, 2020

As a user, I'd like to store data more efficiently than geojson

As a user, I'd like to read networks in less time that it takes to go get coffee

Potential Solutions

Binary data formats such as feather or parquet (compressed version of feather)

Advantage:

  • faster
  • more efficient (e.g. Bay Area links.json: 1.33 GB >>> links.parquet: 508 MB)

Reading speed

Parquet actually slowed read the read speed (!?!?):

image

image

?? Did I do something wrong?

@DavidOry DavidOry added the enhancement New feature or request label Jul 29, 2020
@e-lo e-lo changed the title Ability to do read/write in feather Ability to do read/write in feather/parquet Dec 6, 2022
@e-lo e-lo added this to the v1.0 milestone Dec 6, 2022
@e-lo
Copy link
Collaborator Author

e-lo commented Jul 17, 2024

This was implemented in v1.0-alpha.1 release and will be closed by v1.0 PR

@e-lo
Copy link
Collaborator Author

e-lo commented Aug 27, 2024

Fixed in v1

@e-lo e-lo closed this as completed Aug 27, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants