Join GitHub today
GitHub is home to over 50 million developers working together to host and review code, manage projects, and build software together.
Sign upSupport lazily reading a serialized Vector #233
Closed
Comments
|
Would this approach work? The example there is very similar to yours. https://serde.rs/stream-array.html |
|
Closing this since its been 2 years since response. dtolnay's provided answer seems to be a good way to do this. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
It would be great if it was possible to incrementally read a large file that contains a serialized Vec.
A large file may contain 10 million records and you may not be able to load everything into memory
before processing it. Being able to iterate over elements individually or in chunks would be very helpful.
It would allow for doing something like:
decode_vec(file_reader)
.into_iter()
.filter(|person| person.last_name.starts_with("Jones") )
.map(|person| person.age)
.sum()
Or whatever query one might want to do on a large corpus of structured data.