perf: reduce memory in to_python() (return PyList instead of Vec) #44
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Partial fix #42. Return PyList instead of Vec - remove unnecessary reallocation.
Memory:
![image](https://private-user-images.githubusercontent.com/3132181/287198649-0b02e0a5-ac92-4b15-9536-7c4304d199ba.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjMzMzQyNDksIm5iZiI6MTcyMzMzMzk0OSwicGF0aCI6Ii8zMTMyMTgxLzI4NzE5ODY0OS0wYjAyZTBhNS1hYzkyLTRiMTUtOTUzNi03YzQzMDRkMTk5YmEucG5nP1gtQW16LUFsZ29yaXRobT1BV1M0LUhNQUMtU0hBMjU2JlgtQW16LUNyZWRlbnRpYWw9QUtJQVZDT0RZTFNBNTNQUUs0WkElMkYyMDI0MDgxMCUyRnVzLWVhc3QtMSUyRnMzJTJGYXdzNF9yZXF1ZXN0JlgtQW16LURhdGU9MjAyNDA4MTBUMjM1MjI5WiZYLUFtei1FeHBpcmVzPTMwMCZYLUFtei1TaWduYXR1cmU9NWVkYTY2ZWRiODZjZWYyNWU4NDhiMjVmN2EwNDM2ZjkzMTM4OTNhYWZiYzI4ODRhMmJkZGZjNzMyZTQ3ODJmZCZYLUFtei1TaWduZWRIZWFkZXJzPWhvc3QmYWN0b3JfaWQ9MCZrZXlfaWQ9MCZyZXBvX2lkPTAifQ.2TfZZBiIwCVtNYCiY127UTKXgDusTqzCEHhPCMPEU5s)
example file (saved as xlsx in LibreOffice).
Perormance:
A benchmark for reading files from https://github.com/MarkPflug/Benchmarks/tree/main/source/Benchmarks/Data using
pandas.read_excel
(first - all file, second - first 10 rows).Before
After