Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

use default chunk_size in download #528

Closed
soxofaan opened this issue Jan 17, 2024 · 1 comment · Fixed by #535
Closed

use default chunk_size in download #528

soxofaan opened this issue Jan 17, 2024 · 1 comment · Fixed by #535
Assignees
@VictorVerhaert
Labels
feature request good first issue low hanging fruit

Comments

@soxofaan
Copy link
Member

download currently uses None chunk_size, which apparently is not ideal at the moment (risk of loading download first in memory before dumping to disk)
see #524 and Open-EO/openeo-geopyspark-driver#631

I think we can just start here by using a default of say 10MB. If there is need for making this configurable, we can add that later, but I would expect that a better default than None will cover most use cases
@soxofaan soxofaan mentioned this issue Jan 24, 2024
Closed
@VictorVerhaert VictorVerhaert mentioned this issue Feb 6, 2024
Merged
@VictorVerhaert VictorVerhaert linked a pull request Feb 6, 2024 that will close this issue
changed default download chunk size to 10MB #535
Merged
@VictorVerhaert
Copy link
Contributor

Created PR

VictorVerhaert added a commit that referenced this issue Feb 12, 2024
@VictorVerhaert
created constant for default download chunk size
271c8c5 
issue #528
updated changelog
VictorVerhaert added a commit that referenced this issue Feb 20, 2024
@VictorVerhaert
moved chunk_size to param in connection
ed98ea2 
small feedback points processed
issue #528
VictorVerhaert added a commit that referenced this issue Feb 20, 2024
@VictorVerhaert
added issue #528 to changelog
b7afdb0
soxofaan pushed a commit that referenced this issue Feb 21, 2024
@VictorVerhaert
Changed default download chunk size to 10MB (#528)
b7f9478
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@VictorVerhaert VictorVerhaert

Labels
feature request good first issue low hanging fruit
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

changed default download chunk size to 10MB Open-EO/openeo-python-client
2 participants
@soxofaan @VictorVerhaert