Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

better feedback when chunk size is adjusted #291

Open
burritojustice opened this issue Jan 5, 2021 · 0 comments
Open

better feedback when chunk size is adjusted #291

burritojustice opened this issue Jan 5, 2021 · 0 comments

Comments

@burritojustice
Copy link
Contributor

During upload, if a chunk is too large for the API gateway, we cut it in half and try again (and if it's still too large, we cut it in half and try again (and if…)))

While this is quite robust (assuming there isn't a single feature that is larger than the size of the API gateway), we should give more meaningful feedback when we get to a chunk size that does work:

here xyz upload ee9Ap2Hw -f /Users/joram/Downloads/LROC_5TO20KM_CRATERS_0TO90W_180.geojson,/Users/joram/Downloads/LROC_5TO20KM_CRATERS_0TO90E_180.geojson,/Users/joram/Downloads/LROC_5TO20KM_CRATERS_90ETO180E_180.geojson,/Users/joram/Downloads/LROC_5TO20KM_CRATERS_90WTO180W_180.geojson -s
uploading file - /Users/joram/Downloads/LROC_5TO20KM_CRATERS_0TO90W_180.geojson

uploading chunk size of 200 features failed with 413 Request Entity too large error, trying upload again with smaller chunk of 100

uploading chunk size of 200 features failed with 413 Request Entity too large error, trying upload again with smaller chunk of 100

uploading chunk size of 100 features failed with 413 Request Entity too large error, trying upload again with smaller chunk of 50

uploading chunk size of 200 features failed with 413 Request Entity too large error, trying upload again with smaller chunk of 100

uploading chunk size of 50 features failed with 413 Request Entity too large error, trying upload again with smaller chunk of 25

uploading chunk size of 100 features failed with 413 Request Entity too large error, trying upload again with smaller chunk of 50

uploading chunk size of 100 features failed with 413 Request Entity too large error, trying upload again with smaller chunk of 50

uploading chunk size of 200 features failed with 413 Request Entity too large error, trying upload again with smaller chunk of 100

While it's clear the chunk size is being adjusted, it's not clear what and when actually got uploaded. We should fix that.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant