Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

upload memory issues #86

Closed
surlac opened this issue Dec 16, 2016 · 3 comments
Closed

upload memory issues #86

surlac opened this issue Dec 16, 2016 · 3 comments

Comments

@surlac
Copy link

surlac commented Dec 16, 2016

I wrote simple app to test upload with dropbox-core-sdk 2.1.2.
It opens stream from a file located at Dropbox
InputStream fileContentStream = getClient().files().download(path).getInputStream();
and passes it to upload method (within same account, but should be same even for different accounts):

UploadBuilder builder = getClient().files().uploadBuilder(newPath);
builder.uploadAndFinish(fileContentStream);

The problem is memory consumption rapidly increases from 5MB to 192MB within 10 s., then app crashes:

I/art: Clamp target GC heap from 207MB to 192MB
I/art: HomogeneousSpaceCompact marksweep + semispace GC freed 3(96B) AllocSpace objects, 0(0B) LOS objects, 0% free, 191MB/192MB, paused 624.881ms total 624.881ms
W/art: Throwing OutOfMemoryError "Failed to allocate a 12 byte allocation with 3680 free bytes and 3KB until OOM

File size is 550MB. Allocation trace showing DbxUploadStyleBuilder.uploadAndFinish():92 is the main contributor; it breaks down to com.android.okhttp.okio.SegmentPool.take():46 which is allocating numerous byte[] chunks 12288 each until crash.

Question: am I using the dropbox SDK not properly for uploading of large files or it's an issue in SDK?

@greg-db
Copy link
Contributor

greg-db commented Dec 16, 2016

Thanks for writing this up! We'll look into it for you.

@surlac
Copy link
Author

surlac commented Dec 17, 2016

This issue is reproducible even when uploading a large file (tried 131MB) stored locally on Android device.

try(InputStream is = new FileInputStream("/data/data/org.test/cache/large.file")) {
    client.files().uploadBuilder("/large.file").uploadAndFinish(is);
}

Let me know if any other info is needed.

@zjiuyang
Copy link
Contributor

zjiuyang commented Jan 9, 2017

Hi @rsharifu , there is no guarantee that uploadAndFinish won't load the whole file into memory. To upload a large file on mobile devices we strongly suggest you to split the file into chunks then upload with start/append/finish method.

Here is the example.

@zjiuyang zjiuyang closed this as completed Jan 9, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants