Join GitHub today
Improve memory efficiency of base64-encoded byte array decoding operations #967
Currently, decoding base64-encoded byte array with the Java API client is very inefficient in terms of memory usage.
Reading big attachments from the GMail API in memory-constrained environments like App Engine is quite a challenge because of that.
On a 64 bit Java 7 VM, it required at least 115 MB of heap (-Xmx115M) to read a 12Mb attachment from the Gmail API (which is a 17MB base64 string in the response from the API).
The code to test it is dead simple (MESSAGE_ID / ATTACHMENT_ID references a big attachment):
gmail.users().messages().attachments().get("me", MESSAGE_ID, ATTACHMENT_ID).execute()
Here is the stacktrace when trying to load the attachment with only 110MB or heap:
Obtaining a stream from the data field in the response would help keep the memory usage lower.
Use a manual parsing code:
And InputStreamJsonField is just a custom InputStream wrapper that ignores the first JSON chars and delegates to