Please provide streaming for upload large content

[Vampire]

I have a Gradle build where I build a Docker image tar file which has a size of 166 MiB.
I use the GitHub publish Gradle plugin and tried to upload this Docker image as asset to the release.
This consistently resulted in an OutOfMemoryError.

The GitHub publish Gradle plugin under the hood uses this library.
And I also easily reproduced it stand-alone.
In version 1.1xx, you use an HttpsUrlConnection without setting it to fixed-length streaming or chunking mode and thus the JRE uses a byte-array output stream to colllect the whole body in RAM to determine the size up-front.
In version 1.3xx many things changed, but now you use your GitHubRequest.Builder#with(java.io.InputStream) method to load the whole file into RAM using IOUtils.toByteArray.

So at different places both load the whole file into RAM instead of properly streaming it.
Please support at least giving the size to methods like uploadAsset so that the size can be set as HTTP header but the content be streamed, or if GitHub supports it maybe even using chunked mode additionally if no explicit size was given.

The problem can pretty easily be reproduced using:

github
    .getRepository("my/repository")
    .listReleases()
    .iterator()
    .next()
    .uploadAsset("foo.tar.gz", new FileInputStream("file/that/is/too/large/for/RAM"), "application/octet-stream")

[bitwiseman]

see #1405.

[Vampire]

But that's a different issue, isn't it? There it is about receiving something and according to you it needs significant complexity to be fixed. Here it is about sending an artifact where you (at least in the 1.1xx version) just need to enable streaming mode and eventually let the user specify the file size or the file instead of just a stream. Should be pretty trivial to fix this one, shouldn't it?

[bitwiseman]

@Vampire Sorry for not responding sooner. If you are able to create a PR enabling the behavior you want, I'm open to discussing.

[bitwiseman]

I can see how they are different path but the underlying framework currently treats them the same. I'll keep this open separately as I can see ways that the upload issue could be resolved separately from the download.

[bitwiseman]

@Vampire
The download issue (#1405) is fixed in the latest release.

@atsushieno
If you have have any suggestions on how to fix the upload side, you help would be much appreciated.

[atsushieno]

@atsushieno If you have have any suggestions on how to fix the upload side, you help would be much appreciated.

I don't use this asset uploading feature so I would just point out code line that is still problematic:

At

github-api/src/main/java/org/kohsuke/github/extras/okhttp3/OkHttpGitHubConnector.java

Line 87 in 2274f27

body = RequestBody.create(IOUtils.toByteArray(request.body()));

:

            body = RequestBody.create(IOUtils.toByteArray(request.body()));

This would retrieve the entire request body content to turn it into a byte array, so it will result in the same problem as #1405.