Direct upload with FileContent is very slow (YouTube Data API v3)

[DanielRaapDev]

I used the following code to transfer videos to YouTube. Because the proxy does not support chunking we cannot use the resumable upload method (which also works fine in local tests).

File videoFile = getVideoFile();
AbstractInputStreamContent videoContent = new FileContent(mimeType, videoFile);

Videos.Insert insert = service.videos().insert(EDITED_PARTS, newVideo, videoContent);
MediaHttpUploader uploader = insert.getMediaHttpUploader();
uploader.setDirectUploadEnabled(!chunking);

insert.execute();

Now if chunking is true and therefore the resumable upload is used the upload speed is fast (2 MB/s). But if chunking is false and direct upload is used the upload is extremely slow (20 kB/s). We use java.net for transport.
So I tried to debug this problem with the system property "javax.net.debug=all". And this helped a lot! It showed that always a single byte of plain text was encrypted and 25 bytes SSL were transfered over the wire via HTTPS. So obviously no buffer is used to read the file.
My solution is to not use com.google.api.client.http.FileContent but com.google.api.client.http.InputStreamContent with a BufferedInputStream:

InputStream videoStream = new BufferedInputStream(new FileInputStream(videoFile));
AbstractInputStreamContent videoContent = new InputStreamContent(mimeType, videoStream).setLength(videoFile.length());

Would be nice if this bug can be fixed in this library.

Used versions (in pom.xml):
google-api-services-youtube v3-rev125-1.19.1
google-api-client 1.19.1
google-http-client-jackson2 1.19.0
google-http-client 1.19.0

[pmoor]

I am running into a very similar issue, but I can solve it by disabling gzip content:

NetHttpTransport transport = GoogleNetHttpTransport.newTrustedTransport();
JsonFactory jsonFactory = Utils.getDefaultJsonFactory();
GoogleCredential credential = GoogleCredential.fromStream(
    new FileInputStream("my-credentials.json"), transport, jsonFactory);
credential = credential.createScoped(
    ImmutableSet.of("https://www.googleapis.com/auth/devstorage.read_write"));
Storage client = new Storage.Builder(transport, jsonFactory, credential).build();

byte[] arr = new byte[4 << 10];
new Random().nextBytes(arr);
InputStream is = new ByteArrayInputStream(arr);

Storage.Objects.Insert method = client.objects().insert(
    "my-test-bucket",
    new StorageObject().setName("test/file/somewhere.bin"),
    new InputStreamContent("application/octet-stream", is));
method.setDisableGZipContent(false /* fast with true */);
method.getMediaHttpUploader().setProgressListener(new MediaHttpUploaderProgressListener() {
  @Override public void progressChanged(MediaHttpUploader uploader) throws IOException {
    System.out.printf("progress: %s / %d\n",
        uploader.getUploadState(), uploader.getNumBytesUploaded());
  }
});
method.execute();

When disabling gzip this runs instantaneously, when gzipping is enabled I observe the same thing with -Djavax.net.debug=all: 1-byte plaintext reads, encrypted into 30 byte long chunks, which causes my uplink to saturate at home while making tiny progress at the application level.

Versions in use:
google-api-client 1.19.1
google-http-client-jackson2 1.19.0
google-http-client 1.19.0
google-api-services-storage v1-rev28-1.19.1

[GiovanniFrigo]

I have a similar issue, but in my case using setDisableGZipContent does NOT solve the issue.

InputStreamContent mediaContent = new InputStreamContent("video/*",
        new BufferedInputStream(new URL(contentURL).openStream()));

// Insert the video
YouTube.Videos.Insert videoInsert = youtube.videos()
        .insert("snippet,status,id", videoObjectDefiningMetadata, mediaContent);

// Set the upload type and add an event listener.
MediaHttpUploader uploader = videoInsert.getMediaHttpUploader();

// Indicates that the request will use the resumable media upload 
// protocol, which supports the ability to resume an upload 
// operation after a network interruption or other transmission 
// failure, saving time and bandwidth in the event of network 
// failures.
uploader.setDirectUploadEnabled(false);
uploader.setChunkSize(MediaHttpUploader.MINIMUM_CHUNK_SIZE);
uploader.setDisableGZipContent(true);
uploader.setProgressListener(this);

// Call the API and upload the video.
com.google.api.services.youtube.model.Video returnedVideo = videoInsert.execute();

This uploaded 30 times the size of the original video. Same thing if I set uploader.setDisableGZipContent to false.

[mattwhisenhunt]

This issue may no longer be relevant due to its age. Feel free to re-open.

[lolpany]

its still relevant.
setting any chunk size or enabling/disabling gzip want help.