storage - readStream hangs on long running time pipe

[tomgrossman]

Environment details

• OS: Ubuntu 16.04
• Node.js version: 6.11.2
• npm version: 3.10.10
• google-cloud-node version: @google-cloud/storage@1.2.1

Some more details:
The file size I'm reproducing the issue on is ~2.3 gb.
The issue is happening only when using pipe and only when the download speed is below 5 mb\s.
When the download speed is faster or when using .on('data', (data) => res.write(data)), there is no problem.

Steps to reproduce

1. require @google-cloud/storage
2. use the following route function, where bucketFile is a pure file instance from the storage lib:

function DownloadOva (req, res) {
    console.log(new Date() + ' start');

    res.on('finish', function () {
        console.log(new Date() + ' finish');
    });

    let bucketFile = OvaFileHelpers.GetBucketFile('agent_ova/agent.ova');
    bucketFile.createReadStream({validation: false})
        .on('error', (err) => {
            console.log(new Date() + ' error: ' + err.stack);
        })
        .on('response', (streamResponse) => {
            console.log(new Date() + ' streamResponse');
            res.setHeader('Content-Length', streamResponse.headers['content-length']);
            res.setHeader('Content-Type', streamResponse.headers['content-type']);
            res.setHeader('Content-disposition', 'attachment; filename="agent_download.ova"');
        })
        .on('end', () => {
            console.log(new Date() + ' end');
            res.end();

            return true;
        })
        // .on('data', (data) => {
        //     Response.write(data);
        // })
        .pipe(res);
}

3. limit the download speed in chrome using the throttling option in the network developer tools to 5000 kb/s
4. call the route

Expected results:
all console logs outputs should be written to console.
Actual results:
The file is downloaded successfully
the 'end' and 'finish' events are never called (no console log outputs and doesn't get into the breakpoint).
Can't call the route again until restarting the application

more details:
When the download speed is not limited, it works fine (on high speed connection). but some users with low speed connections, faced this issue (that's why I reproduced this issue).
When using the commented .on('data' ... and commenting out the pipe, it also works fine also on slow download.

It seems that when the pipe takes too long (seems like 3-4 minutes), it looses all the event listening functions and gets hangs even though the download finish successfully on the client side.

Thanks!

[stephenplusplus]

Thank you for reporting. Could you please provide a clonable repo that can reproduce the bug? Since there are other factors at play to load from the browser, I'll need more dependencies and other code to replicate. It's best that I'm using the same dependencies and code that you are.

[tomgrossman]

Hi @stephenplusplus , Yes I'll be happy to provide what ever is possible from my side in order to resolve this issue.
I created a simple public repo that should help reproduce the bug:
https://github.com/tomgrossman/storage-download.git

In this code, you will need to change the projectId, bucket, fileName and the json key path in order to download a file from your storage (I can't give you mine since it's personal. if you will provide me with a free storage just for this issue, I'll be happy to upload something there in order to download the exact same file).
I have added in the repo also some screenshots of the issue that will help you reproduce it:

1. bucket_file - details about the file I'm downloading (large size)
2. throttling_profile.jpg - the throttling details I'm using in chrome in order to reproduce the issue
3. during_download.jpg - so you can see the download speed
4. download_finished.jpg - so you can see that the download finished successfully
5. no_log.jpg - no console logging that the download has finished.

Again, If I'm using my top speed or using the .on('data') instead of pipe, it works fine.

[stephenplusplus]

Thank you very much! I will look ASAP (likely Monday).

[tomgrossman]

@stephenplusplus It seems that this issue is reproducing also with the .on('data'...) function as well and not only with pipe.
In addition, I want to clarify why I think it's a "google" issue.
I tested the exact same file and same speed but from local path using the fs.createReadStream() function and it works fine.
So I think it has to do with the stream that you are creating

[stephenplusplus]

Things are looking good. I should have a PR shortly.

[tomgrossman]

Nice. Can you give more details about what you found?

[stephenplusplus]

PR sent in #2581 - please give it a shot when you can.

[tomgrossman]

@stephenplusplus looks OK from a simple test :)

[lukesneeringer]

This issue was moved to googleapis/nodejs-storage#24