storage: createReadStream to support start,end?

[cristiano-belloni]

As I see in the code, File.createReadStream does not support arguments:

https://github.com/GoogleCloudPlatform/gcloud-node/blob/master/lib/storage/file.js#L229

Could it be possible for it to support options like the fs implementation, and particularly start and end? It would be very useful in ranged requests.

[stephenplusplus]

We wanted to use the method name for the sake of consistency with that API (who wants to remember "fs.createReadStream", "this.createReadable", "that.getReadStream", etc), but it wasn't our intentions to provide parity with the native method. The Storage API only lets us download the file in its entirety (as far as I know), so while we could implement something like start and end offset, it might be a better implementation from your application.

I spent some time looking for a module that would mute a stream outside of a byte range, but couldn't find anything. If you know of one, please let me know! In the meantime, I made what hopefully is a generic solution for any stream where you would want to consume only a portion of the data: https://github.com/stephenplusplus/range-stream. Using it here would look something like:

var rangeStream = require("range-stream")

myBucket.file("file.txt").createReadStream().pipe(rangeStream(50, 100))

If the other guys on this project would like to see this implemented here, it is possible, but I think it's better to keep it in application-land.

[cristiano-belloni]

Thank you for range-stream! Little useful utility that will avoid everyone re inventing the [square] wheel.

As for keeping it in the application land, it makes sense, but it might still be confusing providing an homonym method that doesn't expose the same interface. On the other hand, I understand and agree with the reasons why you chose that name.

If the API doesn't allow to stream ranges, for me this issue is closed. Btw, I'll leave it open for now to allow other people to share their opinion. If you deem necessary to close it, please go ahead.

Thanks,
Cristiano.

[silvolu]

The Storage API only lets us download the file in its entirety (as far as I know),

We could use the range header to download a byte range.
This is how the Download class of apitools does it.

[stephenplusplus]

This API has many tricks up its sleeve! Would you like to see this implemented here? I assume we'd have to shut off the data integrity checks when we're snipping out only a byte range.

[cristiano-belloni]

Imho data integrity is not so important in ranged request cases (mainly on the fly streaming), but start, end would be so useful when, for example, a video or audio element request a small range in a big stored file - this will avoid the application downloading the entire file only to stream a tiny part of it.

[stephenplusplus]

That sounds like a common enough use case, PR sent: #327

[ryanseys]

Fixed in #327