Implement file.on('data'

[retrohacker]

https://github.com/Storj/storj.js#fileondata-function-cbdata-

Will require a PR to core, currently planned but not implemented.

[rockydd]

The getFile() function will download the file in backend before it can generate the blob URL, right? Is this the only way to download file? I want to make Browser to make the download and track the progress. Now it's not possible, correct?

[nginnever]

We currently have two methods for getting a file from farmers via the browser.

1. As you mention you can use the File class to wait for the file to be decrypted and stored in memory before using the blob url.

2. Use the low-level api to get a direct read stream of the decryption stream. https://github.com/Storj/storj.js#var-stream--storjdownloadfileid-bucketid

What we don't have is a standard way to track file upload/download progress. This requires a decision to be made on what constitutes progress since we have a few stages of streaming that we could listen to data events on.

[retrohacker]

Yup, exactly what @nginnever said.

The challenge is that we are using a large data structure that handles most of the upload/download for us. We (currently) don't get access to the data coming over the wire -- i.e. when we receive bits, when those bits are validated as correct, when they are decrypted, etc.-- which means we can't really deliver a reliable metric for file.progress. Not to mention the underlying data structure often returns the entire file in one big 'data' event so progress would jump from 0 to 1 instantly anyways, which is the current behaviour.

The changes we will need to make for this to work are rather large, but planned. Once we get a reliable file.progress, we can implement something along the lines of file.speed (name TBD) which computes how much data is moving over the wire over a unit of time.

Right now this is a lower priority than supporting large files. The size of the file we can reasonable support in the browser without those changes is pretty small. Transfers often take <1 second, while the handling of the data -- decrypting and concatenating streams -- takes quite a bit longer. progress doesn't have as much meaning until we remove that bottleneck.

[RichardLitt]

👋 Hey! Thanks for this contribution. Apologies for the delay in responding!

We've decided to rearchitect Storj, so that we can scale better. You can read more about this decision here. This means that we are entirely focused on v3 at the moment, in the storj/storj repository. Our white paper for v3 is coming very, very soon - follow along on the blog and in our Rocketchat.

As this repository is part of the v2 network, we're no longer maintaining this repository. I am going to close this for now. If you have any questions, I encourage you to jump on Rocketchat and ask them there. Thanks!