Better large file handling

[feross]

Large files fail with RangeError (in Chrome)

[spikebike]

Define large? Just tried a 600MB ISO with no problem. It does seem to assume that all UDP ports are open to the internet, it seems to try a random port each time.

[feross]

I made some changes and this is no longer an issue! I'd still like performance to be better, but this issue can be closed!

[feross]

Actually, this is still broken. I get an error when actually trying to download the file. Something like "Error: No File" (in chrome). However, the data is being transferred, so it's an error creating the final blob and url. Will investigate soon.

[FesterCluck]

You should see how FileSaver.js handled this:

https://github.com/eligrey/FileSaver.js/blob/master/FileSaver.js#L52

and their reasoning:

https://code.google.com/p/chromium/issues/detail?id=375297#c7

[feross]

@FesterCluck thanks will check it out

[feross]

Haven't seen RangeError in a while, but large downloads seem to stall part-way through (around when the tab hits 2GB ram?)

[FesterCluck]

You should check to see if you are hitting your machine's paging limit.

[feross]

For a 105MB file, the page uses 540MB of memory. By commenting out bits of the code at a time, I found this breakdown:

• 90MB just for loading instant.io in chrome (incl. my browser extensions)
• 210MB from drag-drop/buffer
• 105MB from create-torrent in webtorrent.seed
• 105MB from file.getBlobURL in onTorrent on instant.io

Checked off = fixed.

We're creating lots of buffers that don't get released, it seems. Ideally, we only need to hang onto one copy of the buffer.

[feross]

There are still some leaks, but it doesn't look like we're losing any full copies of the file anymore! Progress! There's only one copy of the file in memory.

For a 360MB file:

• 90 MB for base instant.io page
• 360 MB for file
• 160 MB for webtorrent.seed (and leaks?)

cc @jhiesey

[feross]

We're now running into a bug in Chrome where blobs can't be larger than 500MB. https://code.google.com/p/chromium/issues/detail?id=375297 This looks like it's really close to being fixed though! Right now, this means you can't seed a file larger than 500MB from Chrome.

I was able to seed a 1GB file in Firefox and stream to Chrome!

[feross]

_This issue was moved to webtorrent/webtorrent#319._