Web seed download problem (206 Partial Content)

[yanshekki]

What version of WebTorrent?
0.98.14

What operating system and Node.js version?
mac

What browser and version? (if using WebTorrent in the browser)
chrome 56.0.2924.87

What did you expect to happen?
solve 206 Partial Content download problem

What actually happened?
I doing the https server for web seed, but I find a problem that when I direct push the mp4 link to browser this can play specify time, but when I use of web seed that have some wrong:
https://www.extratown.com/public/2017/03/30/螢幕快照-2017-03-30-下午3.59.16.png

below is my nodejs https server code for 206 Partial Content, the same logic was worked in php long time.

            var file = basedir + decodeURIComponent (req.url);
            var mimetype = mime.lookup(file);
            var filename = path.basename(file);
            if (fs.existsSync(file) == true){
                var stats = fs.statSync(file);
                var begin = 0;
                var endin = parseInt(stats.size) - 1;
                if (typeof req.headers.range !== 'undefined'){
                    var range = req.headers.range.match(/bytes=\h*(\d+)-(\d*)[\D.*]?/i);
                    if (typeof range[1] === 'undefined' && typeof range[2] === 'undefined'){
                        res.status(404).send('<h1>404 error</h1>').end();
                    }else{
                        var rangestart = (typeof range[1] !== 'undefined') ? parseInt(range[1]) : 0;
                        var rangeend = (typeof range[2] !== 'undefined') ? parseInt(range[2]) : 0;
                        rangeend = (rangeend > 0) ? rangeend : endin;
                        if (rangestart > rangeend){
                            res.status(404).send('<h1>404 error</h1>').end();
                        }else{
                            res.status(206);
                            res.set('Content-Type', mimetype);
                            res.set('Pragma', 'no-cache');
                            res.set('Accept-Ranges', rangestart + '-' + rangeend);
                            var filelength = (rangeend - rangestart) + 1;
                            res.set('Content-Length', filelength);
                            res.set('Content-Disposition', 'inline; filename=' + filename);
                            res.set('Last-Modified', stats.mtime.toUTCString());
                            res.set('Content-Range', 'bytes ' + rangestart + '-' + rangeend + '/' + stats.size);
                            var filestream = fs.createReadStream(file, {
                                start: rangestart,
                                end: rangeend
                            });
                            filestream.pipe(res);
                        }
                    }
                }

[DiegoRBaquero]

Whats the actual problem/error? I couldn't understand.

[feross]

Yeah, I also don't understand what the issue is that you're reporting.

Also, side note: why don't you use a module to serve static files with the correct range headers? You're re-inventing the wheel.

I personally use https://www.npmjs.com/package/ecstatic or https://www.npmjs.com/package/serve-static (this one is built into express as express.static)

[yanshekki]

Maybe I use the video explain the problem:
http://www.extratown.com/public/2017/03/31/206.m4v

At the webtorrent download is over the file size 9633%, and the player can not play the video.
When I direct use the mp4 link to the browser is work.

I will try you recommended module.

thank you for your help.

[yanshekki]

I tried to use node-ecstatic and serve-static that problem still exist.

[feross]

Are you sure that the torrent file you're adding is actually for Sintel? Is the torrent file actually for the same version of Sintel? The info hash that appears on Instant.io after you dragged that torrent file does not match the hash for the sintel.mp4 file that is used for testing on webtorrent.io.

If you are pointing to an http web seed that does not match the torrent file, you could get an error like this.

[yanshekki]

Hi,

1, I am sure the mp4 file is same, because I just use your sample mp4 to testing.
2, Yes, I just have one file to test.

I tested that in case on the browser use webtorrent to start the seed and then torrent.addWebSeed is no problem, but when I use the create-torrent create the torrent and add urlList as the same time that go wrong.

below is my create-torrent code

        var path = basedir + '/' + dir + '/';
        var torrentpath = basedir + '/' + dir + '.torrent';
        if (fs.existsSync(path) === true){
            var url = 'https://' + config.ip + ':' + config.port + '/' + dir + '/';
            var webseed = [];
            fs.readdir(path, (err, files) => {
                files.forEach(file => {
                    var encodedfile = encodeURIComponent (file);
                    webseed.push(url + encodedfile);
                });
            })
            var foldername = dir.substr(dir.lastIndexOf('/') + 1);
            var trackerserver = config['trackerserver'].toString();
            var tracker = trackerserver.split("\n");
            var server = [];
            for (var k in tracker){
                var arr = [];
                arr.push(tracker[k]);
                server.push(arr);
            }
            var path = basedir + '/' + dir;
            createTorrent(path, {
                name: foldername,
                comment: config['name'],
                createdBy: config['name'],
                private: true,
                announceList: server,
                urlList: webseed
            }, function (err, torrent){
                fs.writeFile(torrentpath, torrent);
            });
        }

[feross]

Your code is incorrect. You're calling an async function fs.readdir but expecting it to run synchronously. So when you call createTorrent, webseed is still an empty array. Try logging the value of webseed right before the createTorrent() call to see for yourself. Try using fs.readdirSync.

[yanshekki]

I changed the code to use readdirSync and double confirm the "webseed" is not empty that console.log(webseed) before createTorrent it had the download link included in array, but the problem is still exist.

[feross]

@yanshekki Can you post a copy of the full code you're using to generate the torrent file, as well as a link to download the torrent file?

[yanshekki]

in the create seed code:

    var dir = 'file/home/786d6fcfa55192740003d27ad6d776fe04d6dc5284a0c83f65e08e59f5cbd23b83a9d666180a93dd078d04fc4763f75123a38265f346865aeb4ab3d96ace3483';
    var config = {
            ip: 192.168.1.199,
            port: 8443,
            trackerserver: 'ws://www.extratown.com:8080\nwss://www.extratown.com:8443\nudp://www.extratown.com:8080\nhttp://www.extratown.com:8080/announce',
            name: 'Human'
    }
    const os = require('os');
    const fs = require('fs');
    const fse = require('fs-extra');
    const createTorrent = require('create-torrent');
    var basedir = os.homedir() + '/extratown';
    var path = basedir + '/' + dir + '/';
    var torrentpath = basedir + '/' + dir + '.torrent';
    if (fs.existsSync(path) === true){
        var url = 'https://' + config.ip + ':' + config.port + '/' + dir + '/';
        var webseed = [];
        var files = fs.readdirSync(path);
        files.forEach(file => {
            var encodedfile = encodeURIComponent (file);
            webseed.push(url + encodedfile);
        });
        var foldername = dir.substr(dir.lastIndexOf('/') + 1);
        var trackerserver = config['trackerserver'].toString();
        var tracker = trackerserver.split("\n");
        var server = [];
        for (var k in tracker){
            var arr = [];
            arr.push(tracker[k]);
            server.push(arr);
        }
        var path = basedir + '/' + dir;
        createTorrent(path, {
            name: foldername,
            comment: config['name'],
            createdBy: config['name'],
            private: true,
            announceList: server,
            urlList: webseed
        }, function (err, torrent){
            fs.writeFile(torrentpath, torrent);
        });
    }

This is the created torrent file:
http://www.extratown.com/public/2017/04/07/786d6fcfa55192740003d27ad6d776fe04d6dc5284a0c83f65e08e59f5cbd23b83a9d666180a93dd078d04fc4763f75123a38265f346865aeb4ab3d96ace3483.torrent

If you need more details, I can record video for all step and console.log all value show to you.
thank you for your help

[feross]

It looks like you're adding a web seed for each file in the torrent. That's not how web seeds work.

You are supposed to specify a URL that ends in /, and the torrent client will append the file name.

So, you would just specify https://192.168.1.199:8443/file/.../ as the web seed. Then the torrent client will request https://192.168.1.199:8443/file/.../sintel.mp4. This is the easiest way to specify a web seed.

For single-file torrents you can just specify the full path to the file, but it's better to just specify a path to the folder so you can do the same thing for single file and multi file torrents. Less special cases.

I suggest re-reading BEP19 to ensure you fully understand it: http://www.bittorrent.org/beps/bep_0019.html

[yanshekki]

Hi,

Thank you for your help, but i still have some problem, I changed the code:

New code:

    var path = basedir + '/' + dir + '/';
    var torrentpath = basedir + '/' + dir + '.torrent';
    if (fs.existsSync(path) === true){
        var foldername = dir.substr(dir.lastIndexOf('/') + 1);
        var url = 'https://' + config.ip + ':' + config.port + '/' + dir + '/';
        var webseed = [];
        var files = fs.readdirSync(path);
        if (files.length == 1){
            path = path + files[0];
            url = url + files[0];
            foldername = files[0];
        }
        var trackerserver = config.trackerserver.toString();
        var tracker = trackerserver.split("\n");
        var server = [];
        for (var k in tracker){
            var arr = [];
            arr.push(tracker[k]);
            server.push(arr);
        }
        console.log(path);
        console.log({
            name: foldername,
            comment: config.name,
            createdBy: config.name,
            private: true,
            announceList: server,
            urlList: url
        });
        createTorrent(path, {
            name: foldername,
            announceList: server,
            urlList: url
        }, function (err, torrent){
            fs.writeFile(torrentpath, torrent);
        });

Below is multi files in createTorrent(path, opts), the result is no any download and reactive:

path: /Users/mac/extratown/file/a7a18ae8511466ef5619f919964096cafd33fd70469a32d2ec29dc0b9f48b12e554dd1aade48bed14eb103249f1c4ffc17f327e215a76677bb8708488bae402e/
opts: { name: 'a7a18ae8511466ef5619f919964096cafd33fd70469a32d2ec29dc0b9f48b12e554dd1aade48bed14eb103249f1c4ffc17f327e215a76677bb8708488bae402e',
comment: 'Humandecsdc',
createdBy: 'Humandecsdc',
private: true,
announceList:
[ [ 'ws://www.extratown.com:8080' ],
[ 'wss://www.extratown.com:8443' ],
[ 'udp://www.extratown.com:8080' ],
[ 'http://www.extratown.com:8080/announce' ] ],
urlList: 'https://192.168.1.199:8443/file/a7a18ae8511466ef5619f919964096cafd33fd70469a32d2ec29dc0b9f48b12e554dd1aade48bed14eb103249f1c4ffc17f327e215a76677bb8708488bae402e/' }

Below is single file in createTorrent(path, opts), the result is can download 100% but the file is not complete, when I use vuze bittorrent client or instant.io:

path: /Users/mac/extratown/file/efa6f663f17a291099a99bc08b470372d84e2ec4a32df37df7849e20cd69cd5b5ee5f7947bc8348dd15099244ccceeb3cc950b00ba0e9c4890a46856e38778d6/sintel.mp4
opts: { name: 'sintel.mp4',
comment: 'Humandecsdc',
createdBy: 'Humandecsdc',
private: true,
announceList:
[ [ 'ws://www.extratown.com:8080' ],
[ 'wss://www.extratown.com:8443' ],
[ 'udp://www.extratown.com:8080' ],
[ 'http://www.extratown.com:8080/announce' ] ],
urlList: 'https://192.168.1.199:8443/file/efa6f663f17a291099a99bc08b470372d84e2ec4a32df37df7849e20cd69cd5b5ee5f7947bc8348dd15099244ccceeb3cc950b00ba0e9c4890a46856e38778d6/sintel.mp4' }

[yanshekki]

Hi,

I find the single file createTorrent maybe fix, but the multi files i am still study.

[yanshekki]

Hi,

All problem is fix, this is my mistake. coz the file is not finish copy and create the torrent. Thank you for your help.

[feross]

No problem, glad you got your issue figured out.

[lock]

This thread has been automatically locked because it has not had recent activity. Please open a new issue for related bugs and link to relevant comments in this thread.