Customize files fragments checking, when torrent is added

[BarushevEA]

Please help me solve a problem.

What version of WebTorrent? - 0.102.4
What operating system and Node.js version? - OS Win 10 64, Node.js - v10.11.0

What actually happened?
Situation by step:

1. Add torrent "test" with many files
2. Some files are 100% downloaded, some files are still downloading
3. Stop and close the application
4. Start application
5. Add torrent "test" again
6. Before torrent is ready, webtorrent checks fragments and this process takes a lot of time

What did you expect to happen?
How i can stop files check, if they are 100% downloaded?

[SilentBot1]

Hi BarushevEA, There is a option on torrents called `skipVerify` which will mark all pieces in a torrent as downloaded. Note that depending how the torrent is added, the metadata will still be needed before the torrent will function normally so saving the .torrent or `parse-torrent` output and recreating the torrent using this is ideal instead of relying on other peers for metadata incase you're the only client. Kind Regards Brad

…

On Fri, 23 Nov 2018, 13:35 BarushevEA ***@***.*** wrote: Please help me solve a problem. *What version of WebTorrent?* - 0.102.4 *What operating system and Node.js version?* - OS Win 10 64, Node.js - v10.11.0 *What actually happened?* Situation by step: 1. Add torrent "test" with many files 2. Some files are 100% downloaded, some files are still downloading 3. Stop and close the application 4. Start application 5. Add torrent "test" again 6. Before torrent is ready, webtorrent checks fragments and this process takes a lot of time *What did you expect to happen?* How i can stop files check, if they are 100% downloaded? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#1553>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AOXi4Q0n6o-qt1kQdFAJuR0qMGZ8XNEoks5ux_mXgaJpZM4Ywqw-> .

[BarushevEA]

Thanks, I will try to use it.

[BarushevEA]

This works for the whole torrent.
But i need to disable scanning selectively, separately for each file.

[BarushevEA]

May be need to create new options/function similar to this:

  _markAllVerified () {
    for (let index = 0; index < this.pieces.length; index++) {
      this._markVerified(index)
    }
  }

But with options for example:

  _markAllSelected (start, stop) {
    // validation start
    // validation stop
    for (let index = start; index < stop; index++) {
      this._markVerified(index)
    }
  }

[SilentBot1]

Hey BarushevEA,

As it stands, like you've said, there is currently no way to do it selectively for each file, only per torrent.

Creating a selectiveVerify torrent option and additional if statement inside this else statement seems like the best way to implement something like this.

[BarushevEA]

Ok.
Thank you very much for your answers!
Are you planning to introduce the ability to selectively disable the scan?

[SilentBot1]

Something possibly like this, which uses parse-numeric-range to select the files to be marked as verified, similar to how I handled select only (BEP53) above.

if(this.selectiveVerify){
  const selectiveVerifyFiles = parseRange(this.selectiveVerify)
  this.files.forEach((v, i)=>{
    if(selectiveVerifyFiles.includes(i)){
      for(let index = v._startPiece; index <= v._endPiece; index++){
        this._markVerified(index)
      }
    }
  })
}

[BarushevEA]

Yes, it is very similar to what I need.

[SilentBot1]

@BarushevEA It could be implemented and I see no reason as to why it wouldn't be as it's quite useful in combination with select only.

I don't have the time at the moment to implement it properly myself, mainly writing the tests for it, but the code above should be functional.

Yourself or anybody else is more than welcome to create the PR to add this feature, sorry I'm not able to do it myself at the moment.

[BarushevEA]

I understand.
I starting this at next week.
I can implement this for my project and try to help you too.
If you do not mind.

[BarushevEA]

I am try to use new functionality, but found error flow.
Files really do not verify, but after this torrent never ready.

I am write code:
to class Torrent

constructor add
this.skipVerifyFiles = !this.skipVerify ? opts.skipVerifyFiles : null;

rewvrite _markAllVerified()

    _markAllVerified() {
        this._markVerifiedGroup(0, this.pieces.length - 1);
    }

    _markVerifiedGroup(start, end) {
        for (let i = start; i <= end; i++) {
            this._markVerified(i);
        }
    }

add function

    /**
     *  Pieces of file mark as verified
     *  verefiedFiles contained object with filenames.
     *  For example - {filename: "filename", filename1: "filename1"}
     */
    _markFileVerified(verefiedFiles) {
        this.files.forEach(file => {
            if (verefiedFiles[file.name]) {
                this._markVerifiedGroup(file._startPiece, file._endPiece);
            }
        });
    }

add to _onMetadata(metadata)

        if (this.skipVerifyFiles) {
            this._markFileVerified(this.skipVerifyFiles);
        }

Use options
{path: downloadPath, skipVerifyFiles: verifiedFiles}

[BarushevEA]

I guess I found a mistake.

Function - _verifyPieces()
File - torrent.js, strings 576 - 578:

                        if (!this.pieces[index]) return
                        this._debug('piece verified %s', index)
                        this._markVerified(index)

Fix:

                        if (this.pieces[index]) {
                            this._debug('piece verified %s', index);
                            this._markVerified(index);
                        }

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.