Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Too many pieces in progress with rarest strategy. #1558

Open
jimmywarting opened this issue Dec 9, 2018 · 2 comments
Open

Too many pieces in progress with rarest strategy. #1558

jimmywarting opened this issue Dec 9, 2018 · 2 comments

Comments

@jimmywarting
Copy link
Contributor

@jimmywarting jimmywarting commented Dec 9, 2018

What version of WebTorrent?
0.102.4

What browser and version? (if using WebTorrent in the browser)
Chrome

What actually happened?
I put the strategy to rarest and tried to monitor the pieces visually and came up with something that looks similar to vuzes:
skarmavbild 2018-12-09 kl 16 51 59
skarmavbild 2018-12-09 kl 16 52 12

  • Gray: Not started
  • Red: Started downloading a piece ( _chunks / _buffered )
  • green: Finish

It looks like many continuous pieces is in progress at the same time. Sometimes up to 200 out of 987 pieces are downloaded simultaneously. Each piece has a number of blocks and they don't get flushed until the piece is finish... Resulting in more memory being used.

That is rather unhealthy in the beginning if there is just one that started seeding something. You as a leacher won't be able to start sharing all uncompleted pieces until all blocks are downloaded, instead you request a few blocks from many pieces at the same time.

What did you expect to happen?
A better strategy would be something along:

  1. Let L be a list of pieces in progress
  2. Loop over the list L and let P be the piece you are iterating over
  3. If peer has that piece P in his bitfield:
    3.1 Request the next unreserved block in that piece
    3.2 If there is no more unreserved blocks left remove that piece P from the list L
  4. if the peer did not make a request select the most rarest piece available that the peer has and push that into the list L and then request the next block in that piece

this will reduce the number of pieces in progress


Another idea is: if you start downloading one block from one piece don't let that peer start on another piece until all blocks in that piece are completed.

this will reduce the number of pieces in progress but not as much. but will allow you to download multiple pieces in parallell, if one peer only have one piece it would be better to just download that from him while the other gives you other pieces so you can download something from everyone

@DiegoRBaquero

This comment has been minimized.

Copy link
Member

@DiegoRBaquero DiegoRBaquero commented Dec 10, 2018

I like this, I need free time >.<

@stale

This comment has been minimized.

Copy link

@stale stale bot commented Mar 14, 2019

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.

@stale stale bot added the stale label Mar 14, 2019
@stale stale bot closed this Mar 21, 2019
@lock lock bot locked as resolved and limited conversation to collaborators Jun 19, 2019
@webtorrent webtorrent unlocked this conversation Sep 6, 2019
@feross feross reopened this Sep 6, 2019
@stale stale bot removed the stale label Sep 6, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
3 participants
You can’t perform that action at this time.